Blog

Apache Flink 1.12.2 Released

03 Mar 2021 Yuan Mei & Roman Khachatryan

The Apache Flink community released the next bugfix version of the Apache Flink 1.12 series.

This release includes 83 fixes and minor improvements for Flink 1.12.1. The list below includes a detailed list of all fixes and improvements.

We highly recommend all users to upgrade to Flink 1.12.2.

Updated Maven dependencies:

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-java</artifactId>
  <version>1.12.2</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-streaming-java_2.11</artifactId>
  <version>1.12.2</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-clients_2.11</artifactId>
  <version>1.12.2</version>
</dependency>

You can find the binaries on the updated Downloads page.

List of resolved issues:

Sub-task

  • [FLINK-21070] - Overloaded aggregate functions cause converter errors
  • [FLINK-21486] - Add sanity check when switching from Rocks to Heap timers

Bug

  • [FLINK-12461] - Document binary compatibility situation with Scala beyond 2.12.8
  • [FLINK-16443] - Fix wrong fix for user-code CheckpointExceptions
  • [FLINK-19771] - NullPointerException when accessing null array from postgres in JDBC Connector
  • [FLINK-20309] - UnalignedCheckpointTestBase.execute is failed
  • [FLINK-20462] - MailboxOperatorTest.testAvoidTaskStarvation
  • [FLINK-20500] - UpsertKafkaTableITCase.testTemporalJoin test failed
  • [FLINK-20565] - Fix typo in EXPLAIN Statements docs.
  • [FLINK-20580] - Missing null value handling for SerializedValue's getByteArray()
  • [FLINK-20654] - Unaligned checkpoint recovery may lead to corrupted data stream
  • [FLINK-20663] - Managed memory may not be released in time when operators use managed memory frequently
  • [FLINK-20675] - Asynchronous checkpoint failure would not fail the job anymore
  • [FLINK-20680] - Fails to call var-arg function with no parameters
  • [FLINK-20798] - Using PVC as high-availability.storageDir could not work
  • [FLINK-20832] - Deliver bootstrap resouces ourselves for website and documentation
  • [FLINK-20848] - Kafka consumer ID is not specified correctly in new KafkaSource
  • [FLINK-20913] - Improve new HiveConf(jobConf, HiveConf.class)
  • [FLINK-20921] - Fix Date/Time/Timestamp in Python DataStream
  • [FLINK-20933] - Config Python Operator Use Managed Memory In Python DataStream
  • [FLINK-20942] - Digest of FLOAT literals throws UnsupportedOperationException
  • [FLINK-20944] - Launching in application mode requesting a ClusterIP rest service type results in an Exception
  • [FLINK-20947] - Idle source doesn't work when pushing watermark into the source
  • [FLINK-20961] - Flink throws NullPointerException for tables created from DataStream with no assigned timestamps and watermarks
  • [FLINK-20992] - Checkpoint cleanup can kill JobMaster
  • [FLINK-20998] - flink-raw-1.12.jar does not exist
  • [FLINK-21009] - Can not disable certain options in Elasticsearch 7 connector
  • [FLINK-21013] - Blink planner does not ingest timestamp into StreamRecord
  • [FLINK-21024] - Dynamic properties get exposed to job's main method if user parameters are passed
  • [FLINK-21028] - Streaming application didn't stop properly
  • [FLINK-21030] - Broken job restart for job with disjoint graph
  • [FLINK-21059] - KafkaSourceEnumerator does not honor consumer properties
  • [FLINK-21069] - Configuration "parallelism.default" doesn't take effect for TableEnvironment#explainSql
  • [FLINK-21071] - Snapshot branches running against flink-docker dev-master branch
  • [FLINK-21104] - UnalignedCheckpointITCase.execute failed with "IllegalStateException"
  • [FLINK-21132] - BoundedOneInput.endInput is called when taking synchronous savepoint
  • [FLINK-21138] - KvStateServerHandler is not invoked with user code classloader
  • [FLINK-21140] - Extract zip file dependencies before adding to PYTHONPATH
  • [FLINK-21144] - KubernetesResourceManagerDriver#tryResetPodCreationCoolDown causes fatal error
  • [FLINK-21155] - FileSourceTextLinesITCase.testBoundedTextFileSourceWithTaskManagerFailover does not pass
  • [FLINK-21158] - wrong jvm metaspace and overhead size show in taskmanager metric page
  • [FLINK-21163] - Python dependencies specified via CLI should not override the dependencies specified in configuration
  • [FLINK-21169] - Kafka flink-connector-base dependency should be scope compile
  • [FLINK-21208] - pyarrow exception when using window with pandas udaf
  • [FLINK-21213] - e2e test fail with 'As task is already not running, no longer decline checkpoint'
  • [FLINK-21215] - Checkpoint was declined because one input stream is finished
  • [FLINK-21216] - StreamPandasConversionTests Fails
  • [FLINK-21225] - OverConvertRule does not consider distinct
  • [FLINK-21226] - Reintroduce TableColumn.of for backwards compatibility
  • [FLINK-21274] - At per-job mode, during the exit of the JobManager process, if ioExecutor exits at the end, the System.exit() method will not be executed.
  • [FLINK-21277] - SQLClientSchemaRegistryITCase fails to download testcontainers/ryuk:0.3.0
  • [FLINK-21312] - SavepointITCase.testStopSavepointWithBoundedInputConcurrently is unstable
  • [FLINK-21323] - Stop-with-savepoint is not supported by SourceOperatorStreamTask
  • [FLINK-21351] - Incremental checkpoint data would be lost once a non-stop savepoint completed
  • [FLINK-21361] - FlinkRelMdUniqueKeys matches on AbstractCatalogTable instead of CatalogTable
  • [FLINK-21412] - pyflink DataTypes.DECIMAL is not available
  • [FLINK-21452] - FLIP-27 sources cannot reliably downscale
  • [FLINK-21453] - BoundedOneInput.endInput is NOT called when doing stop with savepoint WITH drain
  • [FLINK-21490] - UnalignedCheckpointITCase fails on azure
  • [FLINK-21492] - ActiveResourceManager swallows exception stack trace

New Feature

  • [FLINK-20359] - Support adding Owner Reference to Job Manager in native kubernetes setup

Improvement

  • [FLINK-9844] - PackagedProgram does not close URLClassLoader
  • [FLINK-20417] - Handle "Too old resource version" exception in Kubernetes watch more gracefully
  • [FLINK-20491] - Support Broadcast Operation in BATCH execution mode
  • [FLINK-20517] - Support mixed keyed/non-keyed operations in BATCH execution mode
  • [FLINK-20770] - Incorrect description for config option kubernetes.rest-service.exposed.type
  • [FLINK-20907] - Table API documentation promotes deprecated syntax
  • [FLINK-21020] - Bump Jackson to 20.10.5[.1] / 2.12.1
  • [FLINK-21034] - Rework jemalloc switch to use an environment variable
  • [FLINK-21035] - Deduplicate copy_plugins_if_required calls
  • [FLINK-21036] - Consider removing automatic configuration fo number of slots from docker
  • [FLINK-21037] - Deduplicate configuration logic in docker entrypoint
  • [FLINK-21042] - Fix code example in "Aggregate Functions" section in Table UDF page
  • [FLINK-21048] - Refactor documentation related to switch memory allocator
  • [FLINK-21123] - Upgrade Beanutils 1.9.x to 1.9.4
  • [FLINK-21164] - Jar handlers don't cleanup temporarily extracted jars
  • [FLINK-21210] - ApplicationClusterEntryPoints should explicitly close PackagedProgram
  • [FLINK-21381] - Kubernetes HA documentation does not state required service account and role

Task

  • [FLINK-20529] - Publish Dockerfiles for release 1.12.0
  • [FLINK-20534] - Add Flink 1.12 MigrationVersion
  • [FLINK-20536] - Update migration tests in master to cover migration from release-1.12
  • [FLINK-20960] - Add warning in 1.12 release notes about potential corrupt data stream with unaligned checkpoint
  • [FLINK-21358] - Missing snapshot version compatibility for 1.12

Continue reading »

How to natively deploy Flink on Kubernetes with High-Availability (HA)

10 Feb 2021 Yang Wang

Kubernetes provides built-in functionalities that Flink can leverage for JobManager failover. In Flink 1.12 (FLIP-144), the community implemented a Kubernetes High Availability (HA) service as an alternative to ZooKeeper for highly available production setups. In this blogpost, we will have a close look at how to deploy Flink applications natively on Kubernetes cluster with HA.

Continue reading »

Apache Flink 1.10.3 Released

29 Jan 2021 Xintong Song

The Apache Flink community released the third bugfix version of the Apache Flink 1.10 series.

This release includes 36 fixes and minor improvements for Flink 1.10.2. The list below includes a detailed list of all fixes and improvements.

We highly recommend all users to upgrade to Flink 1.10.3.

Updated Maven dependencies:

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-java</artifactId>
  <version>1.10.3</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-streaming-java_2.11</artifactId>
  <version>1.10.3</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-clients_2.11</artifactId>
  <version>1.10.3</version>
</dependency>

You can find the binaries on the updated Downloads page.

List of resolved issues:

Bug

  • [FLINK-14087] - throws java.lang.ArrayIndexOutOfBoundsException when emiting the data using RebalancePartitioner.
  • [FLINK-15170] - WebFrontendITCase.testCancelYarn fails on travis
  • [FLINK-15467] - Should wait for the end of the source thread during the Task cancellation
  • [FLINK-16246] - Exclude "SdkMBeanRegistrySupport" from dynamically loaded AWS connectors
  • [FLINK-17341] - freeSlot in TaskExecutor.closeJobManagerConnection cause ConcurrentModificationException
  • [FLINK-17458] - TaskExecutorSubmissionTest#testFailingScheduleOrUpdateConsumers
  • [FLINK-17677] - FLINK_LOG_PREFIX recommended in docs is not always available
  • [FLINK-18081] - Fix broken links in "Kerberos Authentication Setup and Configuration" doc
  • [FLINK-18196] - flink throws `NullPointerException` when executeCheckpointing
  • [FLINK-18212] - Init lookup join failed when use udf on lookup table
  • [FLINK-18832] - BoundedBlockingSubpartition does not work with StreamTask
  • [FLINK-18959] - Fail to archiveExecutionGraph because job is not finished when dispatcher close
  • [FLINK-19022] - AkkaRpcActor failed to start but no exception information
  • [FLINK-19109] - Split Reader eats chained periodic watermarks
  • [FLINK-19135] - (Stream)ExecutionEnvironment.execute() should not throw ExecutionException
  • [FLINK-19237] - LeaderChangeClusterComponentsTest.testReelectionOfJobMaster failed with "NoResourceAvailableException: Could not allocate the required slot within slot request timeout"
  • [FLINK-19401] - Job stuck in restart loop due to excessive checkpoint recoveries which block the JobMaster
  • [FLINK-19557] - Issue retrieving leader after zookeeper session reconnect
  • [FLINK-19675] - The plan of is incorrect when Calc contains WHERE clause, composite fields access and Python UDF at the same time
  • [FLINK-19901] - Unable to exclude metrics variables for the last metrics reporter.
  • [FLINK-20013] - BoundedBlockingSubpartition may leak network buffer if task is failed or canceled
  • [FLINK-20018] - pipeline.cached-files option cannot escape ':' in path
  • [FLINK-20033] - Job fails when stopping JobMaster
  • [FLINK-20065] - UnalignedCheckpointCompatibilityITCase.test failed with AskTimeoutException
  • [FLINK-20076] - DispatcherTest.testOnRemovedJobGraphDoesNotCleanUpHAFiles does not test the desired functionality
  • [FLINK-20183] - Fix the default PYTHONPATH is overwritten in client side
  • [FLINK-20218] - AttributeError: module 'urllib' has no attribute 'parse'
  • [FLINK-20875] - [CVE-2020-17518] Directory traversal attack: remote file writing through the REST API

Improvement

  • [FLINK-16753] - Exception from AsyncCheckpointRunnable should be wrapped in CheckpointException
  • [FLINK-18287] - Correct the documentation of Python Table API in SQL pages
  • [FLINK-19055] - MemoryManagerSharedResourcesTest contains three tests running extraordinary long
  • [FLINK-19105] - Table API Sample Code Error
  • [FLINK-19252] - Jaas file created under io.tmp.dirs - folder not created if not exists
  • [FLINK-19339] - Support Avro's unions with logical types
  • [FLINK-19523] - Hide sensitive command-line configurations

Task

  • [FLINK-20906] - Update copyright year to 2021 for NOTICE files

Continue reading »

Apache Flink 1.12.1 Released

19 Jan 2021 Xintong Song

The Apache Flink community released the first bugfix version of the Apache Flink 1.12 series.

This release includes 79 fixes and minor improvements for Flink 1.12.0. The list below includes a detailed list of all fixes and improvements.

We highly recommend all users to upgrade to Flink 1.12.1.

Attention: Using unaligned checkpoints in Flink 1.12.0 combined with two/multiple inputs tasks or with union inputs for single input tasks can result in corrupted state.

This can happen if a new checkpoint is triggered before recovery is fully completed. For state to be corrupted a task with two or more input gates must receive a checkpoint barrier exactly at the same time this tasks finishes recovering spilled in-flight data. In such case this new checkpoint can succeed, with corrupted/missing in-flight data, which will result in various deserialisation/corrupted data stream errors when someone attempts to recover from such corrupted checkpoint.

Using unaligned checkpoints in Flink 1.12.1, a corruption may occur in the checkpoint following a declined checkpoint.

A late barrier of a canceled checkpoint may lead to buffers being not written into the successive checkpoint, such that recovery is not possible. This happens, when the next checkpoint barrier arrives at a given operator before all previous barriers arrived, which can only happen after cancellation in unaligned checkpoints.

Updated Maven dependencies:

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-java</artifactId>
  <version>1.12.1</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-streaming-java_2.11</artifactId>
  <version>1.12.1</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-clients_2.11</artifactId>
  <version>1.12.1</version>
</dependency>

You can find the binaries on the updated Downloads page.

List of resolved issues:

Sub-task

  • [FLINK-18897] - Add documentation for the maxwell-json format
  • [FLINK-20352] - Rework command line interface documentation page
  • [FLINK-20353] - Rework logging documentation page
  • [FLINK-20354] - Rework standalone deployment documentation page
  • [FLINK-20355] - Rework K8s deployment documentation page
  • [FLINK-20356] - Rework Mesos deployment documentation page
  • [FLINK-20422] - Remove from .html files in flink documentation
  • [FLINK-20485] - Map views are deserialized multiple times
  • [FLINK-20601] - Rework PyFlink CLI documentation

Bug

  • [FLINK-19369] - BlobClientTest.testGetFailsDuringStreamingForJobPermanentBlob hangs
  • [FLINK-19435] - Deadlock when loading different driver classes concurrently using Class.forName
  • [FLINK-19725] - Logger cannot be initialized due to timeout: LoggerInitializationException is thrown
  • [FLINK-19880] - Fix ignore-parse-errors not work for the legacy JSON format
  • [FLINK-20213] - Partition commit is delayed when records keep coming
  • [FLINK-20221] - DelimitedInputFormat does not restore compressed filesplits correctly leading to dataloss
  • [FLINK-20273] - Fix Table api Kafka connector Sink Partitioner Document Error
  • [FLINK-20372] - Update Kafka SQL connector page to mention properties.* options
  • [FLINK-20389] - UnalignedCheckpointITCase failure caused by NullPointerException
  • [FLINK-20404] - ZooKeeper quorum fails to start due to missing log4j library
  • [FLINK-20419] - Insert fails due to failure to generate execution plan
  • [FLINK-20428] - ZooKeeperLeaderElectionConnectionHandlingTest.testConnectionSuspendedHandlingDuringInitialization failed with "No result is expected since there was no leader elected before stopping the server, yet"
  • [FLINK-20429] - KafkaTableITCase.testKafkaTemporalJoinChangelog failed with unexpected results
  • [FLINK-20433] - UnalignedCheckpointTestBase.execute failed with "TestTimedOutException: test timed out after 300 seconds"
  • [FLINK-20464] - Some Table examples are not built correctly
  • [FLINK-20467] - Fix the Example in Python DataStream Doc
  • [FLINK-20470] - MissingNode can't be casted to ObjectNode when deserializing JSON
  • [FLINK-20476] - New File Sink end-to-end test Failed
  • [FLINK-20486] - Hive temporal join should allow monitor interval smaller than 1 hour
  • [FLINK-20492] - The SourceOperatorStreamTask should implement cancelTask() and finishTask()
  • [FLINK-20493] - SQLClientSchemaRegistryITCase failed with "Could not build the flink-dist image"
  • [FLINK-20521] - Null result values are being swallowed by RPC system
  • [FLINK-20525] - StreamArrowPythonGroupWindowAggregateFunctionOperator doesn't handle rowtime and proctime properly
  • [FLINK-20543] - Fix typo in upsert kafka docs
  • [FLINK-20554] - The Checkpointed Data Size of the Latest Completed Checkpoint is incorrectly displayed on the Overview page of the UI
  • [FLINK-20582] - Fix typos in `CREATE Statements` docs
  • [FLINK-20607] - a wrong example in udfs page.
  • [FLINK-20615] - Local recovery and sticky scheduling end-to-end test timeout with "IOException: Stream Closed"
  • [FLINK-20626] - Canceling a job when it is failing will result in job hanging in CANCELING state
  • [FLINK-20630] - [Kinesis][DynamoDB] DynamoDB Streams Consumer fails to consume from Latest
  • [FLINK-20646] - ReduceTransformation does not work with RocksDBStateBackend
  • [FLINK-20648] - Unable to restore job from savepoint when using Kubernetes based HA services
  • [FLINK-20664] - Support setting service account for TaskManager pod
  • [FLINK-20665] - FileNotFoundException when restore from latest Checkpoint
  • [FLINK-20666] - Fix the deserialized Row losing the field_name information in PyFlink
  • [FLINK-20669] - Add the jzlib LICENSE file in flink-python module
  • [FLINK-20703] - HiveSinkCompactionITCase test timeout
  • [FLINK-20704] - Some rel data type does not implement the digest correctly
  • [FLINK-20756] - PythonCalcSplitConditionRule is not working as expected
  • [FLINK-20764] - BatchGroupedReduceOperator does not emit results for singleton inputs
  • [FLINK-20781] - UnalignedCheckpointITCase failure caused by NullPointerException
  • [FLINK-20784] - .staging_xxx does not exist, when insert into hive
  • [FLINK-20793] - Fix NamesTest due to code style refactor
  • [FLINK-20803] - Version mismatch between spotless-maven-plugin and google-java-format plugin
  • [FLINK-20841] - Fix compile error due to duplicated generated files

Improvement

  • [FLINK-19013] - Log start/end of state restoration
  • [FLINK-19259] - Use classloader release hooks with Kinesis producer to avoid metaspace leak
  • [FLINK-19832] - Improve handling of immediately failed physical slot in SlotSharingExecutionSlotAllocator
  • [FLINK-20055] - Datadog API Key exposed in Flink JobManager logs
  • [FLINK-20168] - Translate page 'Flink Architecture' into Chinese
  • [FLINK-20209] - Add missing checkpoint configuration to Flink UI
  • [FLINK-20298] - Replace usage of in flink documentation
  • [FLINK-20468] - Enable leadership control in MiniCluster to test JM failover
  • [FLINK-20510] - Enable log4j2 monitor interval by default
  • [FLINK-20519] - Extend HBase notice with transitively bundled dependencies
  • [FLINK-20570] - The `NOTE` tip style is different from the others in process_function page.
  • [FLINK-20588] - Add docker-compose as appendix to Mesos documentation
  • [FLINK-20629] - [Kinesis][EFO] Migrate from DescribeStream to DescribeStreamSummary
  • [FLINK-20647] - Use yield to generate output datas in ProcessFunction for Python DataStream
  • [FLINK-20650] - Mark "native-k8s" as deprecated in docker-entrypoint.sh
  • [FLINK-20651] - Use Spotless/google-java-format for code formatting/enforcement
  • [FLINK-20682] - Add configuration options related to hadoop
  • [FLINK-20697] - Correct the Type of "lookup.cache.ttl" in jdbc.md/jdbc.zh.md
  • [FLINK-20790] - Generated classes should not be put under src/ directory
  • [FLINK-20792] - Allow shorthand invocation of spotless
  • [FLINK-20805] - Blink runtime classes partially ignored by spotless
  • [FLINK-20822] - Don't check whether a function is generic in hive catalog
  • [FLINK-20866] - Add how to list jobs in Yarn deployment documentation when HA enabled

Task

  • [FLINK-20300] - Create Flink 1.12 release notes
  • [FLINK-20906] - Update copyright year to 2021 for NOTICE files

Continue reading »

What's New in the Pulsar Flink Connector 2.7.0

07 Jan 2021 Jianyun Zhao (@yihy8023) & Jennifer Huang (@Jennife06125739)

With the unification of batch and streaming regarded as the future in data processing, the Pulsar Flink Connector provides an ideal solution for unified batch and stream processing with Apache Pulsar and Apache Flink. The Pulsar Flink Connector 2.7.0 supports features in Pulsar 2.7 and Flink 1.12 and is fully compatible with Flink's data format. The Pulsar Flink Connector 2.7.0 will be contributed to the Flink repository soon and the contribution process is ongoing.

Continue reading »

Stateful Functions 2.2.2 Release Announcement

02 Jan 2021 Tzu-Li (Gordon) Tai (@tzulitai)

The Apache Flink community released the second bugfix release of the Stateful Functions (StateFun) 2.2 series, version 2.2.2.

The most important change of this bugfix release is upgrading Apache Flink to version 1.11.3. In addition to many stability fixes to the Flink runtime itself, this also allows StateFun applications to safely use savepoints to upgrade from older versions earlier than StateFun 2.2.1. Previously, restoring from savepoints could have failed under certain conditions.

We strongly recommend all users to upgrade to 2.2.2.

You can find the binaries on the updated Downloads page.

This release includes 5 fixes and minor improvements since StateFun 2.2.1. Below is a detailed list of all fixes and improvements:

Improvement

  • [FLINK-20699] - Feedback invocation_id must not be constant.

Task

  • [FLINK-20161] - Consider switching from Travis CI to Github Actions for flink-statefun's CI workflows
  • [FLINK-20189] - Restored feedback events may be silently dropped if per key-group header bytes were not fully read
  • [FLINK-20636] - Require unaligned checkpoints to be disabled in StateFun applications
  • [FLINK-20689] - Upgrade StateFun to Flink 1.11.3

Continue reading »

Apache Flink 1.11.3 Released

18 Dec 2020 Xintong Song

The Apache Flink community released the third bugfix version of the Apache Flink 1.11 series.

This release includes 151 fixes and minor improvements for Flink 1.11.2. The list below includes a detailed list of all fixes and improvements.

We highly recommend all users to upgrade to Flink 1.11.3.

Updated Maven dependencies:

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-java</artifactId>
  <version>1.11.3</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-streaming-java_2.11</artifactId>
  <version>1.11.3</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-clients_2.11</artifactId>
  <version>1.11.3</version>
</dependency>

You can find the binaries on the updated Downloads page.

List of resolved issues:

Sub-task

  • [FLINK-17393] - Improve the `FutureCompletingBlockingQueue` to wakeup blocking put() more elegantly.
  • [FLINK-18604] - HBase ConnectorDescriptor can not work in Table API
  • [FLINK-18673] - Calling ROW() in a UDF results in UnsupportedOperationException
  • [FLINK-18680] - Improve RecordsWithSplitIds API
  • [FLINK-18916] - Add a "Operations" link(linked to dev/table/tableApi.md) under the "Python API" -> "User Guide" -> "Table API" section
  • [FLINK-18918] - Add a "Connectors" document under the "Python API" -> "User Guide" -> "Table API" section
  • [FLINK-18922] - Add a "Catalogs" link (linked to dev/table/catalogs.md) under the "Python API" -> "User Guide" -> "Table API" section
  • [FLINK-18926] - Add a "Environment Variables" document under the "Python API" -> "User Guide" -> "Table API" section
  • [FLINK-19162] - Allow Split Reader based sources to reuse record batches
  • [FLINK-19205] - SourceReaderContext should give access to Configuration and Hostbame
  • [FLINK-20397] - Pass checkpointId to OperatorCoordinator.resetToCheckpoint().

Bug

  • [FLINK-9992] - FsStorageLocationReferenceTest#testEncodeAndDecode failed in CI
  • [FLINK-13733] - FlinkKafkaInternalProducerITCase.testHappyPath fails on Travis
  • [FLINK-15170] - WebFrontendITCase.testCancelYarn fails on travis
  • [FLINK-16246] - Exclude "SdkMBeanRegistrySupport" from dynamically loaded AWS connectors
  • [FLINK-16268] - Failed to run rank over window with Hive built-in functions
  • [FLINK-16768] - HadoopS3RecoverableWriterITCase.testRecoverWithStateWithMultiPart hangs
  • [FLINK-17341] - freeSlot in TaskExecutor.closeJobManagerConnection cause ConcurrentModificationException
  • [FLINK-17458] - TaskExecutorSubmissionTest#testFailingScheduleOrUpdateConsumers
  • [FLINK-17677] - FLINK_LOG_PREFIX recommended in docs is not always available
  • [FLINK-17825] - HA end-to-end gets killed due to timeout
  • [FLINK-18128] - CoordinatedSourceITCase.testMultipleSources gets stuck
  • [FLINK-18196] - flink throws `NullPointerException` when executeCheckpointing
  • [FLINK-18222] - "Avro Confluent Schema Registry nightly end-to-end test" unstable with "Kafka cluster did not start after 120 seconds"
  • [FLINK-18815] - AbstractCloseableRegistryTest.testClose unstable
  • [FLINK-18818] - HadoopRenameCommitterHDFSTest.testCommitOneFile[Override: false] failed with "java.io.IOException: The stream is closed"
  • [FLINK-18836] - Python UDTF doesn't work well when the return type isn't generator
  • [FLINK-18915] - FIXED_PATH(dummy Hadoop Path) with WriterImpl may cause ORC writer OOM
  • [FLINK-19022] - AkkaRpcActor failed to start but no exception information
  • [FLINK-19121] - Avoid accessing HDFS frequently in HiveBulkWriterFactory
  • [FLINK-19135] - (Stream)ExecutionEnvironment.execute() should not throw ExecutionException
  • [FLINK-19138] - Python UDF supports directly specifying input_types as DataTypes.ROW
  • [FLINK-19140] - Join with Table Function (UDTF) SQL example is incorrect
  • [FLINK-19151] - Flink does not normalize container resource with correct configurations when Yarn FairScheduler is used
  • [FLINK-19154] - Application mode deletes HA data in case of suspended ZooKeeper connection
  • [FLINK-19170] - Parameter naming error
  • [FLINK-19201] - PyFlink e2e tests is instable and failed with "Connection broken: OSError"
  • [FLINK-19227] - The catalog is still created after opening failed in catalog registering
  • [FLINK-19237] - LeaderChangeClusterComponentsTest.testReelectionOfJobMaster failed with "NoResourceAvailableException: Could not allocate the required slot within slot request timeout"
  • [FLINK-19244] - CSV format can't deserialize null ROW field
  • [FLINK-19250] - SplitFetcherManager does not propagate errors correctly
  • [FLINK-19253] - SourceReaderTestBase.testAddSplitToExistingFetcher hangs
  • [FLINK-19258] - Fix the wrong example of "csv.line-delimiter" in CSV documentation
  • [FLINK-19280] - The option "sink.buffer-flush.max-rows" for JDBC can't be disabled by set to zero
  • [FLINK-19281] - LIKE cannot recognize full table path
  • [FLINK-19291] - Fix exception for AvroSchemaConverter#convertToSchema when RowType contains multiple row fields
  • [FLINK-19295] - YARNSessionFIFOITCase.checkForProhibitedLogContents found a log with prohibited string
  • [FLINK-19300] - Timer loss after restoring from savepoint
  • [FLINK-19321] - CollectSinkFunction does not define serialVersionUID
  • [FLINK-19338] - New source interface cannot unregister unregistered source
  • [FLINK-19361] - Make HiveCatalog thread safe
  • [FLINK-19398] - Hive connector fails with IllegalAccessError if submitted as usercode
  • [FLINK-19401] - Job stuck in restart loop due to excessive checkpoint recoveries which block the JobMaster
  • [FLINK-19423] - Fix ArrayIndexOutOfBoundsException when executing DELETE statement in JDBC upsert sink
  • [FLINK-19433] - An Error example of FROM_UNIXTIME function in document
  • [FLINK-19448] - CoordinatedSourceITCase.testEnumeratorReaderCommunication hangs
  • [FLINK-19535] - SourceCoordinator should avoid fail job multiple times.
  • [FLINK-19557] - Issue retrieving leader after zookeeper session reconnect
  • [FLINK-19585] - UnalignedCheckpointCompatibilityITCase.test:97->runAndTakeSavepoint: "Not all required tasks are currently running."
  • [FLINK-19587] - Error result when casting binary type as varchar
  • [FLINK-19618] - Broken link in docs
  • [FLINK-19629] - Fix NullPointException when deserializing map field with null value for Avro format
  • [FLINK-19675] - The plan of is incorrect when Calc contains WHERE clause, composite fields access and Python UDF at the same time
  • [FLINK-19695] - Writing Table with RowTime Column of type TIMESTAMP(3) to Kafka fails with ClassCastException
  • [FLINK-19717] - SourceReaderBase.pollNext may return END_OF_INPUT if SplitReader.fetch throws
  • [FLINK-19740] - Error in to_pandas for table containing event time: class java.time.LocalDateTime cannot be cast to class java.sql.Timestamp
  • [FLINK-19741] - InternalTimeServiceManager fails to restore due to corrupt reads if there are other users of raw keyed state streams
  • [FLINK-19748] - KeyGroupRangeOffsets#KeyGroupOffsetsIterator should skip key groups that don't have a defined offset
  • [FLINK-19750] - Deserializer is not opened in Kafka consumer when restoring from state
  • [FLINK-19755] - Fix CEP documentation error of the example in 'After Match Strategy' section
  • [FLINK-19775] - SystemProcessingTimeServiceTest.testImmediateShutdown is instable
  • [FLINK-19777] - Fix NullPointException for WindowOperator.close()
  • [FLINK-19790] - Writing MAP<STRING, STRING> to Kafka with JSON format produces incorrect data.
  • [FLINK-19806] - Job may try to leave SUSPENDED state in ExecutionGraph#failJob()
  • [FLINK-19816] - Flink restored from a wrong checkpoint (a very old one and not the last completed one)
  • [FLINK-19852] - Managed memory released check can block IterativeTask
  • [FLINK-19867] - Validation fails for UDF that accepts var-args
  • [FLINK-19894] - Use iloc for positional slicing instead of direct slicing in from_pandas
  • [FLINK-19901] - Unable to exclude metrics variables for the last metrics reporter.
  • [FLINK-19906] - Incorrect result when compare two binary fields
  • [FLINK-19907] - Channel state (upstream) can be restored after emission of new elements (watermarks)
  • [FLINK-19909] - Flink application in attach mode could not terminate when the only job is canceled
  • [FLINK-19948] - Calling NOW() function throws compile exception
  • [FLINK-20013] - BoundedBlockingSubpartition may leak network buffer if task is failed or canceled
  • [FLINK-20018] - pipeline.cached-files option cannot escape ':' in path
  • [FLINK-20033] - Job fails when stopping JobMaster
  • [FLINK-20050] - SourceCoordinatorProviderTest.testCheckpointAndReset failed with NullPointerException
  • [FLINK-20063] - File Source requests an additional split on every restore.
  • [FLINK-20064] - Broken links in the documentation
  • [FLINK-20065] - UnalignedCheckpointCompatibilityITCase.test failed with AskTimeoutException
  • [FLINK-20068] - KafkaSubscriberTest.testTopicPatternSubscriber failed with unexpected results
  • [FLINK-20069] - docs_404_check doesn't work properly
  • [FLINK-20076] - DispatcherTest.testOnRemovedJobGraphDoesNotCleanUpHAFiles does not test the desired functionality
  • [FLINK-20079] - Modified UnalignedCheckpointITCase...MassivelyParallel fails
  • [FLINK-20081] - ExecutorNotifier should run handler in the main thread when receive an exception from the callable.
  • [FLINK-20143] - use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
  • [FLINK-20165] - YARNSessionFIFOITCase.checkForProhibitedLogContents: Error occurred during initialization of boot layer java.lang.IllegalStateException: Module system already initialized
  • [FLINK-20175] - Avro Confluent Registry SQL format does not support adding nullable columns
  • [FLINK-20183] - Fix the default PYTHONPATH is overwritten in client side
  • [FLINK-20193] - SourceCoordinator should catch exception thrown from SplitEnumerator.start()
  • [FLINK-20194] - KafkaSourceFetcherManager.commitOffsets() should handle the case when there is no split fetcher.
  • [FLINK-20200] - SQL Hints are not supported in "Create View" syntax
  • [FLINK-20213] - Partition commit is delayed when records keep coming
  • [FLINK-20221] - DelimitedInputFormat does not restore compressed filesplits correctly leading to dataloss
  • [FLINK-20222] - The CheckpointCoordinator should reset the OperatorCoordinators when fail before the first checkpoint.
  • [FLINK-20223] - The RecreateOnResetOperatorCoordinator and SourceCoordinator executor thread should use the user class loader.
  • [FLINK-20243] - Remove useless words in documents
  • [FLINK-20262] - Building flink-dist docker image does not work without python2
  • [FLINK-20266] - New Sources prevent JVM shutdown when running a job
  • [FLINK-20270] - Fix the regression of missing ExternallyInducedSource support in FLIP-27 Source.
  • [FLINK-20277] - flink-1.11.2 ContinuousFileMonitoringFunction cannot restore from failure
  • [FLINK-20284] - Error happens in TaskExecutor when closing JobMaster connection if there was a python UDF
  • [FLINK-20285] - LazyFromSourcesSchedulingStrategy is possible to schedule non-CREATED vertices
  • [FLINK-20333] - Flink standalone cluster throws metaspace OOM after submitting multiple PyFlink UDF jobs.
  • [FLINK-20351] - Execution.transitionState does not properly log slot location
  • [FLINK-20382] - Exception thrown from JobMaster.startScheduling() may be ignored.
  • [FLINK-20396] - Add "OperatorCoordinator.resetSubtask()" to fix order problems of "subtaskFailed()"
  • [FLINK-20404] - ZooKeeper quorum fails to start due to missing log4j library
  • [FLINK-20413] - Sources should add splits back in "resetSubtask()", rather than in "subtaskFailed()".
  • [FLINK-20418] - NPE in IteratorSourceReader
  • [FLINK-20442] - Fix license documentation mistakes in flink-python.jar
  • [FLINK-20492] - The SourceOperatorStreamTask should implement cancelTask() and finishTask()
  • [FLINK-20554] - The Checkpointed Data Size of the Latest Completed Checkpoint is incorrectly displayed on the Overview page of the UI

New Feature

  • [FLINK-19934] - [FLIP-27 source] add new API: SplitEnumeratorContext.runInCoordinatorThread(Runnable)

Improvement

  • [FLINK-16753] - Exception from AsyncCheckpointRunnable should be wrapped in CheckpointException
  • [FLINK-18139] - Unaligned checkpoints checks wrong channels for inflight data.
  • [FLINK-18500] - Make the legacy planner exception more clear when resolving computed columns types for schema
  • [FLINK-18545] - Sql api cannot specify flink job name
  • [FLINK-18715] - add cpu usage metric of jobmanager/taskmanager
  • [FLINK-19193] - Recommend stop-with-savepoint in upgrade guidelines
  • [FLINK-19225] - Improve code and logging in SourceReaderBase
  • [FLINK-19245] - Set default queue capacity for FLIP-27 source handover queue to 2
  • [FLINK-19251] - Avoid confusing queue handling in "SplitReader.handleSplitsChanges()"
  • [FLINK-19252] - Jaas file created under io.tmp.dirs - folder not created if not exists
  • [FLINK-19265] - Simplify handling of 'NoMoreSplitsEvent'
  • [FLINK-19339] - Support Avro's unions with logical types
  • [FLINK-19523] - Hide sensitive command-line configurations
  • [FLINK-19569] - Upgrade ICU4J to 67.1+
  • [FLINK-19677] - TaskManager takes abnormally long time to register with JobManager on Kubernetes
  • [FLINK-19698] - Add close() method and onCheckpointComplete() to the Source.
  • [FLINK-19892] - Replace __metaclass__ field with metaclass keyword
  • [FLINK-20049] - Simplify handling of "request split".
  • [FLINK-20055] - Datadog API Key exposed in Flink JobManager logs
  • [FLINK-20142] - Update the document for CREATE TABLE LIKE that source table from different catalog is supported
  • [FLINK-20152] - Document which execution.target values are supported
  • [FLINK-20156] - JavaDocs of WatermarkStrategy.withTimestampAssigner are wrong wrt Java 8
  • [FLINK-20169] - Move emitting MAX_WATERMARK out of SourceOperator processing loop
  • [FLINK-20207] - Improve the error message printed when submitting the pyflink jobs via 'flink run'
  • [FLINK-20296] - Explanation of keyBy was broken by find/replace of deprecated forms of keyBy

Test

  • [FLINK-18725] - "Run Kubernetes test" failed with "30025: provided port is already allocated"

Task

  • [FLINK-20455] - Add check to LicenseChecker for top level /LICENSE files in shaded jars

Continue reading »

  • Previous
  • Page: 1 of 15

2021

2020

2019

2018

2017

2016

2015

2014

Copyright © 2014-2019 The Apache Software Foundation. All Rights Reserved.

Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.

Privacy Policy · RSS feed