The Apache Flink community released the first bugfix version of the Apache Flink 1.12 series.
This release includes 79 fixes and minor improvements for Flink 1.12.0. The list below includes a detailed list of all fixes and improvements.
We highly recommend all users to upgrade to Flink 1.12.1.
This can happen if a new checkpoint is triggered before recovery is fully completed. For state to be corrupted a task with two or more input gates must receive a checkpoint barrier exactly at the same time this tasks finishes recovering spilled in-flight data. In such case this new checkpoint can succeed, with corrupted/missing in-flight data, which will result in various deserialisation/corrupted data stream errors when someone attempts to recover from such corrupted checkpoint.
Using unaligned checkpoints in Flink 1.12.1, a corruption may occur in the checkpoint following a declined checkpoint.
A late barrier of a canceled checkpoint may lead to buffers being not written into the successive checkpoint, such that recovery is not possible. This happens, when the next checkpoint barrier arrives at a given operator before all previous barriers arrived, which can only happen after cancellation in unaligned checkpoints.
Updated Maven dependencies:
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.12.1</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.12.1</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.12.1</version> </dependency>
You can find the binaries on the updated Downloads page.
List of resolved issues:
- [FLINK-18897] - Add documentation for the maxwell-json format
- [FLINK-20352] - Rework command line interface documentation page
- [FLINK-20353] - Rework logging documentation page
- [FLINK-20354] - Rework standalone deployment documentation page
- [FLINK-20355] - Rework K8s deployment documentation page
- [FLINK-20356] - Rework Mesos deployment documentation page
- [FLINK-20422] - Remove from .html files in flink documentation
- [FLINK-20485] - Map views are deserialized multiple times
- [FLINK-20601] - Rework PyFlink CLI documentation
- [FLINK-19369] - BlobClientTest.testGetFailsDuringStreamingForJobPermanentBlob hangs
- [FLINK-19435] - Deadlock when loading different driver classes concurrently using Class.forName
- [FLINK-19725] - Logger cannot be initialized due to timeout: LoggerInitializationException is thrown
- [FLINK-19880] - Fix ignore-parse-errors not work for the legacy JSON format
- [FLINK-20213] - Partition commit is delayed when records keep coming
- [FLINK-20221] - DelimitedInputFormat does not restore compressed filesplits correctly leading to dataloss
- [FLINK-20273] - Fix Table api Kafka connector Sink Partitioner Document Error
- [FLINK-20372] - Update Kafka SQL connector page to mention properties.* options
- [FLINK-20389] - UnalignedCheckpointITCase failure caused by NullPointerException
- [FLINK-20404] - ZooKeeper quorum fails to start due to missing log4j library
- [FLINK-20419] - Insert fails due to failure to generate execution plan
- [FLINK-20428] - ZooKeeperLeaderElectionConnectionHandlingTest.testConnectionSuspendedHandlingDuringInitialization failed with "No result is expected since there was no leader elected before stopping the server, yet"
- [FLINK-20429] - KafkaTableITCase.testKafkaTemporalJoinChangelog failed with unexpected results
- [FLINK-20433] - UnalignedCheckpointTestBase.execute failed with "TestTimedOutException: test timed out after 300 seconds"
- [FLINK-20464] - Some Table examples are not built correctly
- [FLINK-20467] - Fix the Example in Python DataStream Doc
- [FLINK-20470] - MissingNode can't be casted to ObjectNode when deserializing JSON
- [FLINK-20476] - New File Sink end-to-end test Failed
- [FLINK-20486] - Hive temporal join should allow monitor interval smaller than 1 hour
- [FLINK-20492] - The SourceOperatorStreamTask should implement cancelTask() and finishTask()
- [FLINK-20493] - SQLClientSchemaRegistryITCase failed with "Could not build the flink-dist image"
- [FLINK-20521] - Null result values are being swallowed by RPC system
- [FLINK-20525] - StreamArrowPythonGroupWindowAggregateFunctionOperator doesn't handle rowtime and proctime properly
- [FLINK-20543] - Fix typo in upsert kafka docs
- [FLINK-20554] - The Checkpointed Data Size of the Latest Completed Checkpoint is incorrectly displayed on the Overview page of the UI
- [FLINK-20582] - Fix typos in `CREATE Statements` docs
- [FLINK-20607] - a wrong example in udfs page.
- [FLINK-20615] - Local recovery and sticky scheduling end-to-end test timeout with "IOException: Stream Closed"
- [FLINK-20626] - Canceling a job when it is failing will result in job hanging in CANCELING state
- [FLINK-20630] - [Kinesis][DynamoDB] DynamoDB Streams Consumer fails to consume from Latest
- [FLINK-20646] - ReduceTransformation does not work with RocksDBStateBackend
- [FLINK-20648] - Unable to restore job from savepoint when using Kubernetes based HA services
- [FLINK-20664] - Support setting service account for TaskManager pod
- [FLINK-20665] - FileNotFoundException when restore from latest Checkpoint
- [FLINK-20666] - Fix the deserialized Row losing the field_name information in PyFlink
- [FLINK-20669] - Add the jzlib LICENSE file in flink-python module
- [FLINK-20703] - HiveSinkCompactionITCase test timeout
- [FLINK-20704] - Some rel data type does not implement the digest correctly
- [FLINK-20756] - PythonCalcSplitConditionRule is not working as expected
- [FLINK-20764] - BatchGroupedReduceOperator does not emit results for singleton inputs
- [FLINK-20781] - UnalignedCheckpointITCase failure caused by NullPointerException
- [FLINK-20784] - .staging_xxx does not exist, when insert into hive
- [FLINK-20793] - Fix NamesTest due to code style refactor
- [FLINK-20803] - Version mismatch between spotless-maven-plugin and google-java-format plugin
- [FLINK-20841] - Fix compile error due to duplicated generated files
- [FLINK-19013] - Log start/end of state restoration
- [FLINK-19259] - Use classloader release hooks with Kinesis producer to avoid metaspace leak
- [FLINK-19832] - Improve handling of immediately failed physical slot in SlotSharingExecutionSlotAllocator
- [FLINK-20055] - Datadog API Key exposed in Flink JobManager logs
- [FLINK-20168] - Translate page 'Flink Architecture' into Chinese
- [FLINK-20209] - Add missing checkpoint configuration to Flink UI
- [FLINK-20298] - Replace usage of in flink documentation
- [FLINK-20468] - Enable leadership control in MiniCluster to test JM failover
- [FLINK-20510] - Enable log4j2 monitor interval by default
- [FLINK-20519] - Extend HBase notice with transitively bundled dependencies
- [FLINK-20570] - The `NOTE` tip style is different from the others in process_function page.
- [FLINK-20588] - Add docker-compose as appendix to Mesos documentation
- [FLINK-20629] - [Kinesis][EFO] Migrate from DescribeStream to DescribeStreamSummary
- [FLINK-20647] - Use yield to generate output datas in ProcessFunction for Python DataStream
- [FLINK-20650] - Mark "native-k8s" as deprecated in docker-entrypoint.sh
- [FLINK-20651] - Use Spotless/google-java-format for code formatting/enforcement
- [FLINK-20682] - Add configuration options related to hadoop
- [FLINK-20697] - Correct the Type of "lookup.cache.ttl" in jdbc.md/jdbc.zh.md
- [FLINK-20790] - Generated classes should not be put under src/ directory
- [FLINK-20792] - Allow shorthand invocation of spotless
- [FLINK-20805] - Blink runtime classes partially ignored by spotless
- [FLINK-20822] - Don't check whether a function is generic in hive catalog
- [FLINK-20866] - Add how to list jobs in Yarn deployment documentation when HA enabled