Apache Flink 1.10.2 Released

25 Aug 2020 Zhu Zhu (@zhuzhv)

The Apache Flink community released the second bugfix version of the Apache Flink 1.10 series.

This release includes 73 fixes and minor improvements for Flink 1.10.1. The list below includes a detailed list of all fixes and improvements.

We highly recommend all users to upgrade to Flink 1.10.2.

Note After FLINK-18242, the deprecated OptionsFactory and ConfigurableOptionsFactory classes are removed (not applicable for release-1.10), please use RocksDBOptionsFactory and ConfigurableRocksDBOptionsFactory instead. Please also recompile your application codes if any class extending DefaultConfigurableOptionsFactory

Note After FLINK-17800 by default we will set setTotalOrderSeek to true for RocksDB’s ReadOptions, to prevent user from miss using optimizeForPointLookup. Meantime we support customizing ReadOptions through RocksDBOptionsFactory. Please set setTotalOrderSeek back to false if any performance regression observed (normally won’t happen according to our testing).

Updated Maven dependencies:

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-java</artifactId>
  <version>1.10.2</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-streaming-java_2.11</artifactId>
  <version>1.10.2</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-clients_2.11</artifactId>
  <version>1.10.2</version>
</dependency>

You can find the binaries on the updated Downloads page.

List of resolved issues:

Sub-task

  • [FLINK-15836] - Throw fatal error in KubernetesResourceManager when the pods watcher is closed with exception
  • [FLINK-16160] - Schema#proctime and Schema#rowtime don't work in TableEnvironment#connect code path

Bug

  • [FLINK-13689] - Rest High Level Client for Elasticsearch6.x connector leaks threads if no connection could be established
  • [FLINK-14369] - KafkaProducerAtLeastOnceITCase>KafkaProducerTestBase.testOneToOneAtLeastOnceCustomOperator fails on Travis
  • [FLINK-14836] - Unable to set yarn container number for scala shell in yarn mode
  • [FLINK-14894] - HybridOffHeapUnsafeMemorySegmentTest#testByteBufferWrap failed on Travis
  • [FLINK-15758] - Investigate potential out-of-memory problems due to managed unsafe memory allocation
  • [FLINK-15849] - Update SQL-CLIENT document from type to data-type
  • [FLINK-16309] - ElasticSearch 7 connector is missing in SQL connector list
  • [FLINK-16346] - BlobsCleanupITCase.testBlobServerCleanupCancelledJob fails on Travis
  • [FLINK-16432] - Building Hive connector gives problems
  • [FLINK-16451] - Fix IndexOutOfBoundsException for DISTINCT AGG with constants
  • [FLINK-16510] - Task manager safeguard shutdown may not be reliable
  • [FLINK-17092] - Pyflink test BlinkStreamDependencyTests is instable
  • [FLINK-17322] - Enable latency tracker would corrupt the broadcast state
  • [FLINK-17420] - Cannot alias Tuple and Row fields when converting DataStream to Table
  • [FLINK-17466] - toRetractStream doesn't work correctly with Pojo conversion class
  • [FLINK-17555] - docstring of pyflink.table.descriptors.FileSystem:1:duplicate object description of pyflink.table.descriptors.FileSystem
  • [FLINK-17558] - Partitions are released in TaskExecutor Main Thread
  • [FLINK-17562] - POST /jars/:jarid/plan is not working
  • [FLINK-17578] - Union of 2 SideOutputs behaviour incorrect
  • [FLINK-17639] - Document which FileSystems are supported by the StreamingFileSink
  • [FLINK-17643] - LaunchCoordinatorTest fails
  • [FLINK-17700] - The callback client of JavaGatewayServer should run in a daemon thread
  • [FLINK-17744] - StreamContextEnvironment#execute cannot be call JobListener#onJobExecuted
  • [FLINK-17763] - No log files when starting scala-shell
  • [FLINK-17788] - scala shell in yarn mode is broken
  • [FLINK-17800] - RocksDB optimizeForPointLookup results in missing time windows
  • [FLINK-17801] - TaskExecutorTest.testHeartbeatTimeoutWithResourceManager timeout
  • [FLINK-17809] - BashJavaUtil script logic does not work for paths with spaces
  • [FLINK-17822] - Nightly Flink CLI end-to-end test failed with "JavaGcCleanerWrapper$PendingCleanersRunner cannot access class jdk.internal.misc.SharedSecrets" in Java 11
  • [FLINK-17870] - dependent jars are missing to be shipped to cluster in scala shell
  • [FLINK-17891] - FlinkYarnSessionCli sets wrong execution.target type
  • [FLINK-17959] - Exception: "CANCELLED: call already cancelled" is thrown when run python udf
  • [FLINK-18008] - HistoryServer does not log environment information on startup
  • [FLINK-18012] - Deactivate slot timeout if TaskSlotTable.tryMarkSlotActive is called
  • [FLINK-18035] - Executors#newCachedThreadPool could not work as expected
  • [FLINK-18045] - Fix Kerberos credentials checking to unblock Flink on secured MapR
  • [FLINK-18048] - "--host" option could not take effect for standalone application cluster
  • [FLINK-18097] - History server doesn't clean all job json files
  • [FLINK-18168] - Error results when use UDAF with Object Array return type
  • [FLINK-18223] - AvroSerializer does not correctly instantiate GenericRecord
  • [FLINK-18241] - Custom OptionsFactory in user code not working when configured via flink-conf.yaml
  • [FLINK-18242] - Custom OptionsFactory settings seem to have no effect on RocksDB
  • [FLINK-18297] - SQL client: setting execution.type to invalid value shuts down the session
  • [FLINK-18329] - Dist NOTICE issues
  • [FLINK-18352] - org.apache.flink.core.execution.DefaultExecutorServiceLoader not thread safe
  • [FLINK-18517] - kubernetes session test failed with "java.net.SocketException: Broken pipe"
  • [FLINK-18539] - StreamExecutionEnvironment#addSource(SourceFunction, TypeInformation) doesn't use the user defined type information
  • [FLINK-18595] - Deadlock during job shutdown
  • [FLINK-18646] - Managed memory released check can block RPC thread
  • [FLINK-18663] - RestServerEndpoint may prevent server shutdown
  • [FLINK-18677] - ZooKeeperLeaderRetrievalService does not invalidate leader in case of SUSPENDED connection
  • [FLINK-18702] - Flink elasticsearch connector leaks threads and classloaders thereof
  • [FLINK-18815] - AbstractCloseableRegistryTest.testClose unstable
  • [FLINK-18821] - Netty client retry mechanism may cause PartitionRequestClientFactory#createPartitionRequestClient to wait infinitely
  • [FLINK-18859] - ExecutionGraphNotEnoughResourceTest.testRestartWithSlotSharingAndNotEnoughResources failed with "Condition was not met in given timeout."
  • [FLINK-18902] - Cannot serve results of asynchronous REST operations in per-job mode

New Feature

  • [FLINK-17844] - Activate japicmp-maven-plugin checks for @PublicEvolving between bug fix releases (x.y.u -> x.y.v)

Improvement

  • [FLINK-16217] - SQL Client crashed when any uncatched exception is thrown
  • [FLINK-16225] - Metaspace Out Of Memory should be handled as Fatal Error in TaskManager
  • [FLINK-16619] - Misleading SlotManagerImpl logging for slot reports of unknown task manager
  • [FLINK-16717] - Use headless service for rpc and blob port when flink on K8S
  • [FLINK-17248] - Make the thread nums of io executor of ClusterEntrypoint and MiniCluster configurable
  • [FLINK-17503] - Make memory configuration logging more user-friendly
  • [FLINK-17819] - Yarn error unhelpful when forgetting HADOOP_CLASSPATH
  • [FLINK-17920] - Add the Python example of Interval Join in Table API doc
  • [FLINK-17945] - Improve error reporting of Python CI tests
  • [FLINK-17970] - Increase default value of IO pool executor to 4 * #cores
  • [FLINK-18010] - Add more logging to HistoryServer
  • [FLINK-18501] - Mapping of Pluggable Filesystems to scheme is not properly logged
  • [FLINK-18644] - Remove obsolete doc for hive connector
  • [FLINK-18772] - Hide submit job web ui elements when running in per-job/application mode