Apache Flink 1.10.1 Released

May 12, 2020 - Yu Li (@LiyuApache)

The Apache Flink community released the first bugfix version of the Apache Flink 1.10 series.

This release includes 158 fixes and minor improvements for Flink 1.10.0. The list below includes a detailed list of all fixes and improvements.

We highly recommend all users to upgrade to Flink 1.10.1.

Note FLINK-16684 changed the builders of the StreamingFileSink to make them compilable in Scala. This change is source compatible but binary incompatible. If using the StreamingFileSink, please recompile your user code against 1.10.1 before upgrading.
Note FLINK-16683 Flink no longer supports starting clusters with .bat scripts. Users should instead use environments like WSL or Cygwin and work with the .sh scripts.

Updated Maven dependencies:

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-java</artifactId>
  <version>1.10.1</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-streaming-java_2.11</artifactId>
  <version>1.10.1</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-clients_2.11</artifactId>
  <version>1.10.1</version>
</dependency>

You can find the binaries on the updated Downloads page.

List of resolved issues:

Sub-task

  • [FLINK-14126] - Elasticsearch Xpack Machine Learning doesn't support ARM
  • [FLINK-15143] - Create document for FLIP-49 TM memory model and configuration guide
  • [FLINK-15561] - Unify Kerberos credentials checking
  • [FLINK-15790] - Make FlinkKubeClient and its implementations asynchronous
  • [FLINK-15817] - Kubernetes Resource leak while deployment exception happens
  • [FLINK-16049] - Remove outdated "Best Practices" section from Application Development Section
  • [FLINK-16131] - Translate "Amazon S3" page of "File Systems" into Chinese
  • [FLINK-16389] - Bump Kafka 0.10 to 0.10.2.2

Bug

  • [FLINK-2336] - ArrayIndexOufOBoundsException in TypeExtractor when mapping
  • [FLINK-10918] - incremental Keyed state with RocksDB throws cannot create directory error in windows
  • [FLINK-11193] - Rocksdb timer service factory configuration option is not settable per job
  • [FLINK-13483] - PrestoS3FileSystemITCase.testDirectoryListing fails on Travis
  • [FLINK-14038] - ExecutionGraph deploy failed due to akka timeout
  • [FLINK-14311] - Streaming File Sink end-to-end test failed on Travis
  • [FLINK-14316] - Stuck in "Job leader ... lost leadership" error
  • [FLINK-15417] - Remove the docker volume or mount when starting Mesos e2e cluster
  • [FLINK-15669] - SQL client can't cancel flink job
  • [FLINK-15772] - Shaded Hadoop S3A with credentials provider end-to-end test fails on travis
  • [FLINK-15811] - StreamSourceOperatorWatermarksTest.testNoMaxWatermarkOnAsyncCancel fails on Travis
  • [FLINK-15812] - HistoryServer archiving is done in Dispatcher main thread
  • [FLINK-15838] - Dangling CountDownLatch.await(timeout)
  • [FLINK-15852] - Job is submitted to the wrong session cluster
  • [FLINK-15904] - Make Kafka Consumer work with activated "disableGenericTypes()"
  • [FLINK-15936] - TaskExecutorTest#testSlotAcceptance deadlocks
  • [FLINK-15953] - Job Status is hard to read for some Statuses
  • [FLINK-16013] - List and map config options could not be parsed correctly
  • [FLINK-16014] - S3 plugin ClassNotFoundException SAXParser
  • [FLINK-16025] - Service could expose blob server port mismatched with JM Container
  • [FLINK-16026] - Travis failed due to python setup
  • [FLINK-16047] - Blink planner produces wrong aggregate results with state clean up
  • [FLINK-16067] - Flink's CalciteParser swallows error position information
  • [FLINK-16068] - table with keyword-escaped columns and computed_column_expression columns
  • [FLINK-16070] - Blink planner can not extract correct unique key for UpsertStreamTableSink
  • [FLINK-16108] - StreamSQLExample is failed if running in blink planner
  • [FLINK-16111] - Kubernetes deployment does not respect "taskmanager.cpu.cores".
  • [FLINK-16113] - ExpressionReducer shouldn't escape the reduced string value
  • [FLINK-16115] - Aliyun oss filesystem could not work with plugin mechanism
  • [FLINK-16139] - Co-location constraints are not reset on task recovery in DefaultScheduler
  • [FLINK-16161] - Statistics zero should be unknown in HiveCatalog
  • [FLINK-16170] - SearchTemplateRequest ClassNotFoundException when use flink-sql-connector-elasticsearch7
  • [FLINK-16220] - JsonRowSerializationSchema throws cast exception : NullNode cannot be cast to ArrayNode
  • [FLINK-16231] - Hive connector is missing jdk.tools exclusion against Hive 2.x.x
  • [FLINK-16234] - Fix unstable cases in StreamingJobGraphGeneratorTest
  • [FLINK-16241] - Remove the license and notice file in flink-ml-lib module on release-1.10 branch
  • [FLINK-16242] - BinaryGeneric serialization error cause checkpoint failure
  • [FLINK-16262] - Class loader problem with FlinkKafkaProducer.Semantic.EXACTLY_ONCE and usrlib directory
  • [FLINK-16269] - Generic type can not be matched when convert table to stream.
  • [FLINK-16281] - parameter 'maxRetryTimes' can not work in JDBCUpsertTableSink
  • [FLINK-16301] - Annoying "Cannot find FunctionDefinition" messages with SQL for f_proctime or =
  • [FLINK-16308] - SQL connector download links are broken
  • [FLINK-16313] - flink-state-processor-api: surefire execution unstable on Azure
  • [FLINK-16331] - Remove source licenses for old WebUI
  • [FLINK-16345] - Computed column can not refer time attribute column
  • [FLINK-16360] - connector on hive 2.0.1 don't support type conversion from STRING to VARCHAR
  • [FLINK-16371] - HadoopCompressionBulkWriter fails with 'java.io.NotSerializableException'
  • [FLINK-16373] - EmbeddedLeaderService: IllegalStateException: The RPC connection is already closed
  • [FLINK-16413] - Reduce hive source parallelism when limit push down
  • [FLINK-16414] - create udaf/udtf function using sql casuing ValidationException: SQL validation failed. null
  • [FLINK-16433] - TableEnvironmentImpl doesn't clear buffered operations when it fails to translate the operation
  • [FLINK-16435] - Replace since decorator with versionadd to mark the version an API was introduced
  • [FLINK-16467] - MemorySizeTest#testToHumanReadableString() is not portable
  • [FLINK-16526] - Fix exception when computed column expression references a keyword column name
  • [FLINK-16541] - Document of table.exec.shuffle-mode is incorrect
  • [FLINK-16550] - HadoopS3* tests fail with NullPointerException exceptions
  • [FLINK-16560] - Forward Configuration in PackagedProgramUtils#getPipelineFromProgram
  • [FLINK-16567] - Get the API error of the StreamQueryConfig on Page "Query Configuration"
  • [FLINK-16573] - Kinesis consumer does not properly shutdown RecordFetcher threads
  • [FLINK-16576] - State inconsistency on restore with memory state backends
  • [FLINK-16626] - Prevent REST handler from being closed more than once
  • [FLINK-16632] - SqlDateTimeUtils#toSqlTimestamp(String, String) may yield incorrect result
  • [FLINK-16635] - Incompatible okio dependency in flink-metrics-influxdb module
  • [FLINK-16646] - flink read orc file throw a NullPointerException
  • [FLINK-16647] - Miss file extension when inserting to hive table with compression
  • [FLINK-16652] - BytesColumnVector should init buffer in Hive 3.x
  • [FLINK-16662] - Blink Planner failed to generate JobGraph for POJO DataStream converting to Table (Cannot determine simple type name)
  • [FLINK-16664] - Unable to set DataStreamSource parallelism to default (-1)
  • [FLINK-16675] - TableEnvironmentITCase. testClearOperation fails on travis nightly build
  • [FLINK-16684] - StreamingFileSink builder does not work with Scala
  • [FLINK-16696] - Savepoint trigger documentation is insufficient
  • [FLINK-16703] - AkkaRpcActor state machine does not record transition to terminating state.
  • [FLINK-16705] - LocalExecutor tears down MiniCluster before client can retrieve JobResult
  • [FLINK-16718] - KvStateServerHandlerTest leaks Netty ByteBufs
  • [FLINK-16727] - Fix cast exception when having time point literal as parameters
  • [FLINK-16732] - Failed to call Hive UDF with constant return value
  • [FLINK-16740] - OrcSplitReaderUtil::logicalTypeToOrcType fails to create decimal type with precision < 10
  • [FLINK-16759] - HiveModuleTest failed to compile on release-1.10
  • [FLINK-16767] - Failed to read Hive table with RegexSerDe
  • [FLINK-16771] - NPE when filtering by decimal column
  • [FLINK-16821] - Run Kubernetes test failed with invalid named "minikube"
  • [FLINK-16822] - The config set by SET command does not work
  • [FLINK-16825] - PrometheusReporterEndToEndITCase should rely on path returned by DownloadCache
  • [FLINK-16836] - Losing leadership does not clear rpc connection in JobManagerLeaderListener
  • [FLINK-16860] - Failed to push filter into OrcTableSource when upgrading to 1.9.2
  • [FLINK-16888] - Re-add jquery license file under "/licenses"
  • [FLINK-16901] - Flink Kinesis connector NOTICE should have contents of AWS KPL's THIRD_PARTY_NOTICES file manually merged in
  • [FLINK-16913] - ReadableConfigToConfigurationAdapter#getEnum throws UnsupportedOperationException
  • [FLINK-16916] - The logic of NullableSerializer#copy is wrong
  • [FLINK-16944] - Compile error in. DumpCompiledPlanTest and PreviewPlanDumpTest
  • [FLINK-16980] - Python UDF doesn't work with protobuf 3.6.1
  • [FLINK-16981] - flink-runtime tests are crashing the JVM on Java11 because of PowerMock
  • [FLINK-17062] - Fix the conversion from Java row type to Python row type
  • [FLINK-17066] - Update pyarrow version bounds less than 0.14.0
  • [FLINK-17093] - Python UDF doesn't work when the input column is from composite field
  • [FLINK-17107] - CheckpointCoordinatorConfiguration#isExactlyOnce() is inconsistent with StreamConfig#getCheckpointMode()
  • [FLINK-17114] - When the pyflink job runs in local mode and the command "python" points to Python 2.7, the startup of the Python UDF worker will fail.
  • [FLINK-17124] - The PyFlink Job runs into infinite loop if the Python UDF imports job code
  • [FLINK-17152] - FunctionDefinitionUtil generate wrong resultType and acc type of AggregateFunctionDefinition
  • [FLINK-17308] - ExecutionGraphCache cachedExecutionGraphs not cleanup cause OOM Bug
  • [FLINK-17313] - Validation error when insert decimal/varchar with precision into sink using TypeInformation of row
  • [FLINK-17334] - Flink does not support HIVE UDFs with primitive return types
  • [FLINK-17338] - LocalExecutorITCase.testBatchQueryCancel test timeout
  • [FLINK-17359] - Entropy key is not resolved if flink-s3-fs-hadoop is added as a plugin
  • [FLINK-17403] - Fix invalid classpath in BashJavaUtilsITCase
  • [FLINK-17471] - Move LICENSE and NOTICE files to root directory of python distribution
  • [FLINK-17483] - Update flink-sql-connector-elasticsearch7 NOTICE file to correctly reflect bundled dependencies
  • [FLINK-17496] - Performance regression with amazon-kinesis-producer 0.13.1 in Flink 1.10.x
  • [FLINK-17499] - LazyTimerService used to register timers via State Processing API incorrectly mixes event time timers with processing time timers
  • [FLINK-17514] - TaskCancelerWatchdog does not kill TaskManager

New Feature

Improvement

  • [FLINK-9656] - Environment java opts for flink run
  • [FLINK-15094] - Warning about using private constructor of java.nio.DirectByteBuffer in Java 11
  • [FLINK-15584] - Give nested data type of ROWs in ValidationException
  • [FLINK-15616] - Move boot error messages from python-udf-boot.log to taskmanager's log file
  • [FLINK-15989] - Rewrap OutOfMemoryError in allocateUnpooledOffHeap with better message
  • [FLINK-16018] - Improve error reporting when submitting batch job (instead of AskTimeoutException)
  • [FLINK-16125] - Make zookeeper.connect optional for Kafka connectors
  • [FLINK-16167] - Update documentation about python shell execution
  • [FLINK-16191] - Improve error message on Windows when RocksDB Paths are too long
  • [FLINK-16280] - Fix sample code errors in the documentation about elasticsearch connector
  • [FLINK-16288] - Setting the TTL for discarding task pods on Kubernetes.
  • [FLINK-16293] - Document using plugins in Kubernetes
  • [FLINK-16343] - Improve exception message when reading an unbounded source in batch mode
  • [FLINK-16406] - Increase default value for JVM Metaspace to minimise its OutOfMemoryError
  • [FLINK-16538] - Restructure Python Table API documentation
  • [FLINK-16604] - Column key in JM configuration is too narrow
  • [FLINK-16683] - Remove scripts for starting Flink on Windows
  • [FLINK-16697] - Disable JMX rebinding
  • [FLINK-16763] - Should not use BatchTableEnvironment for Python UDF in the document of flink-1.10
  • [FLINK-16772] - Bump derby to 10.12.1.1+ or exclude it
  • [FLINK-16790] - enables the interpretation of backslash escapes
  • [FLINK-16862] - Remove example url in quickstarts
  • [FLINK-16874] - Respect the dynamic options when calculating memory options in taskmanager.sh
  • [FLINK-16942] - ES 5 sink should allow users to select netty transport client
  • [FLINK-17065] - Add documentation about the Python versions supported for PyFlink
  • [FLINK-17125] - Add a Usage Notes Page to Answer Common Questions Encountered by PyFlink Users
  • [FLINK-17254] - Improve the PyFlink documentation and examples to use SQL DDL for source/sink definition
  • [FLINK-17276] - Add checkstyle to training exercises
  • [FLINK-17277] - Apply IntelliJ recommendations to training exercises
  • [FLINK-17278] - Add Travis to the training exercises
  • [FLINK-17279] - Use gradle build scans for training exercises
  • [FLINK-17316] - Have HourlyTips solutions use TumblingEventTimeWindows.of

Task

  • [FLINK-15741] - Fix TTL docs after enabling RocksDB compaction filter by default (needs Chinese translation)
  • [FLINK-15933] - update content of how generic table schema is stored in hive via HiveCatalog
  • [FLINK-15991] - Create Chinese documentation for FLIP-49 TM memory model
  • [FLINK-16004] - Exclude flink-rocksdb-state-memory-control-test jars from the dist
  • [FLINK-16454] - Update the copyright year in NOTICE files
  • [FLINK-16530] - Add documentation about "GROUPING SETS" and "CUBE" support in streaming mode
  • [FLINK-16592] - The doc of Streaming File Sink has a mistake of grammar

Want to contribute translation?

Edit This Page