Release 1.3.0 – Changelog

Changelog

The 1.3.0 release resolved 772 JIRA issues in total.

Sub-task

  • [FLINK-3722] - The divisions in the InMemorySorters' swap/compare methods hurt performance
  • [FLINK-4562] - table examples make an divided module in flink-examples
  • [FLINK-4692] - Add tumbling group-windows for batch tables
  • [FLINK-4693] - Add session group-windows for batch tables
  • [FLINK-4697] - Gather more detailed checkpoint stats in CheckpointStatsTracker
  • [FLINK-4698] - Visualize additional checkpoint information
  • [FLINK-4769] - Migrate Metrics configuration options
  • [FLINK-4912] - Introduce RECONCILING state in ExecutionGraph
  • [FLINK-4959] - Write Documentation for ProcessFunction
  • [FLINK-4994] - Don't Clear Trigger State and Merging Window Set When Purging
  • [FLINK-5132] - Introduce the ResourceSpec for grouping different resource factors in API
  • [FLINK-5133] - Support to set resource for operator in DataStream and DataSet
  • [FLINK-5134] - Aggregate ResourceSpec for chained operators when generating job graph
  • [FLINK-5135] - ResourceProfile for slot request should be expanded to correspond with ResourceSpec
  • [FLINK-5219] - Add non-grouped session windows for batch tables
  • [FLINK-5239] - Properly unpack thrown exceptions in RPC methods
  • [FLINK-5268] - Split TableProgramsTestBase into TableProgramsCollectionTestBase and TableProgramsClusterTestBase
  • [FLINK-5386] - Refactoring Window Clause
  • [FLINK-5417] - Fix the wrong config file name
  • [FLINK-5435] - Cleanup the rules introduced by FLINK-5144 when calcite releases 1.12
  • [FLINK-5454] - Add Documentation about how to tune Checkpointing for large state
  • [FLINK-5456] - Add docs about new state and checkpointing interfaces
  • [FLINK-5457] - Create documentation for Asynchronous I/O
  • [FLINK-5458] - Add documentation how to migrate from Flink 1.1. to Flink 1.2
  • [FLINK-5459] - Add documentation how to debug classloading issues
  • [FLINK-5474] - Extend DC/OS documentation
  • [FLINK-5494] - Improve Mesos documentation
  • [FLINK-5501] - Determine whether the job starts from last JobManager failure
  • [FLINK-5529] - Improve / extends windowing documentation
  • [FLINK-5545] - Remove FlinkAggregateExpandDistinctAggregatesRule when upgrading to Calcite 1.12
  • [FLINK-5555] - Add documentation about debugging watermarks
  • [FLINK-5566] - Introduce structure to hold table and column level statistics
  • [FLINK-5567] - Introduce and migrate current table statistics to FlinkStatistics
  • [FLINK-5570] - Support register external catalog to table environment
  • [FLINK-5618] - Add queryable state documentation
  • [FLINK-5640] - configure the explicit Unit Test file suffix
  • [FLINK-5653] - Add processing time OVER ROWS BETWEEN x PRECEDING aggregation to SQL
  • [FLINK-5654] - Add processing time OVER RANGE BETWEEN x PRECEDING aggregation to SQL
  • [FLINK-5655] - Add event time OVER RANGE BETWEEN x PRECEDING aggregation to SQL
  • [FLINK-5656] - Add processing time OVER ROWS BETWEEN UNBOUNDED PRECEDING aggregation to SQL
  • [FLINK-5658] - Add event time OVER ROWS BETWEEN UNBOUNDED PRECEDING aggregation to SQL
  • [FLINK-5693] - ChecksumHashCode DataSetAnalytic
  • [FLINK-5694] - Collect DataSetAnalytic
  • [FLINK-5716] - Make streaming SourceContexts aware of source idleness
  • [FLINK-5723] - Use "Used" instead of "Initial" to make taskmanager tag more readable
  • [FLINK-5729] - add hostname option in SocketWindowWordCount example to be more convenient
  • [FLINK-5767] - New aggregate function interface and built-in aggregate functions
  • [FLINK-5768] - Apply new aggregation functions for datastream and dataset tables
  • [FLINK-5777] - Pass savepoint information to CheckpointingOperation
  • [FLINK-5794] - update the documentation about “UDF/UDTF" support have parameters constructor.
  • [FLINK-5795] - Improve UDF&UDTF to support constructor with parameter
  • [FLINK-5798] - Let the RPCService provide a ScheduledExecutorService
  • [FLINK-5799] - Let RpcService.scheduleRunnable return ScheduledFuture
  • [FLINK-5803] - Add [partitioned] processing time OVER RANGE BETWEEN UNBOUNDED PRECEDING aggregation to SQL
  • [FLINK-5804] - Add [non-partitioned] processing time OVER RANGE BETWEEN UNBOUNDED PRECEDING aggregation to SQL
  • [FLINK-5810] - Harden SlotManager
  • [FLINK-5821] - Create StateBackend root interface
  • [FLINK-5822] - Make Checkpoint Coordinator aware of State Backend
  • [FLINK-5825] - In yarn mode, a small pic can not be loaded
  • [FLINK-5845] - CEP: unify key and non-keyed operators
  • [FLINK-5846] - CEP: make the operators backwards compatible.
  • [FLINK-5869] - ExecutionGraph use FailoverCoordinator to manage the failover of execution vertexes
  • [FLINK-5881] - ScalarFunction(UDF) should support variable types and variable arguments
  • [FLINK-5882] - TableFunction (UDTF) should support variable types and variable arguments
  • [FLINK-5897] - Untie Checkpoint Externalization from FileSystems
  • [FLINK-5903] - taskmanager.numberOfTaskSlots and yarn.containers.vcores did not work well in YARN mode
  • [FLINK-5906] - Add support to register UDAGG in Table and SQL API
  • [FLINK-5909] - Interface for GraphAlgorithm results
  • [FLINK-5910] - Framework for Gelly examples
  • [FLINK-5911] - Command-line parameters
  • [FLINK-5912] - Inputs for CSV and graph generators
  • [FLINK-5913] - Example drivers
  • [FLINK-5915] - Add support for the aggregate on multi fields
  • [FLINK-5916] - make env.java.opts.jobmanager and env.java.opts.taskmanager working in YARN mode
  • [FLINK-5927] - Remove old Aggregate interface and built-in functions
  • [FLINK-5956] - Add retract method into the aggregateFunction
  • [FLINK-5963] - Remove preparation mapper of DataSetAggregate
  • [FLINK-5981] - SSL version and ciper suites cannot be constrained as configured
  • [FLINK-5990] - Add [partitioned] event time OVER ROWS BETWEEN x PRECEDING aggregation to SQL
  • [FLINK-6011] - Support TUMBLE, HOP, SESSION window in streaming SQL
  • [FLINK-6012] - Support WindowStart / WindowEnd functions in streaming SQL
  • [FLINK-6020] - Blob Server cannot handle multiple job submits (with same content) parallelly
  • [FLINK-6034] - Add KeyedStateHandle for the snapshots in keyed streams
  • [FLINK-6037] - the estimateRowCount method of DataSetCalc didn't work in SQL
  • [FLINK-6089] - Implement decoration phase for rewriting predicated logical plan after volcano optimization phase
  • [FLINK-6107] - Add custom checkstyle for flink-streaming-java
  • [FLINK-6117] - 'zookeeper.sasl.disable' not takes effet when starting CuratorFramework
  • [FLINK-6136] - Separate EmbeddedNonHaServices and NonHaServices
  • [FLINK-6149] - add additional flink logical relation nodes
  • [FLINK-6155] - Allow to specify endpoint names
  • [FLINK-6190] - Write "Serializer Configurations" metainfo along with state
  • [FLINK-6191] - Make non-primitive, internal built-in serializers reconfigurable
  • [FLINK-6195] - Move gelly-examples jar from opt to examples
  • [FLINK-6198] - Update the documentation of the CEP library to include all the new features.
  • [FLINK-6200] - Add event time OVER RANGE BETWEEN UNBOUNDED PRECEDING aggregation to SQL
  • [FLINK-6201] - move python example files from resources to the examples
  • [FLINK-6228] - Integrating the OVER windows in the Table API
  • [FLINK-6240] - codeGen dataStream aggregates that use AggregateAggFunction
  • [FLINK-6241] - codeGen dataStream aggregates that use ProcessFunction
  • [FLINK-6242] - codeGen DataSet Goupingwindow Aggregates
  • [FLINK-6257] - Post-pass OVER windows
  • [FLINK-6261] - Add support for TUMBLE, HOP, SESSION to batch SQL
  • [FLINK-6340] - Introduce a TerminationFuture for Execution
  • [FLINK-6361] - Finalize the AggregateFunction interface and refactoring built-in aggregates
  • [FLINK-6367] - support custom header settings of allow origin
  • [FLINK-6392] - Change the alias of Window from optional to essential.
  • [FLINK-6425] - Integrate serializer reconfiguration into state restore flow to activate serializer upgrades
  • [FLINK-6448] - Web UI TaskManager view: Rename 'Free Memory' to 'JVM Heap'
  • [FLINK-6450] - Web UI Subtasks view for TaskManagers has a misleading name
  • [FLINK-6451] - Web UI: Rename 'Metrics' view to 'Task Metrics'
  • [FLINK-6470] - Add a utility to parse memory sizes with units
  • [FLINK-6475] - Incremental snapshots in RocksDB hold lock during async file upload
  • [FLINK-6478] - Add documentation on how to upgrade serializers for managed state
  • [FLINK-6504] - Lack of synchronization on materializedSstFiles in RocksDBKEyedStateBackend
  • [FLINK-6527] - OperatorSubtaskState has empty implementations of (un)/registerSharedStates
  • [FLINK-6533] - Duplicated registration of new shared state when checkpoint confirmations are still pending
  • [FLINK-6534] - SharedStateRegistry is disposing state handles from main thread
  • [FLINK-6535] - JobID should not be part of the registration key to the SharedStateRegistry
  • [FLINK-6545] - Make incremental checkpoints externalizable
  • [FLINK-6570] - QueryableStateClient constructor in documentation doesn't match actual signature
  • [FLINK-6618] - Fix GroupWindowStringExpressionTest testcase bug
  • [FLINK-6632] - Fix parameter case sensitive error for test passing/rejecting filter API
  • [FLINK-6633] - Register with shared state registry before adding to CompletedCheckpointStore
  • [FLINK-6640] - Ensure registration of shared state happens before externalizing a checkpoint
  • [FLINK-6650] - Fix Non-windowed group-aggregate error when using append-table mode.
  • [FLINK-6676] - API Migration guide: add QueryableStateClient changes
  • [FLINK-6677] - API Migration guide: add Table API changes
  • [FLINK-6678] - API Migration guide: add note about removed log4j default logger from core artefacts
  • [FLINK-6679] - Document HeapStatebackend
  • [FLINK-6680] - App & Flink migration guide: updates for the 1.3 release
  • [FLINK-6681] - Update "Upgrading the Flink Framework Version" section for 1.2 -> 1.3
  • [FLINK-6683] - building with Scala 2.11 no longer uses change-scala-version.sh
  • [FLINK-6700] - Remove "Download and Compile" from quickstart
  • [FLINK-6736] - Fix UDTF codegen bug when window follow by join( UDTF)
  • [FLINK-6737] - Fix over expression parse String error.
  • [FLINK-6782] - Update savepoint documentation

Bug

  • [FLINK-2067] - Chained streaming operators should not throw chained exceptions
  • [FLINK-2662] - CompilerException: "Bug: Plan generation for Unions picked a ship strategy between binary plan operators."
  • [FLINK-2814] - DeltaIteration: DualInputPlanNode cannot be cast to SingleInputPlanNode
  • [FLINK-3679] - Allow Kafka consumer to skip corrupted messages
  • [FLINK-4148] - incorrect calculation distance in QuadTree
  • [FLINK-4255] - Unstable test WebRuntimeMonitorITCase.testNoEscape
  • [FLINK-4813] - Having flink-test-utils as a dependency outside Flink fails the build
  • [FLINK-4848] - keystoreFilePath should be checked against null in SSLUtils#createSSLServerContext
  • [FLINK-4870] - ContinuousFileMonitoringFunction does not properly handle absolut Windows paths
  • [FLINK-4890] - FileInputFormatTest#testExcludeFiles fails on Windows OS
  • [FLINK-4905] - Kafka test instability IllegalStateException: Client is not started
  • [FLINK-5049] - Instability in QueryableStateITCase.testQueryableStateWithTaskManagerFailure
  • [FLINK-5070] - Unable to use Scala's BeanProperty with classes
  • [FLINK-5101] - Test CassandraConnectorITCase instable
  • [FLINK-5118] - Inconsistent records sent/received metrics
  • [FLINK-5119] - Last taskmanager heartbeat not showing in web frontend
  • [FLINK-5122] - Elasticsearch Sink loses documents when cluster has high load
  • [FLINK-5150] - WebUI metric-related resource leak
  • [FLINK-5165] - Checkpointing tests using FsStatebackend fail on Windows
  • [FLINK-5214] - Clean up checkpoint files when failing checkpoint operation on TM
  • [FLINK-5229] - Cleanup StreamTaskStates if a checkpoint operation of a subsequent operator fails
  • [FLINK-5247] - Fix incorrect check in allowedLateness() method. Make it a no-op for non-event time windows.
  • [FLINK-5267] - TaskManager logs not scrollable
  • [FLINK-5298] - TaskManager crashes when TM log not existant
  • [FLINK-5321] - FlinkMiniCluster does not start Jobmanager MetricQueryService
  • [FLINK-5323] - CheckpointNotifier should be removed from docs
  • [FLINK-5329] - Metric list is being cut off in the WebFrontend
  • [FLINK-5345] - IOManager failed to properly clean up temp file directory
  • [FLINK-5349] - Fix code sample for Twitter connector
  • [FLINK-5350] - Don't overwrite existing Jaas config property
  • [FLINK-5353] - Elasticsearch Sink loses well-formed documents when there are malformed documents
  • [FLINK-5359] - Job Exceptions view doesn't scroll
  • [FLINK-5361] - Flink shouldn't require Kerberos credentials
  • [FLINK-5364] - Rework JAAS configuration to support user-supplied entries
  • [FLINK-5365] - Mesos AppMaster/TaskManager should obey sigterm
  • [FLINK-5375] - Fix watermark documentation
  • [FLINK-5379] - Flink CliFrontend does not return when not logged in with kerberos
  • [FLINK-5380] - Number of outgoing records not reported in web interface
  • [FLINK-5381] - Scrolling in some web interface pages doesn't work (taskmanager details, jobmanager config)
  • [FLINK-5382] - Taskmanager log download button causes 404
  • [FLINK-5383] - TaskManager fails with SIGBUS when loading RocksDB
  • [FLINK-5389] - Fails #testAnswerFailureWhenSavepointReadFails
  • [FLINK-5390] - input should be closed in finally block in YarnFlinkApplicationMasterRunner#loadJobGraph()
  • [FLINK-5392] - flink-dist build failed when change scala version to 2.11
  • [FLINK-5397] - Fail to deserialize savepoints in v1.1 when there exist missing fields in class serialization descriptors
  • [FLINK-5400] - Add accessor to folding states in RuntimeContext
  • [FLINK-5402] - Fails AkkaRpcServiceTest#testTerminationFuture
  • [FLINK-5407] - Savepoint for iterative Task fails.
  • [FLINK-5408] - RocksDB initialization can fail with an UnsatisfiedLinkError in the presence of multiple classloaders
  • [FLINK-5419] - Taskmanager metrics not accessible via REST
  • [FLINK-5427] - Typo in the event_timestamps_watermarks doc
  • [FLINK-5432] - ContinuousFileMonitoringFunction is not monitoring nested files
  • [FLINK-5434] - Remove unsupported project() transformation from Scala DataStream docs
  • [FLINK-5444] - Flink UI uses absolute URLs.
  • [FLINK-5450] - WindowOperator logs about "re-registering state from an older Flink version" even though its not a restored window
  • [FLINK-5464] - MetricQueryService throws NullPointerException on JobManager
  • [FLINK-5467] - Stateless chained tasks set legacy operator state
  • [FLINK-5468] - Restoring from a semi async rocksdb statebackend (1.1) to 1.2 fails with ClassNotFoundException
  • [FLINK-5470] - Requesting non-existing log/stdout file from TM crashes the TM
  • [FLINK-5473] - setMaxParallelism() higher than 1 is possible on non-parallel operators
  • [FLINK-5481] - Simplify Row creation
  • [FLINK-5482] - QueryableStateClient does not recover from a failed lookup due to a non-running job
  • [FLINK-5484] - Kryo serialization changed between 1.1 and 1.2
  • [FLINK-5487] - Proper at-least-once support for ElasticsearchSink
  • [FLINK-5489] - maven release:prepare fails due to invalid JDOM comments in pom.xml
  • [FLINK-5492] - BootstrapTools log wrong address of started ActorSystem
  • [FLINK-5493] - FlinkDistributionOverlay does not properly display missing environment variables
  • [FLINK-5495] - ZooKeeperMesosWorkerStore cannot be instantiated
  • [FLINK-5496] - ClassCastException when using Mesos HA mode
  • [FLINK-5504] - mesos-appmaster.sh logs to wrong directory
  • [FLINK-5518] - HadoopInputFormat throws NPE when close() is called before open()
  • [FLINK-5520] - Disable outer joins with non-equality predicates
  • [FLINK-5530] - race condition in AbstractRocksDBState#getSerializedValue
  • [FLINK-5531] - SSl code block formatting is broken
  • [FLINK-5532] - Make the marker WindowAssigners for the fast aligned windows non-extendable.
  • [FLINK-5553] - Job fails during deployment with IllegalStateException from subpartition request
  • [FLINK-5556] - BarrierBuffer resets bytes written on spiller roll over
  • [FLINK-5560] - Header in checkpoint stats summary misaligned
  • [FLINK-5561] - DataInputDeserializer#available returns one too few
  • [FLINK-5562] - Driver fixes
  • [FLINK-5577] - Each time application is submitted to yarn, application id increases by two
  • [FLINK-5585] - NullPointer Exception in JobManager.updateAccumulators
  • [FLINK-5602] - Migration with RocksDB job led to NPE for next checkpoint
  • [FLINK-5608] - Cancel button not always visible
  • [FLINK-5612] - GlobPathFilter not-serializable exception
  • [FLINK-5613] - QueryableState: requesting a non-existing key in RocksDBStateBackend is not consistent with the MemoryStateBackend and FsStateBackend
  • [FLINK-5616] - YarnPreConfiguredMasterHaServicesTest fails sometimes
  • [FLINK-5617] - Check new public APIs in 1.2 release
  • [FLINK-5623] - TempBarrier dam has been closed
  • [FLINK-5626] - Improve resource release in RocksDBKeyedStateBackend
  • [FLINK-5628] - CheckpointStatsTracker implements Serializable but isn't
  • [FLINK-5629] - Unclosed RandomAccessFile in StaticFileServerHandler#respondAsLeader()
  • [FLINK-5631] - [yarn] Support downloading additional jars from non-HDFS paths
  • [FLINK-5636] - IO Metric for StreamTwoInputProcessor
  • [FLINK-5637] - Default Flink configuration contains whitespace characters, causing parser WARNings
  • [FLINK-5638] - Deadlock when closing two chained async I/O operators
  • [FLINK-5643] - StateUtil.discardStateFuture fails when state future contains null value
  • [FLINK-5644] - Task#lastCheckpointSize metric broken
  • [FLINK-5645] - IOMetrics transfer through ExecGraph does not work for failed jobs
  • [FLINK-5646] - REST api documentation missing details on jar upload
  • [FLINK-5650] - Flink-python tests executing cost too long time
  • [FLINK-5652] - Memory leak in AsyncDataStream
  • [FLINK-5660] - Not properly cleaning PendingCheckpoints up
  • [FLINK-5662] - Alias in front of output fails
  • [FLINK-5663] - Checkpoint fails because of closed registry
  • [FLINK-5666] - Blob files are not cleaned up from ZK storage directory
  • [FLINK-5667] - Possible state data loss when task fails while checkpointing
  • [FLINK-5669] - flink-streaming-contrib DataStreamUtils.collect in local environment mode fails when offline
  • [FLINK-5670] - Local RocksDB directories not cleaned up
  • [FLINK-5684] - Add MacOS section to flink-runtime-web README
  • [FLINK-5699] - Cancel with savepoint fails with a NPE if savepoint target directory not set
  • [FLINK-5701] - FlinkKafkaProducer should check asyncException on checkpoints
  • [FLINK-5708] - we should remove duplicated configuration options
  • [FLINK-5712] - update several deprecated configuration options
  • [FLINK-5739] - NullPointerException in CliFrontend
  • [FLINK-5747] - Eager Scheduling should deploy all Tasks together
  • [FLINK-5749] - unset HADOOP_HOME and HADOOP_CONF_DIR to avoid env in build machine failing the UT and IT
  • [FLINK-5751] - 404 in documentation
  • [FLINK-5759] - Set an UncaughtExceptionHandler for all Thread Pools in JobManager
  • [FLINK-5762] - Protect initializeState() and open() by the same lock.
  • [FLINK-5771] - DelimitedInputFormat does not correctly handle multi-byte delimiters
  • [FLINK-5773] - Cannot cast scala.util.Failure to org.apache.flink.runtime.messages.Acknowledge
  • [FLINK-5793] - Running slot may not be add to AllocatedMap in SlotPool
  • [FLINK-5796] - broken links in the docs
  • [FLINK-5797] - incorrect use of port range selector in BootstrapTool
  • [FLINK-5806] - TaskExecutionState toString format have wrong key
  • [FLINK-5811] - Harden YarnClusterDescriptorTest
  • [FLINK-5814] - flink-dist creates wrong symlink when not used with cleaned before
  • [FLINK-5817] - Fix test concurrent execution failure by test dir conflicts.
  • [FLINK-5824] - Fix String/byte conversions without explicit encoding
  • [FLINK-5827] - Exception when do filter after join a udtf which returns a POJO type
  • [FLINK-5828] - BlobServer create cache dir has concurrency safety problem
  • [FLINK-5830] - OutOfMemoryError during notify final state in TaskExecutor may cause job stuck
  • [FLINK-5836] - Race condition between slot offering and task deployment
  • [FLINK-5838] - Print shell script usage
  • [FLINK-5842] - Wrong 'since' version for ElasticSearch 5.x connector
  • [FLINK-5849] - Kafka Consumer checkpointed state may contain undefined offsets
  • [FLINK-5864] - CEP: fix duplicate output patterns problem.
  • [FLINK-5871] - Enforce uniqueness of pattern names in CEP.
  • [FLINK-5872] - WebUI shows "(null)" root-exception even without exception
  • [FLINK-5885] - Java code snippet instead of scala in documentation
  • [FLINK-5890] - GatherSumApply broken when object reuse enabled
  • [FLINK-5899] - Fix the bug in EventTimeTumblingWindow for non-partialMerge aggregate
  • [FLINK-5904] - jobmanager.heap.mb and taskmanager.heap.mb not work in YARN mode
  • [FLINK-5907] - RowCsvInputFormat bug on parsing tsv
  • [FLINK-5928] - Externalized checkpoints overwritting each other
  • [FLINK-5932] - Order of legacy vs new state initialization in the AbstractStreamOperator.
  • [FLINK-5933] - Allow Evictor for merging windows
  • [FLINK-5934] - Scheduler in ExecutionGraph null if failure happens in ExecutionGraph.restoreLatestCheckpointedState
  • [FLINK-5937] - Add documentation about the task lifecycle.
  • [FLINK-5940] - ZooKeeperCompletedCheckpointStore cannot handle broken state handles
  • [FLINK-5942] - Harden ZooKeeperStateHandleStore to deal with corrupted data
  • [FLINK-5945] - Close function in OuterJoinOperatorBase#executeOnCollections
  • [FLINK-5948] - Error in Python zip_with_index documentation
  • [FLINK-5949] - Flink on YARN checks for Kerberos credentials for non-Kerberos authentication methods
  • [FLINK-5955] - Merging a list of buffered records will have problem when ObjectReuse is turned on
  • [FLINK-5962] - Cancel checkpoint canceller tasks in CheckpointCoordinator
  • [FLINK-5965] - Typo on DropWizard wrappers
  • [FLINK-5971] - JobLeaderIdService should time out registered jobs
  • [FLINK-5972] - Don't allow shrinking merging windows
  • [FLINK-5977] - Rename MAX_ATTEMPTS_HISTORY_SIZE key
  • [FLINK-5985] - Flink treats every task as stateful (making topology changes impossible)
  • [FLINK-5994] - Add Janino to flink-table JAR file
  • [FLINK-6000] - Can not start HA cluster with start-cluster.sh
  • [FLINK-6001] - NPE on TumblingEventTimeWindows with ContinuousEventTimeTrigger and allowedLateness
  • [FLINK-6002] - Documentation: 'MacOS X' under 'Download and Start Flink' in Quickstart page is not rendered correctly
  • [FLINK-6007] - ConcurrentModificationException in WatermarkCallbackService
  • [FLINK-6010] - Documentation: correct IntelliJ IDEA Plugins path in 'Installing the Scala plugin' section
  • [FLINK-6023] - Fix Scala snippet into Process Function (Low-level Operations) Doc
  • [FLINK-6025] - User code ClassLoader not used when KryoSerializer fallbacks to serialization for copying
  • [FLINK-6031] - Add parameter for per job yarn clusters to control whether the user code jar is included into the system classloader.
  • [FLINK-6032] - CEP-Clean up the operator state when not needed.
  • [FLINK-6040] - DataStreamUserDefinedFunctionITCase occasionally fails
  • [FLINK-6044] - TypeSerializerSerializationProxy.read() doesn't verify the read buffer length
  • [FLINK-6051] - Wrong metric scope names in documentation
  • [FLINK-6059] - Reject DataSet<Row> and DataStream<Row> without RowTypeInformation
  • [FLINK-6061] - NPE on TypeSerializer.serialize with a RocksDBStateBackend calling entries() on a keyed state in the open() function
  • [FLINK-6078] - ZooKeeper based high availability services should not close the underlying CuratorFramework
  • [FLINK-6079] - Ineffective null check in FlinkKafkaConsumerBase#open()
  • [FLINK-6080] - Unclosed ObjectOutputStream in NFA#serialize()
  • [FLINK-6084] - Cassandra connector does not declare all dependencies
  • [FLINK-6103] - LocalFileSystem rename() uses File.renameTo()
  • [FLINK-6104] - Resource leak in ListViaRangeSpeedMiniBenchmark
  • [FLINK-6123] - Add support for the NOT pattern.
  • [FLINK-6129] - MetricRegistry does not stop query actor
  • [FLINK-6133] - fix build status in README.md
  • [FLINK-6143] - Unprotected access to this.flink in LocalExecutor#endSession()
  • [FLINK-6162] - Fix bug in ByteArrayOutputStreamWithPos#setPosition
  • [FLINK-6170] - Some checkpoint metrics rely on latest stat snapshot
  • [FLINK-6172] - Potentially unclosed RandomAccessFile in HistoryServerStaticFileServerHandler
  • [FLINK-6176] - Add JARs to CLASSPATH deterministically
  • [FLINK-6181] - Zookeeper scripts use invalid regex
  • [FLINK-6182] - Fix possible NPE in SourceStreamTask
  • [FLINK-6183] - TaskMetricGroup may not be cleanup when Task.run() is never called or exits early
  • [FLINK-6194] - More broken links in docs
  • [FLINK-6197] - Add support for iterative conditions.
  • [FLINK-6203] - DataSet Transformations
  • [FLINK-6205] - Put late elements in side output.
  • [FLINK-6207] - Duplicate type serializers for async snapshots of CopyOnWriteStateTable
  • [FLINK-6210] - RocksDB instance should be closed in ListViaMergeSpeedMiniBenchmark
  • [FLINK-6211] - Validation error in Kinesis Consumer when using AT_TIMESTAMP as start position
  • [FLINK-6217] - ContaineredTaskManagerParameters sets off heap memory size incorrectly
  • [FLINK-6246] - Fix generic type of OutputTag in operator Output
  • [FLINK-6256] - Fix documentation of ProcessFunction.
  • [FLINK-6259] - Fix a small spelling error
  • [FLINK-6265] - Fix consecutive() for times() pattern.
  • [FLINK-6271] - NumericBetweenParametersProvider NullPointer
  • [FLINK-6279] - the digest of VolcanoRuleMatch matched different table sources with same field names may be same
  • [FLINK-6282] - Some words was spelled wrong
  • [FLINK-6284] - Incorrect sorting of completed checkpoints in ZooKeeperCompletedCheckpointStore
  • [FLINK-6286] - hbase command not found error
  • [FLINK-6287] - Flakey JobManagerRegistrationTest
  • [FLINK-6290] - SharedBuffer is improperly released when multiple edges between entries
  • [FLINK-6292] - Travis: transfer.sh not accepting uploads via http:// anymore
  • [FLINK-6293] - Flakey JobManagerITCase
  • [FLINK-6295] - Update suspended ExecutionGraph to lower latency
  • [FLINK-6298] - Local execution is not setting RuntimeContext for RichOutputFormat
  • [FLINK-6300] - PID1 of docker images does not behave correctly
  • [FLINK-6311] - NPE in FlinkKinesisConsumer if source was closed before run
  • [FLINK-6312] - Update curator version to 2.12.0
  • [FLINK-6313] - Some words was spelled wrong and incorrect LOG.error without print
  • [FLINK-6317] - History server - wrong default directory
  • [FLINK-6327] - Bug in CommonCalc's estimateRowCount() method
  • [FLINK-6328] - Savepoints must not be counted as retained checkpoints
  • [FLINK-6330] - Improve Docker documentation
  • [FLINK-6341] - JobManager can go to definite message sending loop when TaskManager registered
  • [FLINK-6342] - Start ZooKeeperLeaderElectionService under lock to avoid race condition
  • [FLINK-6356] - Make times() eager and enable allowing combinations.
  • [FLINK-6368] - Grouping keys in stream aggregations have wrong order
  • [FLINK-6371] - Return matched patterns as Map<String, List<T>> instead of Map<String, T>
  • [FLINK-6384] - PythonStreamer does not close python processes
  • [FLINK-6386] - Missing bracket in 'Compiler Limitation' section
  • [FLINK-6394] - GroupCombine reuses instances even though object reuse is disabled
  • [FLINK-6396] - FsSavepointStreamFactoryTest fails on Windows
  • [FLINK-6397] - MultipleProgramsTestBase does not reset ContextEnvironment
  • [FLINK-6398] - RowSerializer's duplicate should always return a new instance
  • [FLINK-6400] - Lack of protection accessing masterHooks in CheckpointCoordinator#triggerCheckpoint
  • [FLINK-6401] - RocksDBPerformanceTest.testRocksDbRangeGetPerformance fails on Travis
  • [FLINK-6404] - Ensure PendingCheckpoint is registered when calling Checkpoint Hooks
  • [FLINK-6409] - TUMBLE/HOP/SESSION_START/END do not resolve time field correctly
  • [FLINK-6410] - build fails after changing Scala to 2.11
  • [FLINK-6411] - YarnApplicationMasterRunner should not interfere with RunningJobsRegistry
  • [FLINK-6415] - Make sure core Flink artifacts have no specific logger dependency
  • [FLINK-6416] - Potential divide by zero issue in InputGateMetrics#refreshAndGetAvg()
  • [FLINK-6427] - BucketingSink does not sync file length in case of cancel
  • [FLINK-6435] - AsyncWaitOperator does not handle exceptions properly
  • [FLINK-6436] - Bug in CommonCorrelate's generateCollector method when using udtf with udf
  • [FLINK-6439] - Unclosed InputStream in OperatorSnapshotUtil#readStateHandle()
  • [FLINK-6440] - Noisy logs from metric fetcher
  • [FLINK-6445] - Fix NullPointerException in CEP pattern without condition
  • [FLINK-6463] - Throw exception when NOT-NEXT is after OPTIONAL
  • [FLINK-6474] - Potential loss of precision in 32 bit integer multiplication
  • [FLINK-6479] - Fix IndexOutOfBoundsException bug
  • [FLINK-6486] - Pass RowTypeInfo to CodeGenerator instead of CRowTypeInfo
  • [FLINK-6491] - Add QueryConfig to specify state retention time for streaming queries
  • [FLINK-6501] - Make sure NOTICE files are bundled into shaded JAR files
  • [FLINK-6506] - Some tests in flink-tests exceed the memory resources in containerized Travis builds
  • [FLINK-6508] - Include license files of packaged dependencies
  • [FLINK-6509] - TestingListener might miss JobLeader notifications
  • [FLINK-6514] - Cannot start Flink Cluster in standalone mode
  • [FLINK-6515] - KafkaConsumer checkpointing fails because of ClassLoader issues
  • [FLINK-6517] - Support multiple consecutive windows
  • [FLINK-6520] - FlinkKafkaConsumer09+ does not overwrite props to disable auto commit offsets when commit mode is OffsetCommitMode.ON_CHECKPOINTS
  • [FLINK-6530] - Unclosed Response in DatadogHttpClient#validateApiKey()
  • [FLINK-6531] - Deserialize checkpoint hooks with user classloader
  • [FLINK-6536] - Improve error message in SharedBuffer::put()
  • [FLINK-6542] - Non-keyed, non-windowed aggregation fails
  • [FLINK-6548] - AvroOutputFormatTest fails on Windows
  • [FLINK-6561] - GlobFilePathFilterTest#testExcludeFilenameWithStart fails on Windows
  • [FLINK-6562] - Support implicit table references for nested fields in SQL
  • [FLINK-6564] - Build fails on file systems that do not distinguish between upper and lower case
  • [FLINK-6574] - Support nested catalogs in ExternalCatalog
  • [FLINK-6578] - SharedBuffer creates self-loops when having elements with same value/timestamp.
  • [FLINK-6579] - Add proper support for BasicArrayTypeInfo
  • [FLINK-6580] - Flink on YARN doesnt start with default parameters
  • [FLINK-6581] - Dynamic property parsing broken for YARN
  • [FLINK-6582] - Project from maven archetype is not buildable by default due to ${scala.binary.version}
  • [FLINK-6583] - Enable QueryConfig in count base GroupWindow
  • [FLINK-6585] - Table examples are not runnable in IDE
  • [FLINK-6586] - InputGateMetrics#refreshAndGetMin returns Integer.MAX_VALUE for local channels
  • [FLINK-6587] - Java Table API cannot parse function names starting with keywords
  • [FLINK-6593] - Fix Bug in ProctimeAttribute or RowtimeAttribute with CodeGenerator
  • [FLINK-6595] - Nested SQL queries do not expose proctime / rowtime attributes
  • [FLINK-6598] - Remove useless param rowRelDataType of DataStreamGroupAggregate.
  • [FLINK-6604] - Remove Java Serialization from the CEP library.
  • [FLINK-6606] - Create checkpoint hook with user classloader
  • [FLINK-6609] - Wrong version assignment when multiple TAKEs transitions
  • [FLINK-6612] - ZooKeeperStateHandleStore does not guard against concurrent delete operations
  • [FLINK-6614] - Applying function on window auxiliary function fails
  • [FLINK-6628] - Cannot start taskmanager with cygwin in directory containing spaces
  • [FLINK-6629] - ClusterClient cannot submit jobs to HA cluster if address not set in configuration
  • [FLINK-6634] - NFA serializer does not serialize the ComputationState counter.
  • [FLINK-6635] - ClientConnectionTest is broken because the ClusterClient lazily connects to the JobManager
  • [FLINK-6639] - Java/Scala code tabs broken in CEP docs
  • [FLINK-6641] - HA recovery on YARN: ClusterClient calls HighAvailabilityServices#closeAndCleanupAll
  • [FLINK-6644] - Don't register HUP unix signal handler on Windows
  • [FLINK-6646] - YARN session doesn't work with HA
  • [FLINK-6651] - Clearing registeredStates map should be protected in SharedStateRegistry#clear
  • [FLINK-6656] - Migrate CEP PriorityQueue to MapState
  • [FLINK-6662] - ClassNotFoundException: o.a.f.r.j.t.JobSnapshottingSettings recovering job
  • [FLINK-6671] - RocksDBStateBackendTest.testCancelRunningSnapshot unstable
  • [FLINK-6702] - SIGABRT after CEPOperatorTest#testCEPOperatorSerializationWRocksDB() during GC
  • [FLINK-6704] - Cannot disable YARN user jar inclusion
  • [FLINK-6714] - Operator state backend should set user classloader as context classloader when snapshotting
  • [FLINK-6753] - Flaky SqlITCase
  • [FLINK-6780] - ExternalTableSource should add time attributes in the row type

Improvement

  • [FLINK-2211] - Generalize ALS API
  • [FLINK-2908] - Web interface redraw web plan when browser resized
  • [FLINK-3123] - Allow setting custom start-offsets for the Kafka consumer
  • [FLINK-3150] - Make YARN container invocation configurable
  • [FLINK-3163] - Configure Flink for NUMA systems
  • [FLINK-3318] - Add support for quantifiers to CEP's pattern API
  • [FLINK-3347] - TaskManager (or its ActorSystem) need to restart in case they notice quarantine
  • [FLINK-3360] - Clear up StateBackend, AbstractStateBackend abstractions
  • [FLINK-3398] - Flink Kafka consumer should support auto-commit opt-outs
  • [FLINK-3427] - Add watermark monitoring to JobManager web frontend
  • [FLINK-3703] - Add sequence matching semantics to discard matched events
  • [FLINK-4288] - Make it possible to unregister tables
  • [FLINK-4326] - Flink start-up scripts should optionally start services on the foreground
  • [FLINK-4396] - GraphiteReporter class not found at startup of jobmanager
  • [FLINK-4410] - Report more information about operator checkpoints
  • [FLINK-4450] - update storm version to 1.0.0
  • [FLINK-4552] - Refactor WindowOperator/Trigger Tests
  • [FLINK-4673] - TypeInfoFactory for Either type
  • [FLINK-4754] - Make number of retained checkpoints user configurable
  • [FLINK-4917] - Deprecate "CheckpointedAsynchronously" interface
  • [FLINK-4920] - Add a Scala Function Gauge
  • [FLINK-4949] - Refactor Gelly driver inputs
  • [FLINK-4953] - Allow access to "time" in ProcessWindowFunction.Context
  • [FLINK-5059] - only serialise events once in RecordWriter#broadcastEvent
  • [FLINK-5066] - add more efficient isEvent check to EventSerializer
  • [FLINK-5067] - Make Flink compile with 1.8 Java compiler
  • [FLINK-5113] - Make all Testing Functions implement CheckpointedFunction Interface.
  • [FLINK-5129] - make the BlobServer use a distributed file system
  • [FLINK-5153] - Allow setting custom application tags for Flink on YARN
  • [FLINK-5183] - [py] Support multiple jobs per Python plan file
  • [FLINK-5222] - Rename StateBackend interface to StateBinder
  • [FLINK-5237] - Consolidate and harmonize Window Translation Tests
  • [FLINK-5277] - missing unit test for ensuring ResultPartition#add always recycles buffers
  • [FLINK-5280] - Refactor TableSource
  • [FLINK-5306] - Display checkpointing configuration details in web UI "Configuration" tab
  • [FLINK-5331] - PythonPlanBinderTest idling extremely long
  • [FLINK-5348] - Support custom field names for RowTypeInfo
  • [FLINK-5358] - Support RowTypeInfo extraction in TypeExtractor by Row instance
  • [FLINK-5368] - Let Kafka consumer show something when it fails to read one topic out of topic list
  • [FLINK-5377] - Improve savepoint docs
  • [FLINK-5385] - Add a help function to create Row
  • [FLINK-5388] - Remove private access of edges and vertices of Gelly Graph class
  • [FLINK-5395] - support locally build distribution by script create_release_files.sh
  • [FLINK-5399] - Add more information to checkpoint result of TriggerSavepointSuccess
  • [FLINK-5404] - Consolidate and update S3 documentation
  • [FLINK-5412] - Enable RocksDB tests on Windows
  • [FLINK-5414] - Bump up Calcite version to 1.11
  • [FLINK-5415] - ContinuousFileProcessingTest failed on travis
  • [FLINK-5424] - Improve Restart Strategy Logging
  • [FLINK-5438] - Typo in JobGraph generator Exception
  • [FLINK-5442] - Add test to fix ordinals of serialized enum StateDescriptor.Type
  • [FLINK-5446] - System metrics reference incomplete
  • [FLINK-5448] - Fix typo in StateAssignmentOperation Exception
  • [FLINK-5451] - Extend JMX metrics documentation section
  • [FLINK-5452] - Make table unit tests pass under cluster mode
  • [FLINK-5461] - Remove Superflous TypeInformation Declaration
  • [FLINK-5466] - Make production environment default in gulpfile
  • [FLINK-5480] - User-provided hashes for operators
  • [FLINK-5485] - Mark compiled web frontend files as binary when processed by git diff
  • [FLINK-5497] - remove duplicated tests
  • [FLINK-5499] - Try to reuse the resource location of prior execution attempt in allocating slot
  • [FLINK-5503] - mesos-appmaster.sh script could print return value message
  • [FLINK-5507] - remove queryable list state sink
  • [FLINK-5508] - Remove Mesos dynamic class loading
  • [FLINK-5515] - fix unused kvState.getSerializedValue call in KvStateServerHandler
  • [FLINK-5516] - Hardcoded paths in flink-python/.../PythonPlanBinder.java
  • [FLINK-5517] - Upgrade hbase version to 1.3.0
  • [FLINK-5519] - scala-maven-plugin version all change to 3.2.2
  • [FLINK-5521] - remove unused KvStateRequestSerializer#serializeList
  • [FLINK-5522] - Storm LocalCluster can't run with powermock
  • [FLINK-5524] - Support early out for code generated conjunctive conditions
  • [FLINK-5528] - tests: reduce the retry delay in QueryableStateITCase
  • [FLINK-5543] - customCommandLine tips in CliFrontend
  • [FLINK-5559] - queryable state: KvStateRequestSerializer#deserializeKeyAndNamespace() throws an IOException without own failure message if deserialisation fails
  • [FLINK-5563] - Add density to vertex metrics
  • [FLINK-5574] - Add checkpoint statistics docs
  • [FLINK-5575] - in old releases, warn users and guide them to the latest stable docs
  • [FLINK-5576] - extend deserialization functions of KvStateRequestSerializer to detect unconsumed bytes
  • [FLINK-5581] - Improve Kerberos security related documentation
  • [FLINK-5583] - Support flexible error handling in the Kafka consumer
  • [FLINK-5586] - Extend TableProgramsTestBase for object reuse modes
  • [FLINK-5590] - Create a proper internal state hierarchy
  • [FLINK-5597] - Improve the LocalClusteringCoefficient documentation
  • [FLINK-5598] - Return jar name when jar is uploaded
  • [FLINK-5609] - Add last update time to docs
  • [FLINK-5610] - Rename Installation and Setup to Project Setup
  • [FLINK-5615] - queryable state: execute the QueryableStateITCase for all three state back-ends
  • [FLINK-5624] - Support tumbling window on streaming tables in the SQL API
  • [FLINK-5625] - Let Date format for timestamp-based start position in Kinesis consumer be configurable.
  • [FLINK-5630] - Followups to AggregationFunction
  • [FLINK-5632] - Typo in StreamGraph
  • [FLINK-5635] - Improve Docker tooling to make it easier to build images and launch Flink via Docker tools
  • [FLINK-5639] - Clarify License implications of RabbitMQ Connector
  • [FLINK-5680] - Document env.ssh.opts
  • [FLINK-5681] - Make ReaperThread for SafetyNetCloseableRegistry a singleton
  • [FLINK-5702] - Kafka Producer docs should warn if using setLogFailuresOnly, at-least-once is compromised
  • [FLINK-5705] - webmonitor's request/response use UTF-8 explicitly
  • [FLINK-5714] - Use a builder pattern for creating CsvTableSource
  • [FLINK-5718] - Handle JVM Fatal Exceptions in Tasks
  • [FLINK-5720] - Deprecate "Folding" in all of DataStream API
  • [FLINK-5721] - Add FoldingState to State Documentation
  • [FLINK-5722] - Implement DISTINCT as dedicated operator
  • [FLINK-5727] - Unify some API of batch and stream TableEnvironment
  • [FLINK-5741] - Add tests for window function wrappers with RichFunctions
  • [FLINK-5742] - Breakpoints on documentation website
  • [FLINK-5743] - Mark WindowedStream.aggregate* methods as PublicEvolving
  • [FLINK-5745] - Set uncaught exception handler for Netty threads
  • [FLINK-5748] - Make the ExecutionGraph's FutureExecutor a ScheduledExecutionService
  • [FLINK-5756] - When there are many values under the same key in ListState, RocksDBStateBackend performances poor
  • [FLINK-5766] - Unify NoAvailableResourceException handling on ExecutionGraph
  • [FLINK-5788] - Document assumptions about File Systems and persistence
  • [FLINK-5792] - Improve “UDF/UDTF" to support constructor with parameter.
  • [FLINK-5800] - Make sure that the CheckpointStreamFactory is instantiated once per operator only
  • [FLINK-5805] - improve docs for ProcessFunction
  • [FLINK-5807] - improved wording for doc home page
  • [FLINK-5812] - Clean up FileSystem
  • [FLINK-5819] - Improve metrics reporting
  • [FLINK-5826] - UDF/UDTF should support variable types and variable arguments
  • [FLINK-5837] - improve readability of the queryable state docs
  • [FLINK-5852] - Move JSON generation code into static methods
  • [FLINK-5854] - Introduce some Flink-specific base Exception types
  • [FLINK-5870] - Make handlers aware of their REST URLs
  • [FLINK-5876] - Mention Scala type fallacies for queryable state client serializers
  • [FLINK-5877] - Fix Scala snippet in Async I/O API doc
  • [FLINK-5887] - Make CheckpointBarrier type immutable
  • [FLINK-5894] - HA docs are misleading re: state backends
  • [FLINK-5895] - Reduce logging aggressiveness of FileSystemSafetyNet
  • [FLINK-5917] - Remove MapState.size()
  • [FLINK-5921] - Adapt time mode indicator functions return custom data types
  • [FLINK-5929] - Allow Access to Per-Window State in ProcessWindowFunction
  • [FLINK-5938] - Replace ExecutionContext by Executor in Scheduler
  • [FLINK-5952] - JobCancellationWithSavepointHandlersTest uses deprecated JsonNode#getValuesAsText
  • [FLINK-5954] - Always assign names to the window in the Stream SQL API
  • [FLINK-5969] - Add savepoint backwards compatibility tests from 1.2 to 1.3
  • [FLINK-5974] - Support Mesos DNS
  • [FLINK-5975] - Mesos should support adding volumes to launched taskManagers
  • [FLINK-5976] - Refactoring duplicate Tokenizer in flink-test
  • [FLINK-5983] - Replace for/foreach/map in aggregates by while loops
  • [FLINK-5984] - Add resetAccumulator method for AggregateFunction
  • [FLINK-5998] - Un-fat Hadoop from Flink fat jar
  • [FLINK-6005] - unit test ArrayList initializations without initial size
  • [FLINK-6009] - Deprecate DataSetUtils#checksumHashCode
  • [FLINK-6013] - Add Datadog HTTP metrics reporter
  • [FLINK-6018] - Properly initialise StateDescriptor in AbstractStateBackend.getPartitionedState()
  • [FLINK-6033] - Support UNNEST query in the stream SQL API
  • [FLINK-6041] - Move StreamingFunctionUtils to 'org.apache.flink.streaming.util'
  • [FLINK-6068] - Support If() as a built-in function of TableAPI
  • [FLINK-6071] - Savepoints should not count in the number of retained checkpoints
  • [FLINK-6076] - Let the HeartbeatManager interface extend HeartbeatTarget
  • [FLINK-6086] - Rework PythonSender/-Streamer generics
  • [FLINK-6111] - Remove sleep after python process generation
  • [FLINK-6122] - add TravisCI build status to README.md
  • [FLINK-6124] - support max/min aggregations for string type
  • [FLINK-6127] - Add MissingDeprecatedCheck to checkstyle
  • [FLINK-6128] - Optimize JVM options for improve test performance
  • [FLINK-6134] - Set UUID(0L, 0L) as default leader session id
  • [FLINK-6138] - Improve UnboundedNonPartitionedProcessingOverProcessFunction
  • [FLINK-6144] - Port job manager related configuration options to ConfigOption
  • [FLINK-6157] - Make TypeInformation fully serializable
  • [FLINK-6164] - Make ProcessWindowFunction a RichFunction
  • [FLINK-6212] - Missing reference to flink-avro dependency
  • [FLINK-6223] - Rework PythonPlanBinder generics
  • [FLINK-6229] - Rework setup&configuration of PythonPlanBinder/operators
  • [FLINK-6230] - Make mmap size configurable
  • [FLINK-6236] - Savepoint page needs to include web console possibility
  • [FLINK-6247] - Build a jar-with-dependencies for flink-table and put it into ./opt
  • [FLINK-6248] - Make the optional() available to all offered patterns.
  • [FLINK-6254] - Consolidate late data methods on PatternStream and WindowedStream
  • [FLINK-6267] - Remove the useless import in FlinkRelBuilder
  • [FLINK-6268] - Object reuse for Either type
  • [FLINK-6270] - Port several network config parameters to ConfigOption
  • [FLINK-6274] - Replace usages of org.codehaus.jackson
  • [FLINK-6280] - Allow logging with Java flags
  • [FLINK-6288] - FlinkKafkaProducer's custom Partitioner is always invoked with number of partitions of default topic
  • [FLINK-6299] - make all IT cases extend from TestLogger
  • [FLINK-6304] - Clear a lot of useless import
  • [FLINK-6307] - Refactor JDBC tests
  • [FLINK-6324] - Refine state access methods in OperatorStateStore
  • [FLINK-6334] - Refactoring UDTF interface
  • [FLINK-6337] - Remove the buffer provider from PartitionRequestServerHandler
  • [FLINK-6338] - SimpleStringUtils should be called StringValueUtils
  • [FLINK-6339] - Remove useless and unused class ConnectorSource
  • [FLINK-6381] - Unnecessary synchronized object in BucketingSink
  • [FLINK-6382] - Support additional types for generated graphs in Gelly examples
  • [FLINK-6395] - TestBases not marked as abstract
  • [FLINK-6406] - Cleanup useless import
  • [FLINK-6438] - Expand docs home page a little
  • [FLINK-6443] - Add more doc links in concepts sections
  • [FLINK-6447] - AWS/EMR docs are out-of-date
  • [FLINK-6459] - Move ACCESS_CONTROL_ALLOW_ORIGIN to JobManagerOptions
  • [FLINK-6462] - Add requiresOver interface for AggregateFunction
  • [FLINK-6476] - Table environment register row data stream
  • [FLINK-6482] - Add nested serializers into configuration snapshots of composite serializers
  • [FLINK-6512] - some code examples are poorly formatted
  • [FLINK-6513] - various typos and grammatical flaws
  • [FLINK-6519] - Integrate BlobStore in HighAvailabilityServices lifecycle management
  • [FLINK-6551] - OutputTag name should not be allowed to be empty
  • [FLINK-6552] - Side outputs don't allow differing output types
  • [FLINK-6554] - CompatibilityResult should contain a notCompatible() option
  • [FLINK-6555] - Generalize ConjunctFuture
  • [FLINK-6558] - Yarn tests fail on Windows
  • [FLINK-6565] - Improve error messages for state restore failures
  • [FLINK-6566] - Narrow down interface for compatibility hook method in VersionedIOReadableWritable
  • [FLINK-6589] - ListSerializer should deserialize as ArrayList with size + 1
  • [FLINK-6596] - Disable javadoc generation in all travis builds
  • [FLINK-6600] - Add key serializer's config snapshot to KeyedBackendSerializationProxy
  • [FLINK-6601] - Use time indicators in DataStreamLogicalWindowAggregateRule
  • [FLINK-6608] - Relax Kerberos login contexts parsing by trimming whitespaces in contexts list
  • [FLINK-6616] - Clarify provenance of official Docker images
  • [FLINK-6653] - Avoid directly serializing AWS's Shard class in Kinesis consumer's checkpoints
  • [FLINK-6660] - expand the streaming connectors overview page
  • [FLINK-6674] - Update release 1.3 docs
  • [FLINK-6708] - Don't let the FlinkYarnSessionCli fail if it cannot retrieve the ClusterStatus
  • [FLINK-6766] - Update documentation with async backends and incremental checkpoints

New Feature

  • [FLINK-1579] - Create a Flink History Server
  • [FLINK-2168] - Add HBaseTableSource
  • [FLINK-3475] - DISTINCT aggregate function support for SQL queries
  • [FLINK-3695] - ValueArray types
  • [FLINK-3849] - Add FilterableTableSource interface and translation rule
  • [FLINK-3871] - Add Kafka TableSource with Avro serialization
  • [FLINK-4280] - New Flink-specific option to set starting position of Kafka consumer without respecting external offsets in ZK / Broker
  • [FLINK-4460] - Side Outputs in Flink
  • [FLINK-4523] - Allow Kinesis Consumer to start from specific timestamp / Date
  • [FLINK-4604] - Add support for standard deviation/variance
  • [FLINK-4686] - Add possibility to get column names
  • [FLINK-4821] - Implement rescalable non-partitioned state for Kinesis Connector
  • [FLINK-4856] - Add MapState for keyed streams
  • [FLINK-4896] - PageRank algorithm for directed graphs
  • [FLINK-4988] - Elasticsearch 5.x support
  • [FLINK-4997] - Extending Window Function Metadata
  • [FLINK-5017] - Introduce StreamStatus stream element to allow for temporarily idle streaming sources
  • [FLINK-5090] - Expose optionally detailed metrics about network queue lengths
  • [FLINK-5157] - Extending AllWindow Function Metadata
  • [FLINK-5265] - Introduce state handle replication mode for CheckpointCoordinator
  • [FLINK-5303] - Add CUBE/ROLLUP/GROUPING SETS operator in SQL
  • [FLINK-5406] - add normalization phase for predicate logical plan rewriting between decorrelate query phase and volcano optimization phase
  • [FLINK-5441] - Directly allow SQL queries on a Table
  • [FLINK-5571] - add open and close methods for UserDefinedFunction in TableAPI & SQL
  • [FLINK-5582] - Add a general distributive aggregate function
  • [FLINK-5692] - Add an Option to Deactivate Kryo Fallback for Serializers
  • [FLINK-5698] - Add NestedFieldsProjectableTableSource interface
  • [FLINK-5710] - Add ProcTime() function to indicate StreamSQL
  • [FLINK-5715] - Asynchronous snapshotting for HeapKeyedStateBackend
  • [FLINK-5884] - Integrate time indicators for Table API & SQL
  • [FLINK-5892] - Recover job state at the granularity of operator
  • [FLINK-5941] - Let handlers take part in job archiving
  • [FLINK-5979] - Backwards compatibility for HeapKeyedStateBackend serialization format (1.2 -> 1.3)
  • [FLINK-5991] - Expose Broadcast Operator State through public APIs
  • [FLINK-6047] - Add support for Retraction in Table API / SQL
  • [FLINK-6048] - Asynchronous snapshots for heap-based operator state backends
  • [FLINK-6112] - Support Calcite 1.12's new numerical functions
  • [FLINK-6139] - Documentation for building / preparing Flink for MapR
  • [FLINK-6165] - Implement internal continuity for looping states.
  • [FLINK-6177] - Add support for "Distributed Cache" in streaming applications
  • [FLINK-6178] - Allow upgrades to state serializers
  • [FLINK-6208] - Implement skip till next match strategy
  • [FLINK-6326] - add ProjectMergeRule at logical optimization stage
  • [FLINK-6336] - Placement Constraints for Mesos
  • [FLINK-6377] - Support map types in the Table / SQL API
  • [FLINK-6390] - Add Trigger Hooks to the Checkpoint Coordinator
  • [FLINK-6393] - Add Evenly Graph Generator to Flink Gelly
  • [FLINK-6483] - Support time materialization

Task

  • [FLINK-2883] - Add documentation to forbid key-modifying ReduceFunction
  • [FLINK-3903] - Homebrew Installation
  • [FLINK-4577] - Re-enable transparent reshard handling in Kinesis Consumer
  • [FLINK-4820] - Slf4j / log4j version upgrade to support dynamic change of log levels --> Make logging framework exchangeable
  • [FLINK-5074] - Implement a RunningJobRegistry based on Zookeeper
  • [FLINK-5084] - Replace Java Table API integration tests by unit tests
  • [FLINK-5829] - Bump Calcite version to 1.12 once available
  • [FLINK-6543] - Deprecate toDataStream

Test

  • [FLINK-5587] - AsyncWaitOperatorTest timed out on Travis
  • [FLINK-5923] - Test instability in SavepointITCase testTriggerSavepointAndResume
  • [FLINK-6175] - HistoryServerTest.testFullArchiveLifecycle fails
  • [FLINK-6471] - RocksDBStateBackendTest#testCancelRunningSnapshot sometimes fails

Wish

  • [FLINK-4545] - Flink automatically manages TM network buffer
  • [FLINK-4644] - Deprecate "flink.base.dir.path" from ConfigConstants
  • [FLINK-5378] - Update Scopt version to 3.5.0
  • [FLINK-6056] - apache-rat exclude flink directory in tools
  • [FLINK-6186] - Remove unused import
  • [FLINK-6269] - var could be a val