Apache Flink 2.3.0 Release Announcement

June 25, 2026 - Apache Flink PMC

The Apache Flink PMC is pleased to announce the release of Apache Flink 2.3.0.

This release significantly expands SQL capabilities with changelog conversion operators, enhances materialized table flexibility, introduces an experimental, high-performance native S3 filesystem, and delivers application management. Flink 2.3.0 brings together contributors from around the globe, implements full or core functionalities of 15 FLIPs (Flink Improvement Proposals), and resolves numerous issues and enhancements.

Key improvements in this release include new SQL operators for changelog manipulation (FROM_CHANGELOG and TO_CHANGELOG), fine-grained control over materialized table refresh strategies, adaptive partition selection for optimized backpressure handling, and a completely redesigned S3 filesystem built on AWS SDK v2. The introduction of application-level lifecycle management provides better visibility and control for production deployments, while enhanced watermark alignment can dramatically improve backlog processing performance. A reworked SinkUpsertMaterializer brings much improved performance for some Flink SQL workloads.

We extend our heartfelt thanks to all contributors for making this release possible!

Let’s dive into the details.

Flink SQL Improvements #

FROM_CHANGELOG and TO_CHANGELOG: Bridging Append-only and Dynamic Changelog Tables #

The DataStream API has long offered toChangelogStream() and fromChangelogStream() for working with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in Process Table Functions:

  • FROM_CHANGELOG converts an append-only stream that carries an operation column into a dynamic table. A configurable op_mapping makes it straightforward to plug in custom CDC formats and controls how rows with unmapped operation codes are treated.
  • TO_CHANGELOG is the inverse: it materializes a dynamic table back into an append-only changelog stream. This is the first SQL-level operator that lets users convert retract or upsert streams into append form — useful for archival, audit, writing to append-only sinks, and working around pipelines that require an append-only table.

The 2.3 release covers limited basic use cases for both. Future versions will extend both functions with PARTITION BY, invalid_op_handling, produces_full_deletes and more to make both features powerful and extensive.

More Information

Materialized Table Evolution: DDL Parity and Refresh Control #

Flink 2.3 brings materialized tables to feature parity with regular tables through two major enhancements.

First, CREATE MATERIALIZED TABLE now accepts explicit column definitions, including watermarks and primary keys, just like regular tables. ALTER MATERIALIZED TABLE gains full DDL capabilities—ADD, MODIFY, and DROP operations for metadata and computed columns, plus RENAME TO, allowing materialized tables to evolve through the same workflow already used for regular Flink tables.

Second, Flink 2.3 introduces granular control over data reprocessing when a materialized table’s query changes. The new START_MODE clause lets you choose exactly where the refresh pipeline begins. There is also special support for attempting to resume processing from the exact source offsets where the previous job instance stopped.

These enhancements eliminate the need to drop and recreate materialized tables when query definitions evolve, and prevent unnecessary reprocessing of historical data when iterating on pipeline logic. While full reprocessing is sometimes unavoidable, specifically when the query optimizer generates a completely new physical plan, forcing this behavior for every evolution is both unnecessary and costly. For many common evolutions, such as adding a nullable column or making compatible logic changes, re-ingesting historical data can now be avoided.

More Information

SinkUpsertMaterializer: Explicit Conflict Handling #

The SinkUpsertMaterializer is required when the upsert key (the unique identifier provided by the stream) is different from the primary key (the unique identifier in the target sink table). This happens in scenarios like multi-stage transformations, projections, or joins.

Previously, the SinkUpsertMaterializer has had poor performance and high resource consumption because it keeps the full history of updates for the primary key, leading to unbounded state growth. Flink 2.3 addresses this with two key improvements.

By default, queries now fail at planning time when upsert and primary keys differ, requiring you to explicitly choose a conflict strategy. This is done with a new ON CONFLICT clause that makes the behavior explicit. You choose how to handle conflicts: DO NOTHING (silent skip), DO ERROR (fail the job), or DO DEDUPLICATE (materialize and deduplicate, similar to what Flink has done until now):

INSERT INTO target_table
SELECT * FROM source
ON CONFLICT DO DEDUPLICATE;

Second, watermark-based compaction reduces state size by cleaning up old changelog records that can no longer affect the final result. Two new configuration options control the compaction behavior:

  • table.exec.sink.upserts.compaction-mode (default: WATERMARK) — WATERMARK or CHECKPOINT.
  • table.exec.sink.upserts.compaction-interval — optional fallback interval for emitting watermarks when none arrive naturally.

More Information

Process Table Function Enhancements #

Process Table Functions (PTFs), introduced in Flink 2.1, gain new capabilities that align them with the DataStream API:

  • Late data handling: PTFs can now react to late records instead of silently dropping them, enabling custom late data strategies at the SQL level.
  • Ordered table arguments: The new ORDER BY clause on table arguments ensures PTFs receive rows in deterministic temporal order within each partition:
SELECT * FROM 
  MyTimestampedPtf(
    input => TABLE events PARTITION BY user_id ORDER BY event_time
  );

Flink 2.3 includes only these enhancements that enable more sophisticated temporal processing patterns and late data handling directly in SQL. The other features described in FLIP-565, around improved state access and broadcast state, are expected to follow in a future release.

More Information

ARTIFACT Keyword for User-Defined Functions #

The USING clause of CREATE FUNCTION now accepts an ARTIFACT keyword as a future-proof alternative to JAR. This generic keyword prepares the syntax for future ecosystem assets like Python wheels, while remaining fully backward compatible:

-- New syntax
CREATE FUNCTION my_func AS 'com.example.MyUdf'
  USING ARTIFACT 's3://bucket/path/my-udf.jar';

-- Existing syntax still works
CREATE FUNCTION my_func AS 'com.example.MyUdf'
  USING JAR 's3://bucket/path/my-udf.jar';

More Information

Critical Bug Fix: MiniBatch Aggregation Record Loss #

Flink 2.3 fixes a critical bug in MiniBatchGroupAggFunction that could silently drop records when mini-batch aggregation was enabled and the planner used a ONE_PHASE aggregation strategy. The bug occurred when a key’s mini-batch contained only retractions with no existing state—the function would incorrectly return early, dropping all remaining keys in the bundle. This has been corrected to ensure all keys are processed.

More Information

Connectors #

Native S3 FileSystem #

Flink 2.3 introduces a ground-up S3 Filesystem with flink-s3-fs-native, a new plugin built directly on AWS SDK v2. This Native S3 FileSystem is experimental in Flink 2.3. It is functionally complete, and replaces the Hadoop and Presto-based connectors with a modern implementation that delivers:

  • Better performance: See Benchmarking Native S3 FileSystem (flink-s3-fs-native) vs Presto S3 (flink-s3-fs-presto) for details
  • Native AWS integration: IAM Roles for Service Accounts (IRSA), modern credential providers, and direct SDK v2 integration
  • Non-blocking I/O: Asynchronous operations for improved throughput
  • Unified implementation: Single plugin provides both FileSystem and RecoverableWriter (exactly-once streaming sinks)
  • Zero Hadoop dependencies: No dependency mess with smaller footprint

The plugin registers the standard s3:// and s3a:// scheme and uses a new s3.* configuration namespace:

s3.region: us-west-2
s3.endpoint: https://s3.us-west-2.amazonaws.com
s3.path-style-access: false
s3.upload.min.part.size: 5MB
s3.upload.max.concurrent.uploads: 4
s3.async.enabled: true
s3.read.buffer.size: 64KB
s3.bulk-copy.enabled: true

Additional configuration supports SSE-KMS encryption, chunked encoding, checksum validation, and entropy injection for high-throughput workloads.

More Information

Runtime Improvements #

Support Adaptive Partition Selection #

Uneven load across tasks in a Flink job may cause backpressure and latency. For the RebalancePartitioner and RescalePartitioner data partitioning modes, Flink 2.3 introduces an adaptive data partitioning feature based on downstream task load. It distributes data dynamically according to downstream workloads to balance traffic and improve job throughput.

The feature is opt-in via two new options:

  • taskmanager.network.adaptive-partitioner.enabled (default: false)
  • taskmanager.network.adaptive-partitioner.max-traverse-size (default: 4) — number of channels examined when selecting the idlest target.

More Information

AdaptiveScheduler Rescale History and Web UI #

Streaming jobs using the adaptive scheduler now record a complete history of rescale events, including parallelism changes, slot allocations, scheduler state transitions, and termination reasons. This data is available through new REST endpoints and visualized in a dedicated “Rescales” tab in the Flink Web UI.

The UI exposes:

  • Rescale event counts and timeline
  • Duration statistics with percentiles
  • Per-event details (parallelism before/after, triggered-by reason)
  • Adaptive scheduler configuration
# Enable rescale history (disabled by default)
web.adaptive-scheduler.rescale-history.size: 100

Setting the history size to a positive integer activates the feature.

More Information

Watermark Alignment for Fast Backlog Processing #

Prior to Flink 2.3, watermark alignment due to the announcement delays was inadvertently limiting how quickly a job could process a backlog. For example with max allowed drift configured to 30s and watermark alignment updated every ~1s, prior to Flink 2.3 watermark alignment was de facto capping the backlog processing speed to:

30 “event time” seconds per each 1 “real world” second

In Flink 2.3 the watermark alignment was redesigned to solve those announcement delays by the introduction of a watermark alignment buffer. By default this buffer has a size of 3 and it delays the application of the watermark alignment algorithm by 3 update intervals. This means in Flink 2.3+, by default watermark alignment will be pausing sources a couple of seconds later than it used to, potentially increasing state size of windowed and temporal operators. However this should be negligible for all practical use cases. Nevertheless, the size of this buffer can be configured:

# Control alignment buffer size (3 by default)
pipeline.watermark-alignment.buffer-size: 3

# Set to 0 to restore Flink 2.2 behavior
pipeline.watermark-alignment.buffer-size: 0

More Information

Checkpointing During Recovery #

Flink now supports triggering checkpoints while a job is still recovering from unaligned checkpoints. Previously, a checkpoint could only be triggered after all restored channel state had been fully consumed; when the in-flight state was large, this meant no new checkpoint could complete for hours, and any restart or rescaling during that window forced the job to redo the entire recovery from the original checkpoint.

With this feature enabled, jobs can checkpoint early during recovery, preserving work across subsequent restarts and scaling events. Exactly-once semantics are unchanged.

The feature is disabled by default and can be enabled via two new options:

  • execution.checkpointing.unaligned.recover-output-on-downstream.enabled (default: false)
  • execution.checkpointing.unaligned.during-recovery.enabled (default: false, requires the option above to be enabled)

Enabling both is recommended for jobs with large unaligned checkpoint state or frequent rescaling.

More Information

Application-Level Lifecycle Management #

Flink 2.3 introduces a first-class application concept that unifies behavior across deployment modes. The cluster-job model is replaced by a cluster-application-job hierarchy, providing better visibility and control for production deployments.

The Web UI gains an Applications tab and a redesigned home page; jobs now link back to their parent application.

More Information

Robust OpenTelemetry Metrics Export #

Jobs with many tasks and operators can produce metric payloads large enough to be rejected by OTel gRPC backends, causing metric loss in production. Flink 2.3 adds two robustness features to the OTel exporter:

# Enable gzip compression (default: none)
metrics.reporter.otel.exporter.compression: gzip

# Split exports into multiple gRPC calls (default: 0, disabled)
metrics.reporter.otel.batch.size: 500

Compression reduces payload size, while batching prevents single-call size limits from dropping data. Both are opt-in and fully backward compatible.

More Information

Documentation #

Restructured Documentation for Easier Navigation #

The Flink documentation has been reorganized to make navigation easier. Highlights:

  • Flink SQL gets a dedicated top-level section, separated from the Table API.
  • Relational streaming concepts (changelogs, dynamic tables, state, etc.) are promoted to a top-level Concepts section.
  • Python documentation is integrated into the relevant API sections instead of living in a standalone area.
  • Contributor-facing content has been relocated outside the main user-facing docs.

Existing URLs continue to work via redirects. The top-level structure is documented on the docs landing page.

More Information

Upgrade Notes #

The Flink community strives to ensure seamless upgrades. However, certain changes may require adjustments when upgrading to version 2.3. These are called out above, and in the release notes.

List of Contributors #

The Apache Flink community would like to express gratitude to all the contributors who made this release possible:

Aleksandr Iushmanov, Aleksandr Savonin, AlexYinHan, Au-Miner, averyzhang, balassai, bgeng777, Biao Geng, bowenli86, bvarghese1, Cameron, Chan hae OH, David Anderson, David Radley, Dawid Wysakowicz, ddebowczyk92, Dian Fu, Dmitriy Linevich, Dong Wang, dylanhz, Efrat Levitan, Fabian Hueske, Fabian Paul, Feat Zhang, Ferenc Csaky, Gabor Somogyi, Gustavo de Morais, Hang Ruan, Hao Li, HundalTaran, ivan.torres, Jacky Lau, Jiaan Geng, Jim Hughes, Jinkun Liu, Juntao Zhang, Kunni, Leonard Xu, Liu Jiangang, Martijn Visser, Mate Czagany, Matt Cuento, mattcuento, Mika Naylor, Mingliang Liu, MUKUL GUPTA, mukul-8, Myracle, Naci Simsek, Natea Eshetu Beshada, Nflrijal, noorall, och5351, Oleksandr Nitavskyi, Pan Yuepeng, Peter Huang, Piotr Nowojski, Piotr Przybylski, Ramin Gharib, RaoraoXiong, Rion Williams, Roman, Roman Khachatryan, Rui Fan, Samrat, Samrat002, Santwana Verma, Sergey Kononov, Sergey Nuyanzin, Shekhar Prasad Rajak, Shengkai, Stepan Stepanishchev, Swapna Marru, Timo Walther, Tom Ho, Vincent-Woo, Vishal, Vishal Kamlapure, Weihua Hu, Wenjin Xie, Wren Chan, Xuannan, xuyang, Xuyang, Yi Zhang, yongliu, Yuan Mei, Yuepeng Pan, Zakelly, Zdenek Tison, Zhanghao Chen

Want to contribute translation?

Edit This Page