Apache Flink 1.12.4 Released

May 21, 2021 - Arvid Heise

The Apache Flink community released the next bugfix version of the Apache Flink 1.12 series. This release includes 21 fixes and minor improvements for Flink 1.12.3. The list below includes a detailed list of all fixes and improvements. We highly recommend all users to upgrade to Flink 1.12.4. Updated Maven dependencies: <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.12.4</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.12.4</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.12.4</version> </dependency> You can find the binaries on the updated Downloads page. ...

Continue reading »

Scaling Flink automatically with Reactive Mode

May 6, 2021 - Robert Metzger (@rmetzger_)

Introduction # Streaming jobs which run for several days or longer usually experience variations in workload during their lifetime. These variations can originate from seasonal spikes, such as day vs. night, weekdays vs. weekend or holidays vs. non-holidays, sudden events or simply the growing popularity of your product. Although some of these variations are more predictable than others, in all cases there is a change in job resource demand that needs to be addressed if you want to ensure the same quality of service for your customers. ...

Continue reading »

Apache Flink 1.13.0 Release Announcement

May 3, 2021 - Stephan Ewen (@StephanEwen) Dawid Wysakowicz (@dwysakowicz)

The Apache Flink community is excited to announce the release of Flink 1.13.0! More than 200 contributors worked on over 1,000 issues for this new version. The release brings us a big step forward in one of our major efforts: Making Stream Processing Applications as natural and as simple to manage as any other application. The new reactive scaling mode means that scaling streaming applications in and out now works like in any other application by just changing the number of parallel processes. ...

Continue reading »

Apache Flink 1.12.3 Released

April 29, 2021 - Arvid Heise

The Apache Flink community released the next bugfix version of the Apache Flink 1.12 series. This release includes 73 fixes and minor improvements for Flink 1.12.2. The list below includes a detailed list of all fixes and improvements. We highly recommend all users to upgrade to Flink 1.12.3. Updated Maven dependencies: <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.12.3</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.12.3</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.12.3</version> </dependency> You can find the binaries on the updated Downloads page. ...

Continue reading »

Stateful Functions 3.0.0: Remote Functions Front and Center

April 15, 2021 - Igal Shilman (@IgalShilman) Tzu-Li (Gordon) Tai (@tzulitai)

The Apache Flink community is happy to announce the release of Stateful Functions (StateFun) 3.0.0! Stateful Functions is a cross-platform stack for building Stateful Serverless applications, making it radically simpler to develop scalable, consistent, and elastic distributed applications. This new release brings remote functions to the front and center of StateFun, making the disaggregated setup that separates the application logic from the StateFun cluster the default. It is now easier, more efficient, and more ergonomic to write applications that live in their own processes or containers. ...

Continue reading »

A Rundown of Batch Execution Mode in the DataStream API

March 11, 2021 - Dawid Wysakowicz (@dwysakowicz)

Flink has been following the mantra that Batch is a Special Case of Streaming since the very early days. As the project evolved to address specific uses cases, different core APIs ended up being implemented for batch (DataSet API) and streaming execution (DataStream API), but the higher-level Table API/SQL was subsequently designed following this mantra of unification. With Flink 1.12, the community worked on bringing a similarly unified behaviour to the DataStream API, and took the first steps towards enabling efficient batch execution in the DataStream API. ...

Continue reading »

Apache Flink 1.12.2 Released

March 3, 2021 - Yuan Mei Roman Khachatryan

The Apache Flink community released the next bugfix version of the Apache Flink 1.12 series. This release includes 83 fixes and minor improvements for Flink 1.12.1. The list below includes a detailed list of all fixes and improvements. We highly recommend all users to upgrade to Flink 1.12.2. Updated Maven dependencies: <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.12.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.12.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.12.2</version> </dependency> You can find the binaries on the updated Downloads page. ...

Continue reading »

How to natively deploy Flink on Kubernetes with High-Availability (HA)

February 10, 2021 - Yang Wang

Flink has supported resource management systems like YARN and Mesos since the early days; however, these were not designed for the fast-moving cloud-native architectures that are increasingly gaining popularity these days, or the growing need to support complex, mixed workloads (e.g. batch, streaming, deep learning, web services). For these reasons, more and more users are using Kubernetes to automate the deployment, scaling and management of their Flink applications. From release to release, the Flink community has made significant progress in integrating natively with Kubernetes, from active resource management to “Zookeeperless” High Availability (HA). ...

Continue reading »

Apache Flink 1.10.3 Released

January 29, 2021 - Xintong Song

The Apache Flink community released the third bugfix version of the Apache Flink 1.10 series. This release includes 36 fixes and minor improvements for Flink 1.10.2. The list below includes a detailed list of all fixes and improvements. We highly recommend all users to upgrade to Flink 1.10.3. Updated Maven dependencies: <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.10.3</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.10.3</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.10.3</version> </dependency> You can find the binaries on the updated Downloads page. ...

Continue reading »

Apache Flink 1.12.1 Released

January 19, 2021 - Xintong Song

The Apache Flink community released the first bugfix version of the Apache Flink 1.12 series. This release includes 79 fixes and minor improvements for Flink 1.12.0. The list below includes a detailed list of all fixes and improvements. We highly recommend all users to upgrade to Flink 1.12.1. Attention: Using unaligned checkpoints in Flink 1.12.0 combined with two/multiple inputs tasks or with union inputs for single input tasks can result in corrupted state. ...

Continue reading »