September 7, 2021 -
Ingo Buerk
Daisy Tsang
Part one of this tutorial will teach you how to build and run a custom source connector to be used with Table API and SQL, two high-level abstractions in Flink. The tutorial comes with a bundled docker-compose setup that lets you easily run the connector. You can then try it out with Flink’s SQL client.
Introduction # Apache Flink is a data processing engine that aims to keep state locally in order to do computations efficiently.
...
Continue reading »
September 7, 2021 -
Ingo Buerk
Daisy Tsang
In part one of this tutorial, you learned how to build a custom source connector for Flink. In part two, you will learn how to integrate the connector with a test email inbox through the IMAP protocol and filter out emails using Flink SQL.
Goals # Part two of the tutorial will teach you how to:
integrate a source connector which connects to a mailbox using the IMAP protocol use Jakarta Mail, a Java library that can send and receive email via the IMAP protocol write Flink SQL and execute the queries in the Ververica Platform for a nicer visualization You are encouraged to follow along with the code in this repository.
...
Continue reading »
August 31, 2021 -
Seth Wiesman
(@sjwiesman)
Igal Shilman
(@IgalShilman)
Tzu-Li (Gordon) Tai
(@tzulitai)
Stateful Functions is a cross-platform stack for building Stateful Serverless applications, making it radically simpler to develop scalable, consistent, and elastic distributed applications. This new release brings various improvements to the StateFun runtime, a leaner way to specify StateFun module components, and a brand new GoLang SDK!
The binary distribution and source artifacts are now available on the updated Downloads page of the Flink website, and the most recent Java SDK, Python SDK, and GoLang SDK distributions are available on Maven, PyPI, and Github repecitvely.
...
Continue reading »
August 31, 2021 -
Johannes Moser
Hint Update 29th of September: Today Apache Flink 1.14 has been released. For sure we'd still like to hear your feedback. Dear Flink Community,
we are excited to announce the first release candidate of Apache Flink 1.14. 🎉
A lot of features and fixes went into this release, including improvements to the unified batch and streaming experience, an increase in fault tolerance by reducing in-flight data, and more developments on connectors and components.
...
Continue reading »
August 9, 2021 -
Xiaoling He
The Apache Flink community released the next bugfix version of the Apache Flink 1.11 series.
This release includes 78 fixes and minor improvements for Flink 1.11.4. The list below includes a detailed list of all fixes and improvements.
We highly recommend all users to upgrade to Flink 1.11.4.
Updated Maven dependencies:
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.11.4</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.11.4</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.11.4</version> </dependency> You can find the binaries on the updated Downloads page.
...
Continue reading »
August 6, 2021 -
Jingsong Lee
The Apache Flink community released the next bugfix version of the Apache Flink 1.12 series.
This release includes 76 fixes and minor improvements for Flink 1.12.4. The list below includes a detailed list of all fixes and improvements.
We highly recommend all users to upgrade to Flink 1.12.5.
Updated Maven dependencies:
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.12.5</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.12.5</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.12.5</version> </dependency> You can find the binaries on the updated Downloads page.
...
Continue reading »
August 6, 2021 -
Yun Tang
The Apache Flink community released the second bugfix version of the Apache Flink 1.13 series.
This release includes 127 fixes and minor improvements for Flink 1.13.2. The list below includes bugfixes and improvements. For a complete list of all changes see: JIRA.
We highly recommend all users to upgrade to Flink 1.13.2.
Updated Maven dependencies:
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.13.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.13.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.13.2</version> </dependency> You can find the binaries on the updated Downloads page.
...
Continue reading »
July 7, 2021 -
Piotr Nowojski
(@PiotrNowojski)
Backpressure monitoring in the web UI
The backpressure topic was tackled from different angles over the last couple of years. However, when it comes to identifying and analyzing sources of backpressure, things have changed quite a bit in the recent Flink releases (especially with new additions to metrics and the web UI in Flink 1.13). This post will try to clarify some of these changes and go into more detail about how to track down the source of backpressure, but first…
...
Continue reading »
May 28, 2021 -
Dawid Wysakowicz
(@dwysakowicz)
The Apache Flink community released the first bugfix version of the Apache Flink 1.13 series.
This release includes 82 fixes and minor improvements for Flink 1.13.1. The list below includes bugfixes and improvements. For a complete list of all changes see: JIRA.
We highly recommend all users to upgrade to Flink 1.13.1.
Updated Maven dependencies:
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.13.1</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.13.1</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.13.1</version> </dependency> You can find the binaries on the updated Downloads page.
...
Continue reading »
May 21, 2021 -
Arvid Heise
The Apache Flink community released the next bugfix version of the Apache Flink 1.12 series.
This release includes 21 fixes and minor improvements for Flink 1.12.3. The list below includes a detailed list of all fixes and improvements.
We highly recommend all users to upgrade to Flink 1.12.4.
Updated Maven dependencies:
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.12.4</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.12.4</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.12.4</version> </dependency> You can find the binaries on the updated Downloads page.
...
Continue reading »