Apache Flink is an open source platform for scalable batch and stream data processing.

Flink’s core is a streaming dataflow engine that provides data distribution, communication, and fault tolerance for distributed computations over data streams.

Flink includes several APIs for creating applications that use the Flink engine:

  1. DataSet API for static data embedded in Java, Scala, and Python,
  2. DataStream API for unbounded streams embedded in Java and Scala, and
  3. Table API with a SQL-like expression language embedded in Java and Scala.

Flink also bundles libraries for domain-specific use cases:

  1. Machine Learning library, and
  2. Gelly, a graph processing API and library.

You can integrate Flink easily with other well-known open source systems both for data input and output as well as deployment.

Check out the features page to get a tour of all major Flink features.

Apache Flink Stack

Fast

State-of-the art performance exploiting in-memory processing and data streaming.

Reliable

Flink is designed to perform very well even when the cluster's memory runs out.

Expressive

Write beautiful, type-safe code in Java and Scala. Execute it on a cluster.

Easy to use

Few configuration parameters required. Cost-based optimizer built in.

Scalable

Tested on clusters of 100s of machines, Google Compute Engine, and Amazon EC2.

Hadoop-compatible

Flink runs on YARN and HDFS and has a Hadoop compatibility package.


Getting Started

Download the latest stable release and run Flink on your machine, cluster, or cloud:

The documentation contains a setup guide for all deployment options.

The programming guide contains all information to get you started with writing and testing your Flink programs.

Check out the documentation for the next steps.

Community

You can post questions to the Flink community on various channels. Pick the one, which suits you best:

  • User mailing list. Subscribe to the mailing list by sending an empty email to user-subscribe@flink.apache.org. Once the subscription is confirmed, you can send questions to user@flink.apache.org.

  • Stack Overflow. Post your questions to Stack Overflow and tag them with #flink.

  • IRC chat. The IRC channel #flink at irc.freenode.org is dedicated to Apache Flink. Join the channel and chat with the Flink community.

Check out the community page for all community-related information. If you want to contribute, make sure to have a look at the contribution guide.