Get a Flink example program up and running in a few simple steps.
Setup: Download and Start Flink
Flink runs on Linux, Mac OS X, and Windows. To be able to run Flink, the only requirement is to have a working Java 8.x installation. Windows users, please take a look at the Flink on Windows guide which describes how to run Flink on Windows for local setups.
You can check the correct installation of Java by issuing the following command:
If you have Java 8, the output will look something like this:
Download a binary from the downloads page. You can pick
any Hadoop/Scala combination you like. If you plan to just use the local file system, any Hadoop
version will work fine.
Go to the download directory.
Unpack the downloaded archive.
For MacOS X users, Flink can be installed through Homebrew.
Start a Local Flink Cluster
Check the Dispatcher’s web frontend at http://localhost:8081 and make sure everything is up and running. The web frontend should report a single available TaskManager instance.
You can also verify that the system is running by checking the log files in the logs directory:
Read the Code
You can find the complete source code for this SocketWindowWordCount example in scala and java on GitHub.
Run the Example
Now, we are going to run this Flink application. It will read text from
a socket and once every 5 seconds print the number of occurrences of
each distinct word during the previous 5 seconds, i.e. a tumbling
window of processing time, as long as words are floating in.
First of all, we use netcat to start local server via
Submit the Flink program:
The program connects to the socket and waits for input. You can check the web interface to verify that the job is running as expected:
Words are counted in time windows of 5 seconds (processing time, tumbling
windows) and are printed to stdout. Monitor the TaskManager’s output file
and write some text in nc (input is sent to Flink line by line after
hitting ):
The .out file will print the counts at the end of each time window as long
as words are floating in, e.g.:
To stop Flink when you’re done type:
Next Steps
Check out some more examples to get a better feel for Flink’s programming APIs. When you are done with that, go ahead and read the streaming guide.