Note: If using MiniKube please make sure to execute minikube ssh 'sudo ip link set docker0 promisc on' before deploying a Flink cluster.
Otherwise Flink components are not able to self reference themselves through a Kubernetes service.
Flink session cluster on Kubernetes
A Flink session cluster is executed as a long-running Kubernetes Deployment.
Note that you can run multiple Flink jobs on a session cluster.
Each job needs to be submitted to the cluster after the cluster has been deployed.
A basic Flink session cluster deployment in Kubernetes has three components:
a Deployment/Job which runs the JobManager
a Deployment for a pool of TaskManagers
a Service exposing the JobManager’s REST and UI ports
Deploy Flink session cluster on Kubernetes
Using the resource definitions for a session cluster, launch the cluster with the kubectl command:
A Flink job cluster is a dedicated cluster which runs a single job.
The job is part of the image and, thus, there is no extra job submission needed.
Creating the job-specific image
The Flink job cluster image needs to contain the user code jars of the job for which the cluster is started.
Therefore, one needs to build a dedicated container image for every job.
Please follow these instructions to build the Docker image.
Deploy Flink job cluster on Kubernetes
In order to deploy the a job cluster on Kubernetes please follow these instructions.