Standalone

Introduction #

Standalone mode is Flink’s simplest deployment mode. This short guide will show you how to download the latest stable version of Flink, install, and run it.
You will also run an example Flink CDC job and view it in the web UI.

Preparation #

Flink runs on all UNIX-like environments, i.e. Linux, Mac OS X, and Cygwin (for Windows).
You can refer overview to check supported versions and download the binary release of Flink, then extract the archive:

tar -xzf flink-*.tgz

You should set FLINK_HOME environment variables like:

export FLINK_HOME=/path/flink-*

Start and stop a local cluster #

To start a local cluster, run the bash script that comes with Flink:

cd /path/flink-*
./bin/start-cluster.sh

Flink is now running as a background process. You can check its status with the following command:

ps aux | grep flink

You should be able to navigate to the web UI at localhost:8081 to view the Flink dashboard and see that the cluster is up and running.

To quickly stop the cluster and all running components, you can use the provided script:

./bin/stop-cluster.sh

Download the tar file of Flink CDC from release page, then extract the archive:

tar -xzf flink-cdc-*.tar.gz

Extracted flink-cdc contains four directories: bin,lib,log and conf.

Download the connector jars from release page, and move it to the lib directory.
Download links are available only for stable releases, SNAPSHOT dependencies need to be built based on specific branch by yourself.

Here is an example file for synchronizing the entire database mysql-to-doris.yaml：

################################################################################
# Description: Sync MySQL all tables to Doris
################################################################################
source:
 type: mysql
 hostname: localhost
 port: 3306
 username: root
 password: 123456
 tables: app_db.\.*
 server-id: 5400-5404
 server-time-zone: UTC

sink:
 type: doris
 fenodes: 127.0.0.1:8030
 username: root
 password: ""

pipeline:
 name: Sync MySQL Database to Doris
 parallelism: 2

You need to modify the configuration file according to your needs, refer to connectors more information.

Finally, submit job to Flink Standalone cluster using Cli.

cd /path/flink-cdc-*
./bin/flink-cdc.sh mysql-to-doris.yaml

After successful submission, the return information is as follows：

Pipeline has been submitted to cluster.
Job ID: ae30f4580f1918bebf16752d4963dc54
Job Description: Sync MySQL Database to Doris

Then you can find a job named Sync MySQL Database to Doris running through Flink Web UI.