Data Source

Definition #

Data Source is used to access metadata and read the changed data from external systems.
A Data Source can read data from multiple tables simultaneously.

Parameters #

To describe a data source, the follows are required:

parameter meaning optional/required
type The type of the source, such as mysql. required
name The name of the source, which is user-defined (a default value provided). optional
configurations of Data Source Configurations to build the Data Source e.g. connection configurations and source table properties. optional

Example #

We could use yaml files to define a mysql source:

source:
    type: mysql
    name: mysql-source   #optional,description information
    host: localhost
    port: 3306
    username: admin
    password: pass
    tables: adb.*, bdb.user_table_[0-9]+, [app|web]_order_\.*