Connectors#

File System#

File Source#

FileEnumeratorProvider(...)

Factory for FileEnumerator which task is to discover all files to be read and to split them into a set of file source splits.

FileSplitAssignerProvider(j_file_split_assigner)

Factory for FileSplitAssigner which is responsible for deciding what split should be processed next by which node.

StreamFormat(j_stream_format)

A reader format that reads individual records from a stream.

FileSourceBuilder(j_file_source_builder)

The builder for the FileSource, to configure the various behaviors.

FileSource(j_file_source)

A unified data source that reads files - both in batch and in streaming mode.

File Sink#

BucketAssigner(j_bucket_assigner)

A BucketAssigner is used with a file sink to determine the bucket each incoming element should be put into.

RollingPolicy(j_rolling_policy)

The policy based on which a Bucket in the FileSink rolls its currently open part file and opens a new one.

OutputFileConfig(part_prefix, part_suffix)

Part file name configuration.

FileSink(j_file_sink)

A unified sink that emits its input elements to FileSystem files within buckets.

StreamingFileSink(j_obj)

Sink that emits its input elements to FileSystem files within buckets.

Number Sequence#

NumberSequenceSource(start, end)

A data source that produces a sequence of numbers (longs).

Kafka#

Kakfa Producer and Consumer#

FlinkKafkaConsumer(topics, ...)

The Flink Kafka Consumer is a streaming data source that pulls a parallel data stream from Apache Kafka.

FlinkKafkaProducer(topic, ...[, ...])

Flink Sink to produce data into a Kafka topic.

Pulsar#

Pulsar Source#

PulsarDeserializationSchema(...)

A schema bridge for deserializing the pulsar's Message into a flink managed instance.

SubscriptionType(value)

Types of subscription supported by Pulsar.

StartCursor(_j_start_cursor)

A factory class for users to specify the start position of a pulsar subscription.

StopCursor(_j_stop_cursor)

A factory class for users to specify the stop position of a pulsar subscription.

PulsarSource(j_pulsar_source)

The Source implementation of Pulsar.

PulsarSourceBuilder()

The builder class for PulsarSource to make it easier for the users to construct a PulsarSource.

Jdbc#

JdbcSink(j_jdbc_sink)

JdbcConnectionOptions(j_jdbc_connection_options)

JDBC connection options.

JdbcExecutionOptions(j_jdbc_execution_options)

JDBC sink batch options.

RMQ#

RMQConnectionConfig(j_rmq_connection_config)

Connection Configuration for RMQ.

RMQSource(connection_config, queue_name, ...)

RMQSink(connection_config, queue_name, ...)