Modifier and Type | Interface and Description |
---|---|
interface |
DeserializationSchema<T>
The deserialization schema describes how to turn the byte messages delivered by certain data
sources (for example Apache Kafka) into data types (Java/Scala objects) that are processed by
Flink.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractDeserializationSchema<T>
The deserialization schema describes how to turn the byte messages delivered by certain data
sources (for example Apache Kafka) into data types (Java/Scala objects) that are processed by
Flink.
|
class |
SimpleStringSchema
Very simple serialization schema for strings.
|
class |
TypeInformationSerializationSchema<T>
A serialization and deserialization schema that uses Flink's serialization stack to transform
typed from and to byte arrays.
|
Modifier and Type | Class and Description |
---|---|
class |
NumberSequenceSource
A data source that produces a sequence of numbers (longs).
|
Modifier and Type | Class and Description |
---|---|
class |
HadoopInputFormat<K,V>
Wrapper for using HadoopInputFormats (mapred-variant) with Flink.
|
Modifier and Type | Class and Description |
---|---|
class |
RowCsvInputFormat
Input format that reads csv into
Row . |
class |
TypeSerializerInputFormat<T>
Reads elements by deserializing them with a given type serializer.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractFileSource<T,SplitT extends FileSourceSplit>
The base class for File Sources.
|
class |
FileSource<T>
A unified data source that reads files - both in batch and in streaming mode.
|
Modifier and Type | Class and Description |
---|---|
class |
FileRecordFormatAdapter<T>
Deprecated.
Please use
StreamFormatAdapter instead. |
class |
StreamFormatAdapter<T>
Adapter to turn a
StreamFormat into a BulkFormat . |
Modifier and Type | Interface and Description |
---|---|
interface |
BulkFormat<T,SplitT extends FileSourceSplit>
The
BulkFormat reads and decodes batches of records at a time. |
interface |
FileRecordFormat<T>
Deprecated.
Please use
StreamFormat instead. The main motivation for removing it is the
inherent design flaw in the batching of FileRecordFormat: StreamFormat can guarantee that
only a certain amount of memory is being used (unless a single record exceeds that already),
but FileRecordFormat can only batch by the number of records. By removing FileRecordFormat,
we relay the responsibility of implementing the batching to the format developer; they need
to use BulkFormat and find a better way than batch by number of records. |
interface |
StreamFormat<T>
A reader format that reads individual records from a stream.
|
Modifier and Type | Class and Description |
---|---|
class |
SimpleStreamFormat<T>
A simple version of the
StreamFormat , for formats that are not splittable. |
class |
TextLineFormat
Deprecated.
Please use
TextLineInputFormat . |
class |
TextLineInputFormat
A reader format that text lines from a file.
|
Modifier and Type | Class and Description |
---|---|
class |
DeserializationSchemaAdapter
Adapter to turn a
DeserializationSchema into a BulkFormat . |
class |
LimitableBulkFormat<T,SplitT extends FileSourceSplit>
A
BulkFormat that can limit output record number. |
Modifier and Type | Class and Description |
---|---|
class |
JdbcInputFormat
InputFormat to read data from a database and generate Rows.
|
Modifier and Type | Class and Description |
---|---|
class |
JdbcRowDataInputFormat
InputFormat for
JdbcDynamicTableSource . |
Modifier and Type | Class and Description |
---|---|
class |
KafkaSource<OUT>
The Source implementation of Kafka.
|
Modifier and Type | Interface and Description |
---|---|
interface |
KafkaRecordDeserializationSchema<T>
An interface for the deserialization of Kafka records.
|
Modifier and Type | Class and Description |
---|---|
class |
PulsarSource<OUT>
The Source implementation of Pulsar.
|
Modifier and Type | Interface and Description |
---|---|
interface |
PulsarDeserializationSchema<T>
A schema bridge for deserializing the pulsar's
Message<byte[]> into a flink managed
instance. |
Modifier and Type | Class and Description |
---|---|
class |
PulsarTypeInformationWrapper<T>
Wrap the flink TypeInformation into a
PulsarDeserializationSchema . |
Modifier and Type | Interface and Description |
---|---|
interface |
DataStreamSinkExternalContext<T>
External context for DataStream sinks.
|
interface |
DataStreamSinkV1ExternalContext<T>
External context for DataStream sinks whose version is V1.
|
interface |
DataStreamSinkV2ExternalContext<T>
External context for DataStream sinks whose version is V2.
|
Modifier and Type | Interface and Description |
---|---|
interface |
DataStreamSourceExternalContext<T>
External context for DataStream sources.
|
Modifier and Type | Class and Description |
---|---|
class |
HiveSource<T>
A unified data source that reads a hive table.
|
Modifier and Type | Class and Description |
---|---|
class |
HiveBulkFormatAdapter
Deprecated.
Please use
HiveInputFormat . |
class |
HiveInputFormat
A BulkFormat implementation for HiveSource.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractAvroBulkFormat<A,T,SplitT extends FileSourceSplit>
Provides a
BulkFormat for Avro records. |
class |
AvroDeserializationSchema<T>
Deserialization schema that deserializes from Avro binary format.
|
class |
AvroInputFormat<E>
Provides a
FileInputFormat for Avro records. |
class |
AvroRowDataDeserializationSchema
Deserialization schema from Avro bytes to
RowData . |
class |
AvroRowDeserializationSchema
Deprecated.
The format was developed for the Table API users and will not be maintained for
DataStream API users anymore. Either use Table API or switch to Data Stream, defining your
own
DeserializationSchema . |
class |
RegistryAvroDeserializationSchema<T>
Deserialization schema that deserializes from Avro binary format using
SchemaCoder . |
Modifier and Type | Class and Description |
---|---|
class |
GlueSchemaRegistryAvroDeserializationSchema<T>
AWS Glue Schema Registry Deserialization schema to de-serialize Avro binary format for Flink
Consumer user.
|
Modifier and Type | Class and Description |
---|---|
class |
ConfluentRegistryAvroDeserializationSchema<T>
Deserialization schema that deserializes from Avro binary format using
SchemaCoder that
uses Confluent Schema Registry. |
Modifier and Type | Class and Description |
---|---|
class |
DebeziumAvroDeserializationSchema
Deserialization schema from Debezium Avro to Flink Table/SQL internal data structure
RowData . |
Modifier and Type | Class and Description |
---|---|
class |
CsvReaderFormat<T>
A
StreamFormat for reading CSV files. |
class |
CsvRowDataDeserializationSchema
Deserialization schema from CSV to Flink Table & SQL internal data structures.
|
class |
CsvRowDeserializationSchema
Deprecated.
The format was developed for the Table API users and will not be maintained for
DataStream API users anymore. Either use Table API or switch to Data Stream, defining your
own
DeserializationSchema . |
Modifier and Type | Class and Description |
---|---|
class |
JsonDeserializationSchema<T>
DeserializationSchema that deserializes a JSON String.
|
class |
JsonNodeDeserializationSchema
Deprecated.
Use
new JsonDeserializationSchema(ObjectNode.class) instead |
class |
JsonRowDataDeserializationSchema
Deserialization schema from JSON to Flink Table/SQL internal data structure
RowData . |
class |
JsonRowDeserializationSchema
Deprecated.
The format was developed for the Table API users and will not be maintained for
DataStream API users anymore. Either use Table API or switch to Data Stream, defining your
own
DeserializationSchema . |
Modifier and Type | Class and Description |
---|---|
class |
CanalJsonDeserializationSchema
Deserialization schema from Canal JSON to Flink Table/SQL internal data structure
RowData . |
Modifier and Type | Class and Description |
---|---|
class |
DebeziumJsonDeserializationSchema
Deserialization schema from Debezium JSON to Flink Table/SQL internal data structure
RowData . |
Modifier and Type | Class and Description |
---|---|
class |
GlueSchemaRegistryJsonDeserializationSchema<T>
AWS Glue Schema Registry Deserialization schema to de-serialize JSON Schema binary format for
Flink Consumer user.
|
Modifier and Type | Class and Description |
---|---|
class |
MaxwellJsonDeserializationSchema
Deserialization schema from Maxwell JSON to Flink Table/SQL internal data structure
RowData . |
Modifier and Type | Class and Description |
---|---|
class |
OggJsonDeserializationSchema
Deserialization schema from Ogg JSON to Flink Table/SQL internal data structure
RowData . |
Modifier and Type | Class and Description |
---|---|
class |
ParquetColumnarRowInputFormat<SplitT extends FileSourceSplit>
A
ParquetVectorizedInputFormat to provide RowData iterator. |
class |
ParquetVectorizedInputFormat<T,SplitT extends FileSourceSplit>
Parquet
BulkFormat that reads data from the file to VectorizedColumnBatch in
vectorized mode. |
Modifier and Type | Class and Description |
---|---|
class |
PbRowDataDeserializationSchema
Deserialization schema from Protobuf to Flink types.
|
Modifier and Type | Class and Description |
---|---|
class |
RawFormatDeserializationSchema
Deserialization schema from raw (byte based) value to Flink Table/SQL internal data structure
RowData . |
Modifier and Type | Class and Description |
---|---|
static class |
ConnectedComponents.CCMessenger<K,VV extends Comparable<VV>>
Sends the current vertex value to all adjacent vertices.
|
static class |
LabelPropagation.SendNewLabelToNeighbors<K,VV extends Comparable<VV>>
Sends the vertex label to all out-neighbors.
|
Modifier and Type | Class and Description |
---|---|
static class |
GraphUtils.MapTo<I,O>
Map each element to a value.
|
Modifier and Type | Class and Description |
---|---|
class |
HadoopMapFunction<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
This wrapper maps a Hadoop Mapper (mapred API) to a Flink FlatMapFunction.
|
class |
HadoopReduceCombineFunction<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
This wrapper maps a Hadoop Reducer and Combiner (mapred API) to a combinable Flink
GroupReduceFunction.
|
class |
HadoopReduceFunction<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
This wrapper maps a Hadoop Reducer (mapred API) to a non-combinable Flink GroupReduceFunction.
|
Modifier and Type | Class and Description |
---|---|
class |
HCatInputFormatBase<T>
A InputFormat to read from HCatalog tables.
|
Modifier and Type | Class and Description |
---|---|
class |
HCatInputFormat<T>
A InputFormat to read from HCatalog tables.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractOrcFileInputFormat<T,BatchT,SplitT extends FileSourceSplit>
The base for ORC readers for the
FileSource . |
class |
OrcColumnarRowFileInputFormat<BatchT,SplitT extends FileSourceSplit>
Deprecated.
Please use
OrcColumnarRowInputFormat . |
class |
OrcColumnarRowInputFormat<BatchT,SplitT extends FileSourceSplit>
An ORC reader that produces a stream of
ColumnarRowData records. |
Modifier and Type | Class and Description |
---|---|
static class |
StreamSQLTestProgram.Generator
Data-generating source function.
|
static class |
StreamSQLTestProgram.KillMapper
Kills the first execution attempt of an application when it receives the second record.
|
Modifier and Type | Interface and Description |
---|---|
interface |
DataStreamPythonFunctionOperator<OUT>
Interface for Python DataStream operators.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractEmbeddedDataStreamPythonFunctionOperator<OUT>
Base class for all Python DataStream operators executed in embedded Python environment.
|
class |
AbstractOneInputEmbeddedPythonFunctionOperator<IN,OUT>
AbstractOneInputEmbeddedPythonFunctionOperator is responsible for run Python DataStream
operators with one input in Embedded Python environment. |
class |
AbstractTwoInputEmbeddedPythonFunctionOperator<IN1,IN2,OUT>
AbstractTwoInputEmbeddedPythonFunctionOperator is responsible for run Python DataStream
operators with two input user defined python function in Embedded Python environment. |
class |
EmbeddedPythonBatchCoBroadcastProcessOperator<IN1,IN2,OUT>
The
EmbeddedPythonBatchCoBroadcastProcessOperator is responsible for executing the Python
CoBroadcastProcess Function under BATCH mode, EmbeddedPythonCoProcessOperator is used
under STREAMING mode. |
class |
EmbeddedPythonBatchKeyedCoBroadcastProcessOperator<K,IN1,IN2,OUT>
The
EmbeddedPythonBatchKeyedCoBroadcastProcessOperator is responsible for executing the
Python CoBroadcastProcess function under BATCH mode, EmbeddedPythonKeyedCoProcessOperator
is used under STREAMING mode. |
class |
EmbeddedPythonCoProcessOperator<IN1,IN2,OUT>
EmbeddedPythonCoProcessOperator is responsible for executing Python CoProcessFunction in
embedded Python environment. |
class |
EmbeddedPythonKeyedCoProcessOperator<K,IN1,IN2,OUT>
EmbeddedPythonKeyedCoProcessOperator is responsible for executing user defined python
KeyedCoProcessFunction in embedded Python environment. |
class |
EmbeddedPythonKeyedProcessOperator<K,IN,OUT>
EmbeddedPythonKeyedProcessOperator is responsible for executing user defined python
KeyedProcessFunction in embedded Python environment. |
class |
EmbeddedPythonProcessOperator<IN,OUT>
EmbeddedPythonProcessOperator is responsible for executing Python ProcessFunction in
embedded Python environment. |
class |
EmbeddedPythonWindowOperator<K,IN,OUT,W extends Window>
EmbeddedPythonWindowOperator is responsible for executing user defined python
ProcessWindowFunction in embedded Python environment. |
Modifier and Type | Class and Description |
---|---|
class |
AbstractExternalDataStreamPythonFunctionOperator<OUT>
Base class for all Python DataStream operators executed in Python process.
|
class |
AbstractExternalOneInputPythonFunctionOperator<IN,OUT>
AbstractExternalOneInputPythonFunctionOperator is responsible for launching beam runner
which will start a python harness to execute user defined python function. |
class |
AbstractExternalTwoInputPythonFunctionOperator<IN1,IN2,OUT>
AbstractExternalTwoInputPythonFunctionOperator is responsible for launching beam runner
which will start a python harness to execute two-input user defined python function. |
class |
ExternalPythonBatchCoBroadcastProcessOperator<IN1,IN2,OUT>
The
ExternalPythonBatchCoBroadcastProcessOperator is responsible for executing the Python
CoBroadcastProcess Function under BATCH mode, ExternalPythonCoProcessOperator is used
under STREAMING mode. |
class |
ExternalPythonBatchKeyedCoBroadcastProcessOperator<OUT>
The
ExternalPythonBatchKeyedCoBroadcastProcessOperator is responsible for executing the
Python CoBroadcastProcess function under BATCH mode, ExternalPythonKeyedCoProcessOperator
is used under STREAMING mode. |
class |
ExternalPythonCoProcessOperator<IN1,IN2,OUT>
The
ExternalPythonCoProcessOperator is responsible for executing the Python CoProcess
Function. |
class |
ExternalPythonKeyedCoProcessOperator<OUT>
ExternalPythonKeyedCoProcessOperator is responsible for launching beam runner which will
start a python harness to execute user defined python CoProcess function. |
class |
ExternalPythonKeyedProcessOperator<OUT>
ExternalPythonKeyedProcessOperator is responsible for launching beam runner which will
start a python harness to execute user defined python function. |
class |
ExternalPythonProcessOperator<IN,OUT>
ExternalPythonProcessOperator is responsible for launching beam runner which will start a
python harness to execute user defined python ProcessFunction. |
Modifier and Type | Class and Description |
---|---|
static class |
DelegateOperatorTransformation.DelegateOperator<OUT>
DelegateOperatorTransformation.DelegateOperator holds configurations, e.g. |
Modifier and Type | Class and Description |
---|---|
class |
PubSubSource<OUT>
PubSub Source, this Source will consume PubSub messages from a subscription and Acknowledge them
on the next checkpoint.
|
Modifier and Type | Interface and Description |
---|---|
interface |
PubSubDeserializationSchema<T>
The deserialization schema describes how to turn the PubsubMessages into data types (Java/Scala
objects) that are processed by Flink.
|
Modifier and Type | Interface and Description |
---|---|
interface |
KafkaDeserializationSchema<T>
The deserialization schema describes how to turn the Kafka ConsumerRecords into data types
(Java/Scala objects) that are processed by Flink.
|
Modifier and Type | Class and Description |
---|---|
class |
FlinkKafkaConsumer<T>
Deprecated.
|
class |
FlinkKafkaConsumerBase<T>
Base class of all Flink Kafka Consumer data sources.
|
Modifier and Type | Class and Description |
---|---|
class |
KafkaDeserializationSchemaWrapper<T>
A simple wrapper for using the DeserializationSchema with the KafkaDeserializationSchema
interface.
|
Modifier and Type | Class and Description |
---|---|
class |
FlinkKafkaShuffleConsumer<T>
Flink Kafka Shuffle Consumer Function.
|
Modifier and Type | Class and Description |
---|---|
class |
FlinkDynamoDBStreamsConsumer<T>
Consume events from DynamoDB streams.
|
class |
FlinkKinesisConsumer<T>
The Flink Kinesis Consumer is an exactly-once parallel streaming data source that subscribes to
multiple AWS Kinesis streams within the same AWS service region, and can handle resharding of
streams.
|
Modifier and Type | Interface and Description |
---|---|
interface |
KinesisDeserializationSchema<T>
This is a deserialization schema specific for the Flink Kinesis Consumer.
|
Modifier and Type | Class and Description |
---|---|
class |
DynamoDBStreamsSchema
Schema used for deserializing DynamoDB streams records.
|
class |
KinesisDeserializationSchemaWrapper<T>
A simple wrapper for using the
DeserializationSchema with the KinesisDeserializationSchema interface. |
Modifier and Type | Class and Description |
---|---|
class |
RowDataKinesisDeserializationSchema
A
KinesisDeserializationSchema adaptor for RowData records that delegates
physical data deserialization to an inner DeserializationSchema and appends requested
metadata to the end of the deserialized RowData record. |
Modifier and Type | Interface and Description |
---|---|
interface |
RMQDeserializationSchema<T>
Interface for the set of methods required to parse an RMQ delivery.
|
Modifier and Type | Class and Description |
---|---|
class |
RMQSource<OUT>
RabbitMQ source (consumer) which reads from a queue and acknowledges messages on checkpoints.
|
Modifier and Type | Class and Description |
---|---|
class |
EventDeSerializationSchema
A serializer and deserializer for the
Event type. |
Modifier and Type | Class and Description |
---|---|
class |
KafkaEventSchema
The serialization schema for the
KafkaEvent type. |
Modifier and Type | Class and Description |
---|---|
static class |
PeriodicStreamingJob.PeriodicSourceGenerator
Data-generating source function.
|
Modifier and Type | Class and Description |
---|---|
static class |
KeySelectorUtil.ArrayKeySelector<IN>
A key selector for selecting individual array fields as keys and returns them as a Tuple.
|
static class |
KeySelectorUtil.ComparableKeySelector<IN>
A key selector for selecting key fields via a TypeComparator.
|
Modifier and Type | Interface and Description |
---|---|
interface |
KeyedDeserializationSchema<T>
Deprecated.
|
Modifier and Type | Class and Description |
---|---|
class |
JSONKeyValueDeserializationSchema
DeserializationSchema that deserializes a JSON String into an ObjectNode.
|
class |
TypeInformationKeyValueSerializationSchema<K,V>
A serialization and deserialization schema for Key Value Pairs that uses Flink's serialization
stack to transform typed from and to byte arrays.
|
Modifier and Type | Class and Description |
---|---|
class |
ChangelogCsvDeserializer
The
ChangelogCsvDeserializer contains a simple parsing logic for converting bytes into
Row of Integer and String with a RowKind . |
class |
SocketSourceFunction
The
SocketSourceFunction opens a socket and consumes bytes. |
Modifier and Type | Class and Description |
---|---|
class |
ArrowSourceFunction
An Arrow
SourceFunction which takes the serialized arrow record batch data as input. |
Modifier and Type | Interface and Description |
---|---|
interface |
RowDataKeySelector
RowDataKeySelector takes an RowData and extracts the deterministic key for the RowData.
|
Modifier and Type | Class and Description |
---|---|
class |
BinaryRowDataKeySelector
A KeySelector which will extract key from RowData.
|
class |
EmptyRowDataKeySelector
A utility class which key is always empty no matter what the input row is.
|
class |
GenericRowDataKeySelector
A KeySelector which will extract key from RowData.
|
Modifier and Type | Class and Description |
---|---|
class |
FilterAllFlatMapFunction
Function filtering out all the input records.
|
class |
IntervalJoinFunction
|
class |
PaddingLeftMapFunction
Function performing left padding.
|
class |
PaddingRightMapFunction
Function performing right padding.
|
Modifier and Type | Class and Description |
---|---|
class |
ValuesInputFormat
Generated ValuesInputFormat.
|
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.