Modifier and Type | Method and Description |
---|---|
static <T> PatternStream<T> |
CEP.pattern(DataStream<T> input,
Pattern<T,?> pattern)
Creates a
PatternStream from an input data stream and a pattern. |
static <T> PatternStream<T> |
CEP.pattern(DataStream<T> input,
Pattern<T,?> pattern,
EventComparator<T> comparator)
Creates a
PatternStream from an input data stream and a pattern. |
Modifier and Type | Method and Description |
---|---|
DataStream<CommittableMessage<FileSinkCommittable>> |
FileSink.addPreCommitTopology(DataStream<CommittableMessage<FileSinkCommittable>> committableStream) |
Modifier and Type | Method and Description |
---|---|
DataStream<CommittableMessage<FileSinkCommittable>> |
FileSink.addPreCommitTopology(DataStream<CommittableMessage<FileSinkCommittable>> committableStream) |
Modifier and Type | Method and Description |
---|---|
static <T> DataStream<PartitionCommitInfo> |
StreamingSink.compactionWriter(ProviderContext providerContext,
DataStream<T> inputStream,
long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<T,String,? extends StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder,
FileSystemFactory fsFactory,
Path path,
CompactReader.Factory<T> readFactory,
long targetFileSize,
int parallelism)
Create a file writer with compaction operators by input stream.
|
static <T> DataStream<PartitionCommitInfo> |
StreamingSink.writer(ProviderContext providerContext,
DataStream<T> inputStream,
long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<T,String,? extends StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder,
int parallelism,
List<String> partitionKeys,
Configuration conf)
Create a file writer by input stream.
|
Modifier and Type | Method and Description |
---|---|
static <T> DataStream<PartitionCommitInfo> |
StreamingSink.compactionWriter(ProviderContext providerContext,
DataStream<T> inputStream,
long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<T,String,? extends StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder,
FileSystemFactory fsFactory,
Path path,
CompactReader.Factory<T> readFactory,
long targetFileSize,
int parallelism)
Create a file writer with compaction operators by input stream.
|
static DataStreamSink<?> |
StreamingSink.sink(ProviderContext providerContext,
DataStream<PartitionCommitInfo> writer,
Path locationPath,
ObjectIdentifier identifier,
List<String> partitionKeys,
TableMetaStoreFactory msFactory,
FileSystemFactory fsFactory,
Configuration options)
Create a sink from file writer.
|
static <T> DataStream<PartitionCommitInfo> |
StreamingSink.writer(ProviderContext providerContext,
DataStream<T> inputStream,
long bucketCheckInterval,
StreamingFileSink.BucketsBuilder<T,String,? extends StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder,
int parallelism,
List<String> partitionKeys,
Configuration conf)
Create a file writer by input stream.
|
Modifier and Type | Method and Description |
---|---|
protected SourceTestSuiteBase.CollectIteratorBuilder<T> |
SourceTestSuiteBase.addCollectSink(DataStream<T> stream)
Add a collect sink in the job.
|
protected CollectResultIterator<T> |
SinkTestSuiteBase.addCollectSink(DataStream<T> stream) |
Modifier and Type | Method and Description |
---|---|
protected DataStream<RowData> |
HiveTableSource.getDataStream(ProviderContext providerContext,
StreamExecutionEnvironment execEnv) |
Modifier and Type | Method and Description |
---|---|
DataStream<Row> |
StreamSQLTestProgram.GeneratorTableSource.getDataStream(StreamExecutionEnvironment execEnv) |
Modifier and Type | Method and Description |
---|---|
<K,T,ACC,R> |
WindowSavepointReader.aggregate(String uid,
AggregateFunction<T,ACC,R> aggregateFunction,
TypeInformation<K> keyType,
TypeInformation<ACC> accType,
TypeInformation<R> outputType)
Reads window state generated using an
AggregateFunction . |
<K,T,ACC,R> |
EvictingWindowSavepointReader.aggregate(String uid,
AggregateFunction<T,ACC,R> aggregateFunction,
TypeInformation<K> keyType,
TypeInformation<T> inputType,
TypeInformation<R> outputType)
Reads window state generated using an
AggregateFunction . |
<K,T,ACC,R,OUT> |
WindowSavepointReader.aggregate(String uid,
AggregateFunction<T,ACC,R> aggregateFunction,
WindowReaderFunction<R,OUT,K,W> readerFunction,
TypeInformation<K> keyType,
TypeInformation<ACC> accType,
TypeInformation<OUT> outputType)
Reads window state generated using an
AggregateFunction . |
<K,T,ACC,R,OUT> |
EvictingWindowSavepointReader.aggregate(String uid,
AggregateFunction<T,ACC,R> aggregateFunction,
WindowReaderFunction<R,OUT,K,W> readerFunction,
TypeInformation<K> keyType,
TypeInformation<T> inputType,
TypeInformation<OUT> outputType)
Reads window state generated using an
AggregateFunction . |
<K,T,OUT> DataStream<OUT> |
WindowSavepointReader.process(String uid,
WindowReaderFunction<T,OUT,K,W> readerFunction,
TypeInformation<K> keyType,
TypeInformation<T> stateType,
TypeInformation<OUT> outputType)
Reads window state generated without any preaggregation such as
WindowedStream#apply
and WindowedStream#process . |
<K,T,OUT> DataStream<OUT> |
EvictingWindowSavepointReader.process(String uid,
WindowReaderFunction<T,OUT,K,W> readerFunction,
TypeInformation<K> keyType,
TypeInformation<T> stateType,
TypeInformation<OUT> outputType)
Reads window state generated without any preaggregation such as
WindowedStream#apply
and WindowedStream#process . |
<K,V> DataStream<Tuple2<K,V>> |
SavepointReader.readBroadcastState(OperatorIdentifier identifier,
String name,
TypeInformation<K> keyTypeInfo,
TypeInformation<V> valueTypeInfo)
Read operator
BroadcastState from a Savepoint . |
<K,V> DataStream<Tuple2<K,V>> |
SavepointReader.readBroadcastState(OperatorIdentifier identifier,
String name,
TypeInformation<K> keyTypeInfo,
TypeInformation<V> valueTypeInfo,
TypeSerializer<K> keySerializer,
TypeSerializer<V> valueSerializer)
Read operator
BroadcastState from a Savepoint when a custom serializer was
used; e.g., a different serializer than the one returned by TypeInformation#createSerializer . |
<K,V> DataStream<Tuple2<K,V>> |
SavepointReader.readBroadcastState(String uid,
String name,
TypeInformation<K> keyTypeInfo,
TypeInformation<V> valueTypeInfo)
|
<K,V> DataStream<Tuple2<K,V>> |
SavepointReader.readBroadcastState(String uid,
String name,
TypeInformation<K> keyTypeInfo,
TypeInformation<V> valueTypeInfo,
TypeSerializer<K> keySerializer,
TypeSerializer<V> valueSerializer)
|
<K,OUT> DataStream<OUT> |
SavepointReader.readKeyedState(OperatorIdentifier identifier,
KeyedStateReaderFunction<K,OUT> function)
Read keyed state from an operator in a
Savepoint . |
<K,OUT> DataStream<OUT> |
SavepointReader.readKeyedState(OperatorIdentifier identifier,
KeyedStateReaderFunction<K,OUT> function,
TypeInformation<K> keyTypeInfo,
TypeInformation<OUT> outTypeInfo)
Read keyed state from an operator in a
Savepoint . |
<K,OUT> DataStream<OUT> |
SavepointReader.readKeyedState(String uid,
KeyedStateReaderFunction<K,OUT> function)
|
<K,OUT> DataStream<OUT> |
SavepointReader.readKeyedState(String uid,
KeyedStateReaderFunction<K,OUT> function,
TypeInformation<K> keyTypeInfo,
TypeInformation<OUT> outTypeInfo)
|
<T> DataStream<T> |
SavepointReader.readListState(OperatorIdentifier identifier,
String name,
TypeInformation<T> typeInfo)
Read operator
ListState from a Savepoint . |
<T> DataStream<T> |
SavepointReader.readListState(OperatorIdentifier identifier,
String name,
TypeInformation<T> typeInfo,
TypeSerializer<T> serializer)
Read operator
ListState from a Savepoint when a custom serializer was used;
e.g., a different serializer than the one returned by TypeInformation#createSerializer . |
<T> DataStream<T> |
SavepointReader.readListState(String uid,
String name,
TypeInformation<T> typeInfo)
|
<T> DataStream<T> |
SavepointReader.readListState(String uid,
String name,
TypeInformation<T> typeInfo,
TypeSerializer<T> serializer)
|
<T> DataStream<T> |
SavepointReader.readUnionState(OperatorIdentifier identifier,
String name,
TypeInformation<T> typeInfo)
Read operator
UnionState from a Savepoint . |
<T> DataStream<T> |
SavepointReader.readUnionState(OperatorIdentifier identifier,
String name,
TypeInformation<T> typeInfo,
TypeSerializer<T> serializer)
Read operator
UnionState from a Savepoint when a custom serializer was used;
e.g., a different serializer than the one returned by TypeInformation#createSerializer . |
<T> DataStream<T> |
SavepointReader.readUnionState(String uid,
String name,
TypeInformation<T> typeInfo)
|
<T> DataStream<T> |
SavepointReader.readUnionState(String uid,
String name,
TypeInformation<T> typeInfo,
TypeSerializer<T> serializer)
|
<T,K> DataStream<T> |
WindowSavepointReader.reduce(String uid,
ReduceFunction<T> function,
TypeInformation<K> keyType,
TypeInformation<T> reduceType)
Reads window state generated using a
ReduceFunction . |
<T,K> DataStream<T> |
EvictingWindowSavepointReader.reduce(String uid,
ReduceFunction<T> function,
TypeInformation<K> keyType,
TypeInformation<T> reduceType)
Reads window state generated using a
ReduceFunction . |
<K,T,OUT> DataStream<OUT> |
WindowSavepointReader.reduce(String uid,
ReduceFunction<T> function,
WindowReaderFunction<T,OUT,K,W> readerFunction,
TypeInformation<K> keyType,
TypeInformation<T> reduceType,
TypeInformation<OUT> outputType)
Reads window state generated using a
ReduceFunction . |
<K,T,OUT> DataStream<OUT> |
EvictingWindowSavepointReader.reduce(String uid,
ReduceFunction<T> function,
WindowReaderFunction<T,OUT,K,W> readerFunction,
TypeInformation<K> keyType,
TypeInformation<T> reduceType,
TypeInformation<OUT> outputType)
Reads window state generated using a
ReduceFunction . |
Modifier and Type | Method and Description |
---|---|
static <T> OneInputStateTransformation<T> |
OperatorTransformation.bootstrapWith(DataStream<T> stream)
Create a new
OneInputStateTransformation from a DataStream . |
Modifier and Type | Method and Description |
---|---|
DataStream<CommittableMessage<CommT>> |
WithPreCommitTopology.addPreCommitTopology(DataStream<CommittableMessage<CommT>> committables)
Intercepts and modifies the committables sent on checkpoint or at end of input.
|
DataStream<InputT> |
WithPreWriteTopology.addPreWriteTopology(DataStream<InputT> inputDataStream)
Adds an arbitrary topology before the writer.
|
Modifier and Type | Method and Description |
---|---|
static <CommT> void |
StandardSinkTopologies.addGlobalCommitter(DataStream<CommittableMessage<CommT>> committables,
SerializableSupplier<Committer<CommT>> committerFactory,
SerializableSupplier<SimpleVersionedSerializer<CommT>> committableSerializer)
Adds a global committer to the pipeline that runs as final operator with a parallelism of
one.
|
void |
WithPostCommitTopology.addPostCommitTopology(DataStream<CommittableMessage<CommT>> committables)
Adds a custom post-commit topology where all committables can be processed.
|
DataStream<CommittableMessage<CommT>> |
WithPreCommitTopology.addPreCommitTopology(DataStream<CommittableMessage<CommT>> committables)
Intercepts and modifies the committables sent on checkpoint or at end of input.
|
DataStream<InputT> |
WithPreWriteTopology.addPreWriteTopology(DataStream<InputT> inputDataStream)
Adds an arbitrary topology before the writer.
|
Modifier and Type | Class and Description |
---|---|
class |
CachedDataStream<T>
CachedDataStream represents a DataStream whose intermediate result will be cached
at the first time when it is computed. |
class |
DataStreamSource<T>
The DataStreamSource represents the starting point of a DataStream.
|
class |
IterativeStream<T>
The iterative data stream represents the start of an iteration in a
DataStream . |
class |
KeyedStream<T,KEY>
A
KeyedStream represents a DataStream on which operator state is partitioned by
key using a provided KeySelector . |
class |
SideOutputDataStream<T>
A
SideOutputDataStream represents a DataStream that contains elements that are
emitted from upstream into a side output with some tag. |
class |
SingleOutputStreamOperator<T>
SingleOutputStreamOperator represents a user defined transformation applied on a DataStream with one predefined output type. |
Modifier and Type | Field and Description |
---|---|
protected DataStream<IN1> |
ConnectedStreams.inputStream1 |
protected DataStream<IN2> |
ConnectedStreams.inputStream2 |
Modifier and Type | Method and Description |
---|---|
<T> DataStream<T> |
CoGroupedStreams.WithWindow.apply(CoGroupFunction<T1,T2,T> function)
Completes the co-group operation with the user function that is executed for windowed
groups.
|
<T> DataStream<T> |
CoGroupedStreams.WithWindow.apply(CoGroupFunction<T1,T2,T> function,
TypeInformation<T> resultType)
Completes the co-group operation with the user function that is executed for windowed
groups.
|
<T> DataStream<T> |
JoinedStreams.WithWindow.apply(FlatJoinFunction<T1,T2,T> function)
Completes the join operation with the user function that is executed for each combination
of elements with the same key in a window.
|
<T> DataStream<T> |
JoinedStreams.WithWindow.apply(FlatJoinFunction<T1,T2,T> function,
TypeInformation<T> resultType)
Completes the join operation with the user function that is executed for each combination
of elements with the same key in a window.
|
<T> DataStream<T> |
JoinedStreams.WithWindow.apply(JoinFunction<T1,T2,T> function)
Completes the join operation with the user function that is executed for each combination
of elements with the same key in a window.
|
<T> DataStream<T> |
JoinedStreams.WithWindow.apply(JoinFunction<T1,T2,T> function,
TypeInformation<T> resultType)
Completes the join operation with the user function that is executed for each combination
of elements with the same key in a window.
|
DataStream<T> |
DataStream.broadcast()
Sets the partitioning of the
DataStream so that the output elements are broadcasted
to every parallel instance of the next operation. |
DataStream<F> |
IterativeStream.ConnectedIterativeStreams.closeWith(DataStream<F> feedbackStream)
Closes the iteration.
|
DataStream<T> |
IterativeStream.closeWith(DataStream<T> feedbackStream)
Closes the iteration.
|
DataStream<T> |
DataStream.forward()
Sets the partitioning of the
DataStream so that the output elements are forwarded to
the local subtask of the next operation. |
DataStream<IN1> |
BroadcastConnectedStream.getFirstInput()
Returns the non-broadcast
DataStream . |
DataStream<IN1> |
ConnectedStreams.getFirstInput()
Returns the first
DataStream . |
DataStream<IN2> |
ConnectedStreams.getSecondInput()
Returns the second
DataStream . |
DataStream<T> |
DataStream.global()
Sets the partitioning of the
DataStream so that the output values all go to the first
instance of the next processing operator. |
<K> DataStream<T> |
DataStream.partitionCustom(Partitioner<K> partitioner,
int field)
Deprecated.
|
<K> DataStream<T> |
DataStream.partitionCustom(Partitioner<K> partitioner,
KeySelector<T,K> keySelector)
Partitions a DataStream on the key returned by the selector, using a custom partitioner.
|
<K> DataStream<T> |
DataStream.partitionCustom(Partitioner<K> partitioner,
String field)
Deprecated.
|
DataStream<T> |
DataStream.rebalance()
Sets the partitioning of the
DataStream so that the output elements are distributed
evenly to instances of the next operation in a round-robin fashion. |
DataStream<T> |
DataStream.rescale()
Sets the partitioning of the
DataStream so that the output elements are distributed
evenly to a subset of instances of the next operation in a round-robin fashion. |
protected DataStream<T> |
DataStream.setConnectionType(StreamPartitioner<T> partitioner)
Internal function for setting the partitioner for the DataStream.
|
protected DataStream<T> |
KeyedStream.setConnectionType(StreamPartitioner<T> partitioner) |
DataStream<T> |
DataStream.shuffle()
Sets the partitioning of the
DataStream so that the output elements are shuffled
uniformly randomly to the next operation. |
DataStream<T> |
DataStream.union(DataStream<T>... streams)
Creates a new
DataStream by merging DataStream outputs of the same type with
each other. |
Modifier and Type | Method and Description |
---|---|
DataStream<F> |
IterativeStream.ConnectedIterativeStreams.closeWith(DataStream<F> feedbackStream)
Closes the iteration.
|
DataStream<T> |
IterativeStream.closeWith(DataStream<T> feedbackStream)
Closes the iteration.
|
<T2> CoGroupedStreams<T,T2> |
DataStream.coGroup(DataStream<T2> otherStream)
Creates a join operation.
|
static <OUT> Iterator<OUT> |
DataStreamUtils.collect(DataStream<OUT> stream)
Deprecated.
Please use
executeAndCollect() . |
static <OUT> Iterator<OUT> |
DataStreamUtils.collect(DataStream<OUT> stream,
String executionJobName)
Deprecated.
Please use
executeAndCollect() . |
static <E> List<E> |
DataStreamUtils.collectBoundedStream(DataStream<E> stream,
String jobName)
Deprecated.
Please use
executeAndCollect() . |
static <E> List<E> |
DataStreamUtils.collectUnboundedStream(DataStream<E> stream,
int numElements,
String jobName)
Deprecated.
Please use
executeAndCollect() . |
static <OUT> ClientAndIterator<OUT> |
DataStreamUtils.collectWithClient(DataStream<OUT> stream,
String jobExecutionName)
Deprecated.
Please use
executeAndCollect() . |
<R> ConnectedStreams<T,R> |
DataStream.connect(DataStream<R> dataStream)
Creates a new
ConnectedStreams by connecting DataStream outputs of (possible)
different types with each other. |
static <T> DataStreamSink<T> |
DataStreamSink.forSink(DataStream<T> inputStream,
Sink<T> sink,
CustomSinkOperatorUidHashes customSinkOperatorUidHashes) |
static <T> DataStreamSink<T> |
DataStreamSink.forSinkV1(DataStream<T> inputStream,
Sink<T,?,?,?> sink,
CustomSinkOperatorUidHashes customSinkOperatorUidHashes) |
<T2> JoinedStreams<T,T2> |
DataStream.join(DataStream<T2> otherStream)
Creates a join operation.
|
static <IN,OUT> SingleOutputStreamOperator<OUT> |
AsyncDataStream.orderedWait(DataStream<IN> in,
AsyncFunction<IN,OUT> func,
long timeout,
TimeUnit timeUnit)
Adds an AsyncWaitOperator.
|
static <IN,OUT> SingleOutputStreamOperator<OUT> |
AsyncDataStream.orderedWait(DataStream<IN> in,
AsyncFunction<IN,OUT> func,
long timeout,
TimeUnit timeUnit,
int capacity)
Adds an AsyncWaitOperator.
|
static <IN,OUT> SingleOutputStreamOperator<OUT> |
AsyncDataStream.orderedWaitWithRetry(DataStream<IN> in,
AsyncFunction<IN,OUT> func,
long timeout,
TimeUnit timeUnit,
AsyncRetryStrategy<OUT> asyncRetryStrategy)
Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction.
|
static <IN,OUT> SingleOutputStreamOperator<OUT> |
AsyncDataStream.orderedWaitWithRetry(DataStream<IN> in,
AsyncFunction<IN,OUT> func,
long timeout,
TimeUnit timeUnit,
int capacity,
AsyncRetryStrategy<OUT> asyncRetryStrategy)
Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction.
|
static <T,K> KeyedStream<T,K> |
DataStreamUtils.reinterpretAsKeyedStream(DataStream<T> stream,
KeySelector<T,K> keySelector)
|
static <T,K> KeyedStream<T,K> |
DataStreamUtils.reinterpretAsKeyedStream(DataStream<T> stream,
KeySelector<T,K> keySelector,
TypeInformation<K> typeInfo)
|
DataStream<T> |
DataStream.union(DataStream<T>... streams)
Creates a new
DataStream by merging DataStream outputs of the same type with
each other. |
static <IN,OUT> SingleOutputStreamOperator<OUT> |
AsyncDataStream.unorderedWait(DataStream<IN> in,
AsyncFunction<IN,OUT> func,
long timeout,
TimeUnit timeUnit)
Adds an AsyncWaitOperator.
|
static <IN,OUT> SingleOutputStreamOperator<OUT> |
AsyncDataStream.unorderedWait(DataStream<IN> in,
AsyncFunction<IN,OUT> func,
long timeout,
TimeUnit timeUnit,
int capacity)
Adds an AsyncWaitOperator.
|
static <IN,OUT> SingleOutputStreamOperator<OUT> |
AsyncDataStream.unorderedWaitWithRetry(DataStream<IN> in,
AsyncFunction<IN,OUT> func,
long timeout,
TimeUnit timeUnit,
AsyncRetryStrategy<OUT> asyncRetryStrategy)
Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction.
|
static <IN,OUT> SingleOutputStreamOperator<OUT> |
AsyncDataStream.unorderedWaitWithRetry(DataStream<IN> in,
AsyncFunction<IN,OUT> func,
long timeout,
TimeUnit timeUnit,
int capacity,
AsyncRetryStrategy<OUT> asyncRetryStrategy)
Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction.
|
Constructor and Description |
---|
AllWindowedStream(DataStream<T> input,
WindowAssigner<? super T,W> windowAssigner) |
BroadcastConnectedStream(StreamExecutionEnvironment env,
DataStream<IN1> input1,
BroadcastStream<IN2> input2,
List<MapStateDescriptor<?,?>> broadcastStateDescriptors) |
BroadcastStream(StreamExecutionEnvironment env,
DataStream<T> input,
MapStateDescriptor<?,?>... broadcastStateDescriptors) |
CoGroupedStreams(DataStream<T1> input1,
DataStream<T2> input2)
Creates new CoGrouped data streams, which are the first step towards building a streaming
co-group.
|
CoGroupedStreams(DataStream<T1> input1,
DataStream<T2> input2)
Creates new CoGrouped data streams, which are the first step towards building a streaming
co-group.
|
ConnectedIterativeStreams(DataStream<I> input,
TypeInformation<F> feedbackType,
long waitTime) |
ConnectedStreams(StreamExecutionEnvironment env,
DataStream<IN1> input1,
DataStream<IN2> input2) |
ConnectedStreams(StreamExecutionEnvironment env,
DataStream<IN1> input1,
DataStream<IN2> input2) |
IterativeStream(DataStream<T> dataStream,
long maxWaitTime) |
JoinedStreams(DataStream<T1> input1,
DataStream<T2> input2)
Creates new JoinedStreams data streams, which are the first step towards building a streaming
co-group.
|
JoinedStreams(DataStream<T1> input1,
DataStream<T2> input2)
Creates new JoinedStreams data streams, which are the first step towards building a streaming
co-group.
|
KeyedStream(DataStream<T> dataStream,
KeySelector<T,KEY> keySelector)
Creates a new
KeyedStream using the given KeySelector to partition operator
state by key. |
KeyedStream(DataStream<T> dataStream,
KeySelector<T,KEY> keySelector,
TypeInformation<KEY> keyType)
Creates a new
KeyedStream using the given KeySelector to partition operator
state by key. |
StreamProjection(DataStream<IN> dataStream,
int[] fieldIndexes) |
WithWindow(DataStream<T1> input1,
DataStream<T2> input2,
KeySelector<T1,KEY> keySelector1,
KeySelector<T2,KEY> keySelector2,
TypeInformation<KEY> keyType,
WindowAssigner<? super CoGroupedStreams.TaggedUnion<T1,T2>,W> windowAssigner,
Trigger<? super CoGroupedStreams.TaggedUnion<T1,T2>,? super W> trigger,
Evictor<? super CoGroupedStreams.TaggedUnion<T1,T2>,? super W> evictor,
Time allowedLateness) |
WithWindow(DataStream<T1> input1,
DataStream<T2> input2,
KeySelector<T1,KEY> keySelector1,
KeySelector<T2,KEY> keySelector2,
TypeInformation<KEY> keyType,
WindowAssigner<? super CoGroupedStreams.TaggedUnion<T1,T2>,W> windowAssigner,
Trigger<? super CoGroupedStreams.TaggedUnion<T1,T2>,? super W> trigger,
Evictor<? super CoGroupedStreams.TaggedUnion<T1,T2>,? super W> evictor,
Time allowedLateness) |
WithWindow(DataStream<T1> input1,
DataStream<T2> input2,
KeySelector<T1,KEY> keySelector1,
KeySelector<T2,KEY> keySelector2,
TypeInformation<KEY> keyType,
WindowAssigner<? super CoGroupedStreams.TaggedUnion<T1,T2>,W> windowAssigner,
Trigger<? super CoGroupedStreams.TaggedUnion<T1,T2>,? super W> trigger,
Evictor<? super CoGroupedStreams.TaggedUnion<T1,T2>,? super W> evictor,
Time allowedLateness) |
WithWindow(DataStream<T1> input1,
DataStream<T2> input2,
KeySelector<T1,KEY> keySelector1,
KeySelector<T2,KEY> keySelector2,
TypeInformation<KEY> keyType,
WindowAssigner<? super CoGroupedStreams.TaggedUnion<T1,T2>,W> windowAssigner,
Trigger<? super CoGroupedStreams.TaggedUnion<T1,T2>,? super W> trigger,
Evictor<? super CoGroupedStreams.TaggedUnion<T1,T2>,? super W> evictor,
Time allowedLateness) |
Modifier and Type | Method and Description |
---|---|
DataStream<String> |
StreamExecutionEnvironment.readFileStream(String filePath,
long intervalMillis,
FileMonitoringFunction.WatchType watchType)
Deprecated.
|
Constructor and Description |
---|
CollectStreamSink(DataStream<T> inputStream,
CollectSinkOperatorFactory<T> factory) |
Modifier and Type | Method and Description |
---|---|
DataStream<InputT> |
SinkTransformation.getInputStream() |
Constructor and Description |
---|
SinkTransformation(DataStream<InputT> inputStream,
Sink<InputT> sink,
TypeInformation<OutputT> outputType,
String name,
int parallelism,
CustomSinkOperatorUidHashes customSinkOperatorUidHashes) |
Modifier and Type | Field and Description |
---|---|
protected DataStream<IN> |
CassandraSink.CassandraSinkBuilder.input |
Modifier and Type | Method and Description |
---|---|
static <IN> CassandraSink.CassandraSinkBuilder<IN> |
CassandraSink.addSink(DataStream<IN> input)
Writes a DataStream into a Cassandra database.
|
DataStreamSink<?> |
CassandraAppendTableSink.consumeDataStream(DataStream<Row> dataStream) |
Constructor and Description |
---|
CassandraPojoSinkBuilder(DataStream<IN> input,
TypeInformation<IN> typeInfo,
TypeSerializer<IN> serializer) |
CassandraRowSinkBuilder(DataStream<Row> input,
TypeInformation<Row> typeInfo,
TypeSerializer<Row> serializer) |
CassandraScalaProductSinkBuilder(DataStream<IN> input,
TypeInformation<IN> typeInfo,
TypeSerializer<IN> serializer) |
CassandraSinkBuilder(DataStream<IN> input,
TypeInformation<IN> typeInfo,
TypeSerializer<IN> serializer) |
CassandraTupleSinkBuilder(DataStream<IN> input,
TypeInformation<IN> typeInfo,
TypeSerializer<IN> serializer) |
Modifier and Type | Method and Description |
---|---|
static <T> KeyedStream<T,Tuple> |
FlinkKafkaShuffle.persistentKeyBy(DataStream<T> dataStream,
String topic,
int producerParallelism,
int numberOfPartitions,
Properties properties,
int... fields)
Uses Kafka as a message bus to persist keyBy shuffle.
|
static <T,K> KeyedStream<T,K> |
FlinkKafkaShuffle.persistentKeyBy(DataStream<T> dataStream,
String topic,
int producerParallelism,
int numberOfPartitions,
Properties properties,
KeySelector<T,K> keySelector)
Uses Kafka as a message bus to persist keyBy shuffle.
|
static <T> void |
FlinkKafkaShuffle.writeKeyBy(DataStream<T> dataStream,
String topic,
Properties kafkaProperties,
int... fields)
|
static <T,K> void |
FlinkKafkaShuffle.writeKeyBy(DataStream<T> dataStream,
String topic,
Properties kafkaProperties,
KeySelector<T,K> keySelector)
|
Modifier and Type | Method and Description |
---|---|
static DataStream<Tuple2<String,Integer>> |
WindowJoinSampleData.GradeSource.getSource(StreamExecutionEnvironment env,
long rate) |
static DataStream<Tuple2<String,Integer>> |
WindowJoinSampleData.SalarySource.getSource(StreamExecutionEnvironment env,
long rate) |
static DataStream<Tuple3<String,Integer,Integer>> |
WindowJoin.runWindowJoin(DataStream<Tuple2<String,Integer>> grades,
DataStream<Tuple2<String,Integer>> salaries,
long windowSize) |
Modifier and Type | Method and Description |
---|---|
static DataStream<Tuple3<String,Integer,Integer>> |
WindowJoin.runWindowJoin(DataStream<Tuple2<String,Integer>> grades,
DataStream<Tuple2<String,Integer>> salaries,
long windowSize) |
static DataStream<Tuple3<String,Integer,Integer>> |
WindowJoin.runWindowJoin(DataStream<Tuple2<String,Integer>> grades,
DataStream<Tuple2<String,Integer>> salaries,
long windowSize) |
Modifier and Type | Method and Description |
---|---|
protected <T> DataStream<T> |
AbstractStreamTableEnvironmentImpl.toStreamInternal(Table table,
ModifyOperation modifyOperation) |
protected <T> DataStream<T> |
AbstractStreamTableEnvironmentImpl.toStreamInternal(Table table,
SchemaTranslator.ProducingResult schemaTranslationResult,
ChangelogMode changelogMode) |
Modifier and Type | Method and Description |
---|---|
protected <T> DataStreamQueryOperation<T> |
AbstractStreamTableEnvironmentImpl.asQueryOperation(DataStream<T> dataStream,
Optional<List<Expression>> fields) |
protected <T> Table |
AbstractStreamTableEnvironmentImpl.fromStreamInternal(DataStream<T> dataStream,
Schema schema,
String viewPath,
ChangelogMode changelogMode) |
Modifier and Type | Method and Description |
---|---|
<T> DataStream<T> |
StreamTableEnvironment.toAppendStream(Table table,
Class<T> clazz)
Deprecated.
Use
StreamTableEnvironment.toDataStream(Table, Class) instead. It integrates with the new type
system and supports all kinds of DataTypes that the table runtime can produce.
The semantics might be slightly different for raw and structured types. Use toDataStream(DataTypes.of(TypeInformation.of(Class))) if TypeInformation should
be used as source of truth. |
<T> DataStream<T> |
StreamTableEnvironment.toAppendStream(Table table,
TypeInformation<T> typeInfo)
Deprecated.
Use
StreamTableEnvironment.toDataStream(Table, Class) instead. It integrates with the new type
system and supports all kinds of DataTypes that the table runtime can produce.
The semantics might be slightly different for raw and structured types. Use toDataStream(DataTypes.of(TypeInformation.of(Class))) if TypeInformation should
be used as source of truth. |
DataStream<Row> |
StreamTableEnvironment.toChangelogStream(Table table)
Converts the given
Table into a DataStream of changelog entries. |
DataStream<Row> |
StreamTableEnvironment.toChangelogStream(Table table,
Schema targetSchema)
Converts the given
Table into a DataStream of changelog entries. |
DataStream<Row> |
StreamTableEnvironment.toChangelogStream(Table table,
Schema targetSchema,
ChangelogMode changelogMode)
Converts the given
Table into a DataStream of changelog entries. |
DataStream<Row> |
StreamTableEnvironment.toDataStream(Table table)
Converts the given
Table into a DataStream . |
<T> DataStream<T> |
StreamTableEnvironment.toDataStream(Table table,
AbstractDataType<?> targetDataType)
|
<T> DataStream<T> |
StreamTableEnvironment.toDataStream(Table table,
Class<T> targetClass)
|
<T> DataStream<Tuple2<Boolean,T>> |
StreamTableEnvironment.toRetractStream(Table table,
Class<T> clazz)
Deprecated.
Use
StreamTableEnvironment.toChangelogStream(Table, Schema) instead. It integrates with the new
type system and supports all kinds of DataTypes and every ChangelogMode
that the table runtime can produce. |
<T> DataStream<Tuple2<Boolean,T>> |
StreamTableEnvironment.toRetractStream(Table table,
TypeInformation<T> typeInfo)
Deprecated.
Use
StreamTableEnvironment.toChangelogStream(Table, Schema) instead. It integrates with the new
type system and supports all kinds of DataTypes and every ChangelogMode
that the table runtime can produce. |
Modifier and Type | Method and Description |
---|---|
<T> void |
StreamTableEnvironment.createTemporaryView(String path,
DataStream<T> dataStream)
Creates a view from the given
DataStream in a given path. |
<T> void |
StreamTableEnvironment.createTemporaryView(String path,
DataStream<T> dataStream,
Expression... fields)
Deprecated.
Use
StreamTableEnvironment.createTemporaryView(String, DataStream, Schema) instead. In most
cases, StreamTableEnvironment.createTemporaryView(String, DataStream) should already be sufficient. It
integrates with the new type system and supports all kinds of DataTypes that the
table runtime can consume. The semantics might be slightly different for raw and
structured types. |
<T> void |
StreamTableEnvironment.createTemporaryView(String path,
DataStream<T> dataStream,
Schema schema)
Creates a view from the given
DataStream in a given path. |
Table |
StreamTableEnvironment.fromChangelogStream(DataStream<Row> dataStream)
Converts the given
DataStream of changelog entries into a Table . |
Table |
StreamTableEnvironment.fromChangelogStream(DataStream<Row> dataStream,
Schema schema)
Converts the given
DataStream of changelog entries into a Table . |
Table |
StreamTableEnvironment.fromChangelogStream(DataStream<Row> dataStream,
Schema schema,
ChangelogMode changelogMode)
Converts the given
DataStream of changelog entries into a Table . |
<T> Table |
StreamTableEnvironment.fromDataStream(DataStream<T> dataStream)
Converts the given
DataStream into a Table . |
<T> Table |
StreamTableEnvironment.fromDataStream(DataStream<T> dataStream,
Expression... fields)
Deprecated.
Use
StreamTableEnvironment.fromDataStream(DataStream, Schema) instead. In most cases, StreamTableEnvironment.fromDataStream(DataStream) should already be sufficient. It integrates with the new
type system and supports all kinds of DataTypes that the table runtime can
consume. The semantics might be slightly different for raw and structured types. |
<T> Table |
StreamTableEnvironment.fromDataStream(DataStream<T> dataStream,
Schema schema)
Converts the given
DataStream into a Table . |
<T> void |
StreamTableEnvironment.registerDataStream(String name,
DataStream<T> dataStream)
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
<T> DataStream<T> |
StreamTableEnvironmentImpl.toAppendStream(Table table,
Class<T> clazz) |
<T> DataStream<T> |
StreamTableEnvironmentImpl.toAppendStream(Table table,
TypeInformation<T> typeInfo) |
DataStream<Row> |
StreamTableEnvironmentImpl.toChangelogStream(Table table) |
DataStream<Row> |
StreamTableEnvironmentImpl.toChangelogStream(Table table,
Schema targetSchema) |
DataStream<Row> |
StreamTableEnvironmentImpl.toChangelogStream(Table table,
Schema targetSchema,
ChangelogMode changelogMode) |
DataStream<Row> |
StreamTableEnvironmentImpl.toDataStream(Table table) |
<T> DataStream<T> |
StreamTableEnvironmentImpl.toDataStream(Table table,
AbstractDataType<?> targetDataType) |
<T> DataStream<T> |
StreamTableEnvironmentImpl.toDataStream(Table table,
Class<T> targetClass) |
<T> DataStream<Tuple2<Boolean,T>> |
StreamTableEnvironmentImpl.toRetractStream(Table table,
Class<T> clazz) |
<T> DataStream<Tuple2<Boolean,T>> |
StreamTableEnvironmentImpl.toRetractStream(Table table,
TypeInformation<T> typeInfo) |
Modifier and Type | Method and Description |
---|---|
<T> void |
StreamTableEnvironmentImpl.createTemporaryView(String path,
DataStream<T> dataStream) |
<T> void |
StreamTableEnvironmentImpl.createTemporaryView(String path,
DataStream<T> dataStream,
Expression... fields) |
<T> void |
StreamTableEnvironmentImpl.createTemporaryView(String path,
DataStream<T> dataStream,
Schema schema) |
Table |
StreamTableEnvironmentImpl.fromChangelogStream(DataStream<Row> dataStream) |
Table |
StreamTableEnvironmentImpl.fromChangelogStream(DataStream<Row> dataStream,
Schema schema) |
Table |
StreamTableEnvironmentImpl.fromChangelogStream(DataStream<Row> dataStream,
Schema schema,
ChangelogMode changelogMode) |
<T> Table |
StreamTableEnvironmentImpl.fromDataStream(DataStream<T> dataStream) |
<T> Table |
StreamTableEnvironmentImpl.fromDataStream(DataStream<T> dataStream,
Expression... fields) |
<T> Table |
StreamTableEnvironmentImpl.fromDataStream(DataStream<T> dataStream,
Schema schema) |
<T> void |
StreamTableEnvironmentImpl.registerDataStream(String name,
DataStream<T> dataStream) |
Modifier and Type | Method and Description |
---|---|
default DataStreamSink<?> |
DataStreamSinkProvider.consumeDataStream(DataStream<RowData> dataStream)
Deprecated.
Use
DataStreamSinkProvider.consumeDataStream(ProviderContext, DataStream)
and correctly set a unique identifier for each data stream transformation. |
default DataStreamSink<?> |
DataStreamSinkProvider.consumeDataStream(ProviderContext providerContext,
DataStream<RowData> dataStream)
Consumes the given Java
DataStream and returns the sink transformation DataStreamSink . |
Modifier and Type | Method and Description |
---|---|
default DataStream<RowData> |
DataStreamScanProvider.produceDataStream(ProviderContext providerContext,
StreamExecutionEnvironment execEnv)
Creates a scan Java
DataStream from a StreamExecutionEnvironment . |
default DataStream<RowData> |
DataStreamScanProvider.produceDataStream(StreamExecutionEnvironment execEnv)
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
DataStream<E> |
ExternalQueryOperation.getDataStream() |
DataStream<E> |
DataStreamQueryOperation.getDataStream()
Deprecated.
|
Constructor and Description |
---|
DataStreamQueryOperation(DataStream<E> dataStream,
int[] fieldIndices,
ResolvedSchema resolvedSchema)
Deprecated.
|
DataStreamQueryOperation(ObjectIdentifier identifier,
DataStream<E> dataStream,
int[] fieldIndices,
ResolvedSchema resolvedSchema)
Deprecated.
|
ExternalQueryOperation(ContextResolvedTable contextResolvedTable,
DataStream<E> dataStream,
DataType physicalDataType,
boolean isTopLevelRecord,
ChangelogMode changelogMode) |
Modifier and Type | Method and Description |
---|---|
static org.apache.calcite.rel.RelNode |
DynamicSourceUtils.convertDataStreamToRel(boolean isBatchMode,
ReadableConfig config,
FlinkRelBuilder relBuilder,
ContextResolvedTable contextResolvedTable,
DataStream<?> dataStream,
DataType physicalDataType,
boolean isTopLevelRecord,
ChangelogMode changelogMode)
Converts a given
DataStream to a RelNode . |
Modifier and Type | Method and Description |
---|---|
DataStream<E> |
InternalDataStreamQueryOperation.getDataStream()
Deprecated.
|
Constructor and Description |
---|
InternalDataStreamQueryOperation(ObjectIdentifier identifier,
DataStream<E> dataStream,
int[] fieldIndices,
ResolvedSchema resolvedSchema,
boolean[] fieldNullables,
org.apache.flink.table.planner.plan.stats.FlinkStatistic statistic)
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
DataStream<?> |
BatchExecBoundedStreamScan.getDataStream() |
Constructor and Description |
---|
BatchExecBoundedStreamScan(ReadableConfig tableConfig,
DataStream<?> dataStream,
DataType sourceType,
int[] fieldIndexes,
List<String> qualifiedName,
RowType outputType,
String description) |
Modifier and Type | Method and Description |
---|---|
DataStream<?> |
StreamExecDataStreamScan.getDataStream() |
Constructor and Description |
---|
StreamExecDataStreamScan(ReadableConfig tableConfig,
DataStream<?> dataStream,
DataType sourceType,
int[] fieldIndexes,
String[] fieldNames,
List<String> qualifiedName,
RowType outputType,
String description) |
Modifier and Type | Method and Description |
---|---|
DataStream<RowData> |
ArrowTableSource.getDataStream(StreamExecutionEnvironment execEnv) |
Modifier and Type | Method and Description |
---|---|
DataStreamSink<?> |
CsvTableSink.consumeDataStream(DataStream<Row> dataStream)
Deprecated.
|
DataStreamSink<?> |
StreamTableSink.consumeDataStream(DataStream<T> dataStream)
Deprecated.
Consumes the DataStream and return the sink transformation
DataStreamSink . |
DataStreamSink<T> |
OutputFormatTableSink.consumeDataStream(DataStream<T> dataStream)
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
DataStream<T> |
StreamTableSource.getDataStream(StreamExecutionEnvironment execEnv)
Deprecated.
Returns the data of the table as a
DataStream . |
DataStream<Row> |
CsvTableSource.getDataStream(StreamExecutionEnvironment execEnv)
Deprecated.
|
DataStream<T> |
InputFormatTableSource.getDataStream(StreamExecutionEnvironment execEnv)
Deprecated.
|
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.