Package | Description |
---|---|
org.apache.flink.streaming.api.datastream | |
org.apache.flink.streaming.api.graph | |
org.apache.flink.streaming.api.transformations | |
org.apache.flink.streaming.runtime.io | |
org.apache.flink.streaming.runtime.io.recovery | |
org.apache.flink.streaming.runtime.partitioner | |
org.apache.flink.streaming.runtime.tasks |
This package contains classes that realize streaming tasks.
|
org.apache.flink.table.runtime.partitioner |
Modifier and Type | Method and Description |
---|---|
protected DataStream<T> |
DataStream.setConnectionType(StreamPartitioner<T> partitioner)
Internal function for setting the partitioner for the DataStream.
|
protected DataStream<T> |
KeyedStream.setConnectionType(StreamPartitioner<T> partitioner) |
Modifier and Type | Method and Description |
---|---|
StreamPartitioner<?> |
StreamEdge.getPartitioner() |
Modifier and Type | Method and Description |
---|---|
void |
StreamGraph.addVirtualPartitionNode(Integer originalId,
Integer virtualId,
StreamPartitioner<?> partitioner,
ShuffleMode shuffleMode)
Adds a new virtual node that is used to connect a downstream vertex to an input with a
certain partitioning.
|
void |
StreamEdge.setPartitioner(StreamPartitioner<?> partitioner) |
Constructor and Description |
---|
StreamEdge(StreamNode sourceVertex,
StreamNode targetVertex,
int typeNumber,
long bufferTimeout,
StreamPartitioner<?> outputPartitioner,
OutputTag outputTag,
ShuffleMode shuffleMode,
int uniqueId) |
StreamEdge(StreamNode sourceVertex,
StreamNode targetVertex,
int typeNumber,
StreamPartitioner<?> outputPartitioner,
OutputTag outputTag) |
StreamEdge(StreamNode sourceVertex,
StreamNode targetVertex,
int typeNumber,
StreamPartitioner<?> outputPartitioner,
OutputTag outputTag,
ShuffleMode shuffleMode,
int uniqueId) |
Modifier and Type | Method and Description |
---|---|
StreamPartitioner<T> |
PartitionTransformation.getPartitioner()
Returns the
StreamPartitioner that must be used for partitioning the elements of the
input Transformation . |
Constructor and Description |
---|
PartitionTransformation(Transformation<T> input,
StreamPartitioner<T> partitioner)
Creates a new
PartitionTransformation from the given input and StreamPartitioner . |
PartitionTransformation(Transformation<T> input,
StreamPartitioner<T> partitioner,
ShuffleMode shuffleMode)
Creates a new
PartitionTransformation from the given input and StreamPartitioner . |
Modifier and Type | Method and Description |
---|---|
static <IN1,IN2> StreamTwoInputProcessor<IN1,IN2> |
StreamTwoInputProcessorFactory.create(AbstractInvokable ownerTask,
CheckpointedInputGate[] checkpointedInputGates,
IOManager ioManager,
MemoryManager memoryManager,
TaskIOMetricGroup taskIOMetricGroup,
StreamStatusMaintainer streamStatusMaintainer,
TwoInputStreamOperator<IN1,IN2,?> streamOperator,
WatermarkGauge input1WatermarkGauge,
WatermarkGauge input2WatermarkGauge,
BoundedMultiInput endOfInputAware,
StreamConfig streamConfig,
Configuration taskManagerConfig,
Configuration jobConfig,
ExecutionConfig executionConfig,
ClassLoader userClassloader,
Counter numRecordsIn,
InflightDataRescalingDescriptor inflightDataRescalingDescriptor,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners,
TaskInfo taskInfo) |
static StreamMultipleInputProcessor |
StreamMultipleInputProcessorFactory.create(AbstractInvokable ownerTask,
CheckpointedInputGate[] checkpointedInputGates,
StreamConfig.InputConfig[] configuredInputs,
IOManager ioManager,
MemoryManager memoryManager,
TaskIOMetricGroup ioMetricGroup,
Counter mainOperatorRecordsIn,
StreamStatusMaintainer streamStatusMaintainer,
MultipleInputStreamOperator<?> mainOperator,
WatermarkGauge[] inputWatermarkGauges,
StreamConfig streamConfig,
Configuration taskManagerConfig,
Configuration jobConfig,
ExecutionConfig executionConfig,
ClassLoader userClassloader,
OperatorChain<?,?> operatorChain,
InflightDataRescalingDescriptor inflightDataRescalingDescriptor,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners,
TaskInfo taskInfo) |
static <T> StreamTaskInput<T> |
StreamTaskNetworkInputFactory.create(CheckpointedInputGate checkpointedInputGate,
TypeSerializer<T> inputSerializer,
IOManager ioManager,
StatusWatermarkValve statusWatermarkValve,
int inputIndex,
InflightDataRescalingDescriptor rescalingDescriptorinflightDataRescalingDescriptor,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners,
TaskInfo taskInfo)
Factory method for
StreamTaskNetworkInput or RescalingStreamTaskNetworkInput
depending on InflightDataRescalingDescriptor . |
Constructor and Description |
---|
RescalingStreamTaskNetworkInput(CheckpointedInputGate checkpointedInputGate,
TypeSerializer<T> inputSerializer,
IOManager ioManager,
StatusWatermarkValve statusWatermarkValve,
int inputIndex,
InflightDataRescalingDescriptor inflightDataRescalingDescriptor,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners,
TaskInfo taskInfo) |
Modifier and Type | Class and Description |
---|---|
class |
BroadcastPartitioner<T>
Partitioner that selects all the output channels.
|
class |
CustomPartitionerWrapper<K,T>
Partitioner that selects the channel with a user defined partitioner function on a key.
|
class |
ForwardPartitioner<T>
Partitioner that forwards elements only to the locally running downstream operation.
|
class |
GlobalPartitioner<T>
Partitioner that sends all elements to the downstream operator with subtask ID=0.
|
class |
KeyGroupStreamPartitioner<T,K>
Partitioner selects the target channel based on the key group index.
|
class |
RebalancePartitioner<T>
Partitioner that distributes the data equally by cycling through the output channels.
|
class |
RescalePartitioner<T>
Partitioner that distributes the data equally by cycling through the output channels.
|
class |
ShufflePartitioner<T>
Partitioner that distributes the data equally by selecting one output channel randomly.
|
Modifier and Type | Method and Description |
---|---|
StreamPartitioner<T> |
ShufflePartitioner.copy() |
StreamPartitioner<T> |
BroadcastPartitioner.copy() |
StreamPartitioner<T> |
GlobalPartitioner.copy() |
StreamPartitioner<T> |
CustomPartitionerWrapper.copy() |
StreamPartitioner<T> |
KeyGroupStreamPartitioner.copy() |
StreamPartitioner<T> |
RescalePartitioner.copy() |
StreamPartitioner<T> |
ForwardPartitioner.copy() |
StreamPartitioner<T> |
RebalancePartitioner.copy() |
abstract StreamPartitioner<T> |
StreamPartitioner.copy() |
Modifier and Type | Method and Description |
---|---|
protected void |
MultipleInputStreamTask.createInputProcessor(List<IndexedInputGate>[] inputGates,
StreamConfig.InputConfig[] inputs,
WatermarkGauge[] inputWatermarkGauges,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners) |
protected void |
TwoInputStreamTask.createInputProcessor(List<IndexedInputGate> inputGates1,
List<IndexedInputGate> inputGates2,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners) |
protected abstract void |
AbstractTwoInputStreamTask.createInputProcessor(List<IndexedInputGate> inputGates1,
List<IndexedInputGate> inputGates2,
java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners) |
Modifier and Type | Class and Description |
---|---|
class |
BinaryHashPartitioner
Hash partitioner for
BinaryRowData . |
Modifier and Type | Method and Description |
---|---|
StreamPartitioner<RowData> |
BinaryHashPartitioner.copy() |
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.