org.apache.flink.streaming.api.operators.python.embedded

Class AbstractEmbeddedDataStreamPythonFunctionOperator<OUT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedDataStreamPythonFunctionOperator<OUT>

All Implemented Interfaces:

Serializable, CheckpointListener, ResultTypeQueryable<OUT>, KeyContext, KeyContextHandler, DataStreamPythonFunctionOperator<OUT>, SetupableStreamOperator<OUT>, StreamOperator<OUT>, StreamOperatorStateHandler.CheckpointedStreamOperator

Direct Known Subclasses:

AbstractOneInputEmbeddedPythonFunctionOperator, AbstractTwoInputEmbeddedPythonFunctionOperator

@Internal
public abstract class AbstractEmbeddedDataStreamPythonFunctionOperator<OUT>
extends AbstractEmbeddedPythonFunctionOperator<OUT>
implements DataStreamPythonFunctionOperator<OUT>

Base class for all Python DataStream operators executed in embedded Python environment.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected TimestampedCollector<OUT>	collector
protected boolean	hasSideOutput
protected TypeInformation<OUT>	outputTypeInfo The TypeInformation of output data.
protected org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedDataStreamPythonFunctionOperator.SideOutputContext	sideOutputContext

Fields inherited from class org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedPythonFunctionOperator

interpreter

Fields inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

bundleFinishedCallback, config, elementCount, lastFinishBundleTime, maxBundleSize, systemEnvEnabled

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, latencyStats, LOG, metrics, output, processingTimeService

Constructor Summary

Constructors

Constructor and Description
AbstractEmbeddedDataStreamPythonFunctionOperator(Configuration config, DataStreamPythonFunctionInfo pythonFunctionInfo, TypeInformation<OUT> outputTypeInfo)

Method Summary

Modifier and Type	Method and Description
void	addSideOutputTags(Collection<OutputTag<?>> outputTags) Add a collection of OutputTags to the operator.
Map<String,String>	getJobParameters()
TypeInformation<OUT>	getProducedType() Gets the data type (as a TypeInformation) produced by this function or input format.
DataStreamPythonFunctionInfo	getPythonFunctionInfo() Returns the underlying DataStreamPythonFunctionInfo.
Collection<OutputTag<?>>	getSideOutputTags() Gets the OutputTags belongs to the operator.
void	open() This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g.
void	setNumPartitions(int numPartitions) Sets the number of partitions.

Methods inherited from class org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedPythonFunctionOperator

close, createPythonEnvironmentManager, invokeFinishBundle, openPythonInterpreter

Methods inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

checkInvokeFinishBundleByCount, finish, getConfiguration, getFlinkMetricContainer, isBundleFinished, prepareSnapshotPreBarrier, processWatermark, setCurrentKey

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.python.DataStreamPythonFunctionOperator

copy

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Field Detail

outputTypeInfo

protected final TypeInformation<OUT> outputTypeInfo

The TypeInformation of output data.

collector

protected transient TimestampedCollector<OUT> collector

hasSideOutput

protected transient boolean hasSideOutput

sideOutputContext

protected transient org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedDataStreamPythonFunctionOperator.SideOutputContext sideOutputContext

Constructor Detail

AbstractEmbeddedDataStreamPythonFunctionOperator

public AbstractEmbeddedDataStreamPythonFunctionOperator(Configuration config,
                                                        DataStreamPythonFunctionInfo pythonFunctionInfo,
                                                        TypeInformation<OUT> outputTypeInfo)

Method Detail

open

public void open()
          throws Exception

Description copied from class: AbstractStreamOperator

This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.

The default implementation does nothing.

Specified by:

open in interface StreamOperator<OUT>

Overrides:

open in class AbstractEmbeddedPythonFunctionOperator<OUT>

Throws:

Exception - An exception in this method causes the operator to fail.

getProducedType

public TypeInformation<OUT> getProducedType()

Description copied from interface: ResultTypeQueryable

Gets the data type (as a TypeInformation) produced by this function or input format.

Specified by:

getProducedType in interface ResultTypeQueryable<OUT>

Returns:

The data type produced by this function or input format.

setNumPartitions

public void setNumPartitions(int numPartitions)

Description copied from interface: DataStreamPythonFunctionOperator

Sets the number of partitions. This is used for partitionCustom which takes the number of partitions to partition into as input.

Specified by:

setNumPartitions in interface DataStreamPythonFunctionOperator<OUT>

getPythonFunctionInfo

public DataStreamPythonFunctionInfo getPythonFunctionInfo()

Description copied from interface: DataStreamPythonFunctionOperator

Returns the underlying DataStreamPythonFunctionInfo.

Specified by:

getPythonFunctionInfo in interface DataStreamPythonFunctionOperator<OUT>

addSideOutputTags

public void addSideOutputTags(Collection<OutputTag<?>> outputTags)

Description copied from interface: DataStreamPythonFunctionOperator

Add a collection of OutputTags to the operator.

Specified by:

addSideOutputTags in interface DataStreamPythonFunctionOperator<OUT>

getSideOutputTags

public Collection<OutputTag<?>> getSideOutputTags()

Description copied from interface: DataStreamPythonFunctionOperator

Gets the OutputTags belongs to the operator.

Specified by:

getSideOutputTags in interface DataStreamPythonFunctionOperator<OUT>

getJobParameters

public Map<String,String> getJobParameters()