org.apache.flink.streaming.api.operators.python.embedded

Class AbstractEmbeddedPythonFunctionOperator<OUT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedPythonFunctionOperator<OUT>

All Implemented Interfaces:

Serializable, CheckpointListener, KeyContext, KeyContextHandler, SetupableStreamOperator<OUT>, StreamOperator<OUT>, StreamOperatorStateHandler.CheckpointedStreamOperator, YieldingOperator<OUT>

Direct Known Subclasses:

AbstractEmbeddedDataStreamPythonFunctionOperator, AbstractEmbeddedStatelessFunctionOperator

@Internal
public abstract class AbstractEmbeddedPythonFunctionOperator<OUT>
extends AbstractPythonFunctionOperator<OUT>

Abstract class for all stream operators to execute Python functions in embedded Python environment.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected pemja.core.PythonInterpreter	interpreter Every operator will hold the only python interpreter.

Fields inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

bundleFinishedCallback, config, elementCount, lastFinishBundleTime, maxBundleSize, systemEnvEnabled

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

Constructor Summary

Constructors

Constructor and Description
AbstractEmbeddedPythonFunctionOperator(Configuration config)

Method Summary

Modifier and Type	Method and Description
void	close() This method is called at the very end of the operator's life, both in the case of a successful completion of the operation, and in the case of a failure and canceling.
protected EmbeddedPythonEnvironmentManager	createPythonEnvironmentManager()
protected void	invokeFinishBundle()
void	open() This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.
abstract void	openPythonInterpreter() Setup method for Python Interpreter.

Methods inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

checkInvokeFinishBundleByCount, finish, getConfiguration, getFlinkMetricContainer, isBundleFinished, prepareSnapshotPreBarrier, processWatermark, setCurrentKey

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState, useSplittableTimers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

getOperatorAttributes

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Field Detail

interpreter

protected transient pemja.core.PythonInterpreter interpreter

Every operator will hold the only python interpreter.

Constructor Detail

AbstractEmbeddedPythonFunctionOperator

public AbstractEmbeddedPythonFunctionOperator(Configuration config)

Method Detail

open

public void open()
          throws Exception

Description copied from class: AbstractStreamOperator

This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.

The default implementation does nothing.

Specified by:

open in interface StreamOperator<OUT>

Overrides:

open in class AbstractPythonFunctionOperator<OUT>

Throws:

Exception - An exception in this method causes the operator to fail.

close

public void close()
           throws Exception

Description copied from interface: StreamOperator

This method is called at the very end of the operator's life, both in the case of a successful completion of the operation, and in the case of a failure and canceling.

This method is expected to make a thorough effort to release all resources that the operator has acquired.

NOTE:It can not emit any records! If you need to emit records at the end of processing, do so in the StreamOperator.finish() method.

Specified by:

close in interface StreamOperator<OUT>

Overrides:

close in class AbstractPythonFunctionOperator<OUT>

Throws:

Exception

createPythonEnvironmentManager

protected EmbeddedPythonEnvironmentManager createPythonEnvironmentManager()

Specified by:

createPythonEnvironmentManager in class AbstractPythonFunctionOperator<OUT>

invokeFinishBundle

protected void invokeFinishBundle()
                           throws Exception

Specified by:

invokeFinishBundle in class AbstractPythonFunctionOperator<OUT>

Throws:

Exception

openPythonInterpreter

public abstract void openPythonInterpreter()

Setup method for Python Interpreter.