@Internal public class BatchArrowPythonGroupWindowAggregateFunctionOperator extends AbstractArrowPythonAggregateFunctionOperator
AggregateFunction Operator for Group Window Aggregation.arrowSerializer, currentBatchCount, pandasAggFunctions, reuseJoinedRow, rowDataWrapperbais, baisWrapper, baos, baosWrapper, forwardedInputQueue, inputType, udfInputType, udfOutputTypepythonFunctionRunnerbundleFinishedCallback, config, elementCount, lastFinishBundleTime, maxBundleSize, systemEnvEnabledchainingStrategy, latencyStats, LOG, metrics, output, processingTimeService| Constructor and Description |
|---|
BatchArrowPythonGroupWindowAggregateFunctionOperator(Configuration config,
PythonFunctionInfo[] pandasAggFunctions,
RowType inputType,
RowType udfInputType,
RowType udfOutputType,
int inputTimeFieldIndex,
int maxLimitSize,
long windowSize,
long slideSize,
int[] namedProperties,
GeneratedProjection inputGeneratedProjection,
GeneratedProjection groupKeyGeneratedProjection,
GeneratedProjection groupSetGeneratedProjection) |
| Modifier and Type | Method and Description |
|---|---|
void |
bufferInput(RowData input)
Buffers the specified input, it will be used to construct the operator result together with
the user-defined function execution result.
|
void |
close()
This method is called at the very end of the operator's life, both in the case of a
successful completion of the operation, and in the case of a failure and canceling.
|
void |
emitResult(Tuple3<String,byte[],Integer> resultTuple)
Sends the execution result to the downstream operator.
|
void |
endInput()
It is notified that no more data will arrive from the input.
|
void |
finish()
This method is called at the end of data processing.
|
protected void |
invokeCurrentBatch() |
void |
open()
This method is called immediately before any elements are processed, it should contain the
operator's initialization logic, e.g.
|
void |
processElementInternal(RowData value) |
createInputCoderInfoDescriptor, createOutputCoderInfoDescriptor, createUserDefinedFunctionsProto, getFunctionInput, getFunctionUrn, getPythonEnv, isBundleFinished, processElementcreatePythonFunctionRunnercreatePythonEnvironmentManager, emitResults, invokeFinishBundlecheckInvokeFinishBundleByCount, getConfiguration, getFlinkMetricContainer, prepareSnapshotPreBarrier, processWatermark, setCurrentKeygetChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotStateclone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitsetKeyContextElementgetMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotStatenotifyCheckpointAborted, notifyCheckpointCompletegetCurrentKey, setCurrentKeyprocessLatencyMarker, processWatermark, processWatermarkStatushasKeyContextpublic BatchArrowPythonGroupWindowAggregateFunctionOperator(Configuration config, PythonFunctionInfo[] pandasAggFunctions, RowType inputType, RowType udfInputType, RowType udfOutputType, int inputTimeFieldIndex, int maxLimitSize, long windowSize, long slideSize, int[] namedProperties, GeneratedProjection inputGeneratedProjection, GeneratedProjection groupKeyGeneratedProjection, GeneratedProjection groupSetGeneratedProjection)
public void open()
throws Exception
AbstractStreamOperatorThe default implementation does nothing.
open in interface StreamOperator<RowData>Exception - An exception in this method causes the operator to fail.public void close()
throws Exception
StreamOperatorThis method is expected to make a thorough effort to release all resources that the operator has acquired.
NOTE:It can not emit any records! If you need to emit records at the end of
processing, do so in the StreamOperator.finish() method.
close in interface StreamOperator<RowData>close in class AbstractArrowPythonAggregateFunctionOperatorExceptionpublic void bufferInput(RowData input) throws Exception
AbstractStatelessFunctionOperatorbufferInput in class AbstractStatelessFunctionOperator<RowData,RowData,RowData>Exceptionpublic void processElementInternal(RowData value) throws Exception
processElementInternal in class AbstractStatelessFunctionOperator<RowData,RowData,RowData>Exceptionpublic void emitResult(Tuple3<String,byte[],Integer> resultTuple) throws Exception
AbstractExternalPythonFunctionOperatoremitResult in class AbstractExternalPythonFunctionOperator<RowData>Exceptionpublic void endInput()
throws Exception
BoundedOneInputWARNING: It is not safe to use this method to commit any transactions or other side
effects! You can use this method to flush any buffered data that can later on be committed
e.g. in a CheckpointListener.notifyCheckpointComplete(long).
NOTE: Given it is semantically very similar to the StreamOperator.finish()
method. It might be dropped in favour of the other method at some point in time.
endInput in interface BoundedOneInputendInput in class AbstractOneInputPythonFunctionOperator<RowData,RowData>Exceptionpublic void finish()
throws Exception
StreamOperatorThe method is expected to flush all remaining buffered data. Exceptions during this flushing of buffered data should be propagated, in order to cause the operation to be recognized as failed, because the last data items are not processed properly.
After this method is called, no more records can be produced for the downstream operators.
WARNING: It is not safe to use this method to commit any transactions or other side
effects! You can use this method to flush any buffered data that can later on be committed
e.g. in a CheckpointListener.notifyCheckpointComplete(long).
NOTE:This method does not need to close any resources. You should release external
resources in the StreamOperator.close() method.
finish in interface StreamOperator<RowData>finish in class AbstractPythonFunctionOperator<RowData>Exception - An exception in this method causes the operator to fail.Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.