org.apache.flink.streaming.runtime.tasks

Class SourceStreamTask<OUT,SRC extends SourceFunction<OUT>,OP extends StreamSource<OUT,SRC>>

java.lang.Object

org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable

org.apache.flink.streaming.runtime.tasks.StreamTask<OUT,OP>

org.apache.flink.streaming.runtime.tasks.SourceStreamTask<OUT,SRC,OP>

Type Parameters:

OUT - Type of the output elements of this source.

SRC - Type of the source function for the stream source operator

OP - Type of the stream source operator

All Implemented Interfaces:

AsyncExceptionHandler

@Internal
public class SourceStreamTask<OUT,SRC extends SourceFunction<OUT>,OP extends StreamSource<OUT,SRC>>
extends StreamTask<OUT,OP>

StreamTask for executing a StreamSource.

One important aspect of this is that the checkpointing and the emission of elements must never occur at the same time. The execution must be serial. This is achieved by having the contract with the SourceFunction that it must only modify its state or emit elements in a synchronized block that locks on the lock Object. Also, the modification of the state and the emission of elements must happen in the same block of code that is protected by the synchronized block.

Field Summary

Fields inherited from class org.apache.flink.streaming.runtime.tasks.StreamTask

configuration, headOperator, inputProcessor, LOG, mailboxProcessor, operatorChain, stateBackend, timerService, TRIGGER_THREAD_GROUP

Constructor Summary

Constructors

Constructor and Description
SourceStreamTask(Environment env)

Method Summary

Modifier and Type	Method and Description
protected void	advanceToEndOfEventTime() Emits the MAX_WATERMARK so that all registered timers are fired.
protected void	cancelTask()
protected void	cleanup()
protected void	cleanUpInvoke()
protected void	declineCheckpoint(long checkpointId)
protected void	finishTask() Instructs the task to go through its normal termination routine, i.e.
protected CompletableFuture<Void>	getCompletionFuture()
protected void	init()
protected void	processInput(MailboxDefaultAction.Controller controller) This method implements the default action of the task (e.g.
Future<Boolean>	triggerCheckpointAsync(CheckpointMetaData checkpointMetaData, CheckpointOptions checkpointOptions) This method is called to trigger a checkpoint, asynchronously by the checkpoint coordinator.

Methods inherited from class org.apache.flink.streaming.runtime.tasks.StreamTask

abortCheckpointOnBarrier, afterInvoke, beforeInvoke, cancel, createRecordWriterDelegate, createStreamTaskStateInitializer, dispatchOperatorEvent, executeInTaskThread, finalize, getAsyncCheckpointStartDelayNanos, getAsyncOperationsThreadPool, getCancelables, getCheckpointStorage, getConfiguration, getMailboxExecutorFactory, getName, getProcessingTimeServiceFactory, getStreamStatusMaintainer, handleAsyncException, invoke, isCanceled, isFailing, isMailboxLoopRunning, isRunning, notifyCheckpointAbortAsync, notifyCheckpointCompleteAsync, runMailboxStep, setupNumRecordsInCounter, toString, triggerCheckpointOnBarrier

Methods inherited from class org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable

getCurrentNumberOfSubtasks, getEnvironment, getExecutionConfig, getIndexInSubtaskGroup, getJobConfiguration, getTaskConfiguration, getUserCodeClassLoader, setShouldInterruptOnCancel, shouldInterruptOnCancel

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

SourceStreamTask

public SourceStreamTask(Environment env)
                 throws Exception

Throws:

Exception

Method Detail

init

protected void init()

Specified by:

init in class StreamTask<OUT,OP extends StreamSource<OUT,SRC>>

advanceToEndOfEventTime

protected void advanceToEndOfEventTime()
                                throws Exception

Description copied from class: StreamTask

Emits the MAX_WATERMARK so that all registered timers are fired.

This is used by the source task when the job is TERMINATED. In the case, we want all the timers registered throughout the pipeline to fire and the related state (e.g. windows) to be flushed.

For tasks other than the source task, this method does nothing.

Overrides:

advanceToEndOfEventTime in class StreamTask<OUT,OP extends StreamSource<OUT,SRC>>

Throws:

Exception

cleanup

protected void cleanup()

Overrides:

cleanup in class StreamTask<OUT,OP extends StreamSource<OUT,SRC>>

processInput

protected void processInput(MailboxDefaultAction.Controller controller)
                     throws Exception

Description copied from class: StreamTask

This method implements the default action of the task (e.g. processing one event from the input). Implementations should (in general) be non-blocking.

Overrides:

processInput in class StreamTask<OUT,OP extends StreamSource<OUT,SRC>>

Parameters:

controller - controller object for collaborative interaction between the action and the stream task.

Throws:

Exception - on any problems in the action.

cleanUpInvoke

protected void cleanUpInvoke()
                      throws Exception

Overrides:

cleanUpInvoke in class StreamTask<OUT,OP extends StreamSource<OUT,SRC>>

Throws:

Exception

cancelTask

protected void cancelTask()

Overrides:

cancelTask in class StreamTask<OUT,OP extends StreamSource<OUT,SRC>>

finishTask

protected void finishTask()

Description copied from class: StreamTask

Instructs the task to go through its normal termination routine, i.e. exit the run-loop and call StreamOperator.close() and StreamOperator.dispose() on its operators.

This is used by the source task to get out of the run-loop when the job is stopped with a savepoint.

For tasks other than the source task, this method does nothing.

Overrides:

finishTask in class StreamTask<OUT,OP extends StreamSource<OUT,SRC>>

getCompletionFuture

protected CompletableFuture<Void> getCompletionFuture()

Overrides:

getCompletionFuture in class StreamTask<OUT,OP extends StreamSource<OUT,SRC>>

triggerCheckpointAsync

public Future<Boolean> triggerCheckpointAsync(CheckpointMetaData checkpointMetaData,
                                              CheckpointOptions checkpointOptions)

Description copied from class: AbstractInvokable

This method is called to trigger a checkpoint, asynchronously by the checkpoint coordinator.

This method is called for tasks that start the checkpoints by injecting the initial barriers, i.e., the source tasks. In contrast, checkpoints on downstream operators, which are the result of receiving checkpoint barriers, invoke the AbstractInvokable.triggerCheckpointOnBarrier(CheckpointMetaData, CheckpointOptions, CheckpointMetrics) method.

Overrides:

triggerCheckpointAsync in class StreamTask<OUT,OP extends StreamSource<OUT,SRC>>

Parameters:

checkpointMetaData - Meta data for about this checkpoint

checkpointOptions - Options for performing this checkpoint

Returns:

future with value of false if the checkpoint was not carried out, true otherwise

declineCheckpoint

protected void declineCheckpoint(long checkpointId)

Overrides:

declineCheckpoint in class StreamTask<OUT,OP extends StreamSource<OUT,SRC>>