org.apache.flink.runtime.iterative.task

Class AbstractIterativeTask<S extends Function,OT>

java.lang.Object

org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable

org.apache.flink.runtime.operators.BatchTask<S,OT>

org.apache.flink.runtime.iterative.task.AbstractIterativeTask<S,OT>

All Implemented Interfaces:

Terminable, TaskContext<S,OT>

Direct Known Subclasses:

IterationHeadTask, IterationIntermediateTask, IterationTailTask

public abstract class AbstractIterativeTask<S extends Function,OT>
extends BatchTask<S,OT>
implements Terminable

The abstract base class for all tasks able to participate in an iteration.

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	isSolutionSetUpdate
protected boolean	isWorksetIteration
protected boolean	isWorksetUpdate
protected LongSumAggregator	worksetAggregator
protected BlockingBackChannel	worksetBackChannel

Fields inherited from class org.apache.flink.runtime.operators.BatchTask

accumulatorMap, broadcastInputReaders, broadcastInputSerializers, chainedTasks, config, driver, eventualOutputs, inputComparators, inputIterators, inputReaders, inputs, inputSerializers, iterativeBroadcastInputs, iterativeInputs, localStrategies, LOG, output, resettableInputs, running, runtimeUdfContext, stub, tempBarriers

Constructor Summary

Constructors

Constructor and Description
AbstractIterativeTask(Environment environment) Create an Invokable task and set its environment.

Method Summary

Modifier and Type	Method and Description
String	brokerKey()
Future<Void>	cancel() This method is called when a task is canceled either as a result of a user abort or an execution failure.
protected void	closeLocalStrategiesAndCaches()
DistributedRuntimeUDFContext	createRuntimeContext(MetricGroup metrics)
protected Collector<OT>	createSolutionSetUpdateOutputCollector(Collector<OT> delegate) Creates a new solution set update output collector.
protected Collector<OT>	createWorksetUpdateOutputCollector()
protected Collector<OT>	createWorksetUpdateOutputCollector(Collector<OT> delegate) Creates a new WorksetUpdateOutputCollector.
protected int	currentIteration()
RuntimeAggregatorRegistry	getIterationAggregators()
protected void	incrementIterationCounter()
protected boolean	inFirstIteration()
protected void	initialize()
void	requestTermination()
void	run()
void	terminationCompleted()
boolean	terminationRequested()
protected void	verifyEndOfSuperstepState()

Methods inherited from class org.apache.flink.runtime.operators.BatchTask

cancelChainedTasks, clearReaders, clearWriters, closeChainedTasks, closeUserCode, constructLogString, createInputIterator, excludeFromReset, formatLogString, getContainingTask, getDriverComparator, getInput, getInputSerializer, getIOManager, getLastOutputCollector, getLastTasksConfig, getMemoryManager, getMetricGroup, getNumTaskInputs, getOutputCollector, getOutputCollector, getStub, getTaskConfig, getTaskManagerInfo, initBroadcastInputReaders, initBroadcastInputsSerializers, initInputReaders, initInputsSerializersAndComparators, initLocalStrategies, initOutputs, initOutputs, initStub, instantiateUserCode, invoke, logAndThrowException, openChainedTasks, openUserCode, readAndSetBroadcastInput, releaseBroadcastVariables, resetAllInputs, setLastOutputCollector

Methods inherited from class org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable

abortCheckpointOnBarrier, dispatchOperatorEvent, getCurrentNumberOfSubtasks, getEnvironment, getExecutionConfig, getIndexInSubtaskGroup, getJobConfiguration, getTaskConfiguration, getUserCodeClassLoader, notifyCheckpointAbortAsync, notifyCheckpointCompleteAsync, setShouldInterruptOnCancel, shouldInterruptOnCancel, triggerCheckpointAsync, triggerCheckpointOnBarrier

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.operators.TaskContext

getExecutionConfig, getUserCodeClassLoader

Field Detail

worksetAggregator

protected LongSumAggregator worksetAggregator

worksetBackChannel

protected BlockingBackChannel worksetBackChannel

isWorksetIteration

protected boolean isWorksetIteration

isWorksetUpdate

protected boolean isWorksetUpdate

isSolutionSetUpdate

protected boolean isSolutionSetUpdate

Constructor Detail

AbstractIterativeTask

public AbstractIterativeTask(Environment environment)

Create an Invokable task and set its environment.

Parameters:

environment - The environment assigned to this invokable.

Method Detail

initialize

protected void initialize()
                   throws Exception

Overrides:

initialize in class BatchTask<S extends Function,OT>

Throws:

Exception

run

public void run()
         throws Exception

Overrides:

run in class BatchTask<S extends Function,OT>

Throws:

Exception

closeLocalStrategiesAndCaches

protected void closeLocalStrategiesAndCaches()

Overrides:

closeLocalStrategiesAndCaches in class BatchTask<S extends Function,OT>

createRuntimeContext

public DistributedRuntimeUDFContext createRuntimeContext(MetricGroup metrics)

Overrides:

createRuntimeContext in class BatchTask<S extends Function,OT>

inFirstIteration

protected boolean inFirstIteration()

currentIteration

protected int currentIteration()

incrementIterationCounter

protected void incrementIterationCounter()

brokerKey

public String brokerKey()

getIterationAggregators

public RuntimeAggregatorRegistry getIterationAggregators()

verifyEndOfSuperstepState

protected void verifyEndOfSuperstepState()
                                  throws IOException

Throws:

IOException

terminationRequested

public boolean terminationRequested()

Specified by:

terminationRequested in interface Terminable

requestTermination

public void requestTermination()

Specified by:

requestTermination in interface Terminable

terminationCompleted

public void terminationCompleted()

Specified by:

terminationCompleted in interface Terminable

cancel

public Future<Void> cancel()
                    throws Exception

Description copied from class: AbstractInvokable

This method is called when a task is canceled either as a result of a user abort or an execution failure. It can be overwritten to respond to shut down the user code properly.

Overrides:

cancel in class BatchTask<S extends Function,OT>

Returns:

a future that is completed when this AbstractInvokable is fully terminated. Note that it may never complete if the invokable is stuck.

Throws:

Exception - thrown if any exception occurs during the execution of the user code

createWorksetUpdateOutputCollector

protected Collector<OT> createWorksetUpdateOutputCollector(Collector<OT> delegate)

Creates a new WorksetUpdateOutputCollector.

This collector is used by IterationIntermediateTask or IterationTailTask to update the workset.

If a non-null delegate is given, the new Collector will write to the solution set and also call collect(T) of the delegate.

Parameters:

delegate - null -OR- the delegate on which to call collect() by the newly created collector

Returns:

a new WorksetUpdateOutputCollector

createWorksetUpdateOutputCollector

protected Collector<OT> createWorksetUpdateOutputCollector()

createSolutionSetUpdateOutputCollector

protected Collector<OT> createSolutionSetUpdateOutputCollector(Collector<OT> delegate)

Creates a new solution set update output collector.

This collector is used by IterationIntermediateTask or IterationTailTask to update the solution set of workset iterations. Depending on the task configuration, either a fast (non-probing) SolutionSetFastUpdateOutputCollector or normal (re-probing) SolutionSetUpdateOutputCollector is created.

If a non-null delegate is given, the new Collector will write back to the solution set and also call collect(T) of the delegate.

Parameters:

delegate - null -OR- a delegate collector to be called by the newly created collector

Returns:

a new SolutionSetFastUpdateOutputCollector or SolutionSetUpdateOutputCollector