@PublicEvolving public class StatefulSequenceSource extends RichParallelSourceFunction<Long> implements Checkpointed<Long>
SourceFunction.SourceContext<T>| Constructor and Description |
|---|
StatefulSequenceSource(long start,
long end)
Creates a source that emits all numbers from the given interval exactly once.
|
| Modifier and Type | Method and Description |
|---|---|
void |
cancel()
Cancels the source.
|
void |
restoreState(Long state)
Restores the state of the function or operator to that of a previous checkpoint.
|
void |
run(SourceFunction.SourceContext<Long> ctx)
Starts the source.
|
Long |
snapshotState(long checkpointId,
long checkpointTimestamp)
Gets the current state of the function of operator.
|
close, getIterationRuntimeContext, getRuntimeContext, open, setRuntimeContextpublic StatefulSequenceSource(long start,
long end)
start - Start of the range of numbers to emit.end - End of the range of numbers to emit.public void run(SourceFunction.SourceContext<Long> ctx) throws Exception
SourceFunctionSourceFunction.SourceContext emit
elements.
Sources that implement Checkpointed
must lock on the checkpoint lock (using a synchronized block) before updating internal
state and emitting elements, to make both an atomic operation:
public class ExampleSource<T> implements SourceFunction<T>, Checkpointed<Long> {
private long count = 0L;
private volatile boolean isRunning = true;
public void run(SourceContext<T> ctx) {
while (isRunning && count < 1000) {
synchronized (ctx.getCheckpointLock()) {
ctx.collect(count);
count++;
}
}
}
public void cancel() {
isRunning = false;
}
public Long snapshotState(long checkpointId, long checkpointTimestamp) { return count; }
public void restoreState(Long state) { this.count = state; }
}
run in interface SourceFunction<Long>ctx - The context to emit elements to and for accessing locks.Exceptionpublic void cancel()
SourceFunctionSourceFunction.run(SourceContext) method. The implementation needs to ensure that the
source will break out of that loop after this method is called.
A typical pattern is to have an "volatile boolean isRunning" flag that is set to
false in this method. That flag is checked in the loop condition.
When a source is canceled, the executing thread will also be interrupted
(via Thread.interrupt()). The interruption happens strictly after this
method has been called, so any interruption handler can rely on the fact that
this method has completed. It is good practice to make any flags altered by
this method "volatile", in order to guarantee the visibility of the effects of
this method to any interruption handler.
cancel in interface SourceFunction<Long>public Long snapshotState(long checkpointId, long checkpointTimestamp)
CheckpointedsnapshotState in interface Checkpointed<Long>checkpointId - The ID of the checkpoint.checkpointTimestamp - The timestamp of the checkpoint, as derived by
System.currentTimeMillis() on the JobManager.public void restoreState(Long state)
CheckpointedrestoreState in interface Checkpointed<Long>state - The state to be restored.Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.