@Internal public class BarrierBuffer extends Object implements CheckpointBarrierHandler
CheckpointBarrierHandler that blocks inputs with barriers until
all inputs have received the barrier for a given checkpoint.
To avoid back-pressuring the input streams (which may cause distributed deadlocks), the BarrierBuffer continues receiving buffers from the blocked channels and stores them internally until the blocks are released.
| Constructor and Description |
|---|
BarrierBuffer(InputGate inputGate,
BufferBlocker bufferBlocker)
Creates a new checkpoint stream aligner.
|
BarrierBuffer(InputGate inputGate,
BufferBlocker bufferBlocker,
long maxBufferedBytes)
Creates a new checkpoint stream aligner.
|
| Modifier and Type | Method and Description |
|---|---|
void |
cleanup()
Cleans up all internally held resources.
|
long |
getAlignmentDurationNanos()
Gets the time that the latest alignment took, in nanoseconds.
|
long |
getCurrentCheckpointId()
Gets the ID defining the current pending, or just completed, checkpoint.
|
BufferOrEvent |
getNextNonBlocked()
Returns the next
BufferOrEvent that the operator may consume. |
boolean |
isEmpty()
Checks if the barrier handler has buffered any data internally.
|
void |
registerCheckpointEventHandler(AbstractInvokable toNotifyOnCheckpoint)
Registers the task be notified once all checkpoint barriers have been received for a checkpoint.
|
String |
toString() |
public BarrierBuffer(InputGate inputGate, BufferBlocker bufferBlocker) throws IOException
There is no limit to how much data may be buffered during an alignment.
inputGate - The input gate to draw the buffers and events from.bufferBlocker - The buffer blocker to hold the buffers and events for channels with barrier.IOException - Thrown, when the spilling to temp files cannot be initialized.public BarrierBuffer(InputGate inputGate, BufferBlocker bufferBlocker, long maxBufferedBytes) throws IOException
The aligner will allow only alignments that buffer up to the given number of bytes. When that number is exceeded, it will stop the alignment and notify the task that the checkpoint has been cancelled.
inputGate - The input gate to draw the buffers and events from.bufferBlocker - The buffer blocker to hold the buffers and events for channels with barrier.maxBufferedBytes - The maximum bytes to be buffered before the checkpoint aborts.IOException - Thrown, when the spilling to temp files cannot be initialized.public BufferOrEvent getNextNonBlocked() throws Exception
CheckpointBarrierHandlerBufferOrEvent that the operator may consume.
This call blocks until the next BufferOrEvent is available, or until the stream
has been determined to be finished.getNextNonBlocked in interface CheckpointBarrierHandlernull, if the stream is finished.IOException - Thrown if the network or local disk I/O fails.InterruptedException - Thrown if the thread is interrupted while blocking during
waiting for the next BufferOrEvent to become available.Exception - Thrown in case that a checkpoint fails that is started as the result of receiving
the last checkpoint barrierpublic void registerCheckpointEventHandler(AbstractInvokable toNotifyOnCheckpoint)
CheckpointBarrierHandlerregisterCheckpointEventHandler in interface CheckpointBarrierHandlertoNotifyOnCheckpoint - The task to notifypublic boolean isEmpty()
CheckpointBarrierHandlerisEmpty in interface CheckpointBarrierHandlerTrue, if no data is buffered internally, false otherwise.public void cleanup()
throws IOException
CheckpointBarrierHandlercleanup in interface CheckpointBarrierHandlerIOException - Thrown if the cleanup of I/O resources failed.public long getCurrentCheckpointId()
public long getAlignmentDurationNanos()
CheckpointBarrierHandlergetAlignmentDurationNanos in interface CheckpointBarrierHandlerCopyright © 2014–2019 The Apache Software Foundation. All rights reserved.