public class TieredStorageResultSubpartitionView extends Object implements ResultSubpartitionView
TieredStorageResultSubpartitionView
is the implementation of ResultSubpartitionView
of TieredResultPartition
.ResultSubpartitionView.AvailabilityWithBacklog
Constructor and Description |
---|
TieredStorageResultSubpartitionView(BufferAvailabilityListener availabilityListener,
List<NettyPayloadManager> nettyPayloadManagers,
List<NettyConnectionId> nettyConnectionIds,
List<NettyServiceProducer> serviceProducers) |
Modifier and Type | Method and Description |
---|---|
void |
acknowledgeAllDataProcessed() |
ResultSubpartitionView.AvailabilityWithBacklog |
getAvailabilityAndBacklog(boolean isCreditAvailable)
Get the availability and backlog of the view.
|
Throwable |
getFailureCause()
ResultSubpartitionView can decide whether the failure cause should be reported to
consumer as failure (primary failure) or ProducerFailedException (secondary failure). |
ResultSubpartition.BufferAndBacklog |
getNextBuffer()
Returns the next
Buffer instance of this queue iterator. |
int |
getNumberOfQueuedBuffers() |
boolean |
isReleased() |
void |
notifyDataAvailable() |
void |
notifyNewBufferSize(int newBufferSize) |
void |
notifyRequiredSegmentId(int subpartitionId,
int segmentId)
In tiered storage shuffle mode, only required segments will be sent to prevent the redundant
buffer usage.
|
int |
peekNextBufferSubpartitionId()
Returns the index of the subpartition where the next buffer locates, or -1 if there is no
buffer available and the subpartition to be consumed is not determined.
|
void |
releaseAllResources() |
void |
resumeConsumption() |
int |
unsynchronizedGetNumberOfQueuedBuffers() |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
notifyPriorityEvent
public TieredStorageResultSubpartitionView(BufferAvailabilityListener availabilityListener, List<NettyPayloadManager> nettyPayloadManagers, List<NettyConnectionId> nettyConnectionIds, List<NettyServiceProducer> serviceProducers)
@Nullable public ResultSubpartition.BufferAndBacklog getNextBuffer() throws IOException
ResultSubpartitionView
Buffer
instance of this queue iterator.
If there is currently no instance available, it will return null
. This might
happen for example when a pipelined queue producer is slower than the consumer or a spilled
queue needs to read in more data.
Important: The consumer has to make sure that each buffer instance will
eventually be recycled with Buffer.recycleBuffer()
after it has been consumed.
getNextBuffer
in interface ResultSubpartitionView
IOException
public ResultSubpartitionView.AvailabilityWithBacklog getAvailabilityAndBacklog(boolean isCreditAvailable)
ResultSubpartitionView
getAvailabilityAndBacklog
in interface ResultSubpartitionView
isCreditAvailable
- the availability of credits for this ResultSubpartitionView
.public void notifyRequiredSegmentId(int subpartitionId, int segmentId)
ResultSubpartitionView
notifyRequiredSegmentId
in interface ResultSubpartitionView
subpartitionId
- The id of the corresponding subpartition.segmentId
- The id of required segment.public int peekNextBufferSubpartitionId()
ResultSubpartitionView
peekNextBufferSubpartitionId
in interface ResultSubpartitionView
public void releaseAllResources() throws IOException
releaseAllResources
in interface ResultSubpartitionView
IOException
public boolean isReleased()
isReleased
in interface ResultSubpartitionView
public Throwable getFailureCause()
ResultSubpartitionView
ResultSubpartitionView
can decide whether the failure cause should be reported to
consumer as failure (primary failure) or ProducerFailedException
(secondary failure).
Secondary failure can be reported only if producer (upstream task) is guaranteed to failover.
BEWARE: Incorrectly reporting failure cause as primary failure, can hide the root cause of the failure from the user.
getFailureCause
in interface ResultSubpartitionView
public int unsynchronizedGetNumberOfQueuedBuffers()
unsynchronizedGetNumberOfQueuedBuffers
in interface ResultSubpartitionView
public int getNumberOfQueuedBuffers()
getNumberOfQueuedBuffers
in interface ResultSubpartitionView
public void notifyDataAvailable()
notifyDataAvailable
in interface ResultSubpartitionView
public void resumeConsumption()
resumeConsumption
in interface ResultSubpartitionView
public void acknowledgeAllDataProcessed()
acknowledgeAllDataProcessed
in interface ResultSubpartitionView
public void notifyNewBufferSize(int newBufferSize)
notifyNewBufferSize
in interface ResultSubpartitionView
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.