public class HsSubpartitionConsumer extends Object implements ResultSubpartitionView, HsSubpartitionConsumerInternalOperations
ResultSubpartitionView.AvailabilityWithBacklog
Constructor and Description |
---|
HsSubpartitionConsumer(BufferAvailabilityListener availabilityListener) |
Modifier and Type | Method and Description |
---|---|
void |
acknowledgeAllDataProcessed() |
ResultSubpartitionView.AvailabilityWithBacklog |
getAvailabilityAndBacklog(boolean isCreditAvailable)
Get the availability and backlog of the view.
|
int |
getConsumingOffset(boolean withLock)
Get the latest consuming offset of the subpartition.
|
Throwable |
getFailureCause()
ResultSubpartitionView can decide whether the failure cause should be reported to
consumer as failure (primary failure) or ProducerFailedException (secondary failure). |
ResultSubpartition.BufferAndBacklog |
getNextBuffer()
Returns the next
Buffer instance of this queue iterator. |
int |
getNumberOfQueuedBuffers() |
boolean |
isReleased() |
void |
notifyDataAvailable()
Callback for new data become available.
|
void |
notifyNewBufferSize(int newBufferSize) |
int |
peekNextBufferSubpartitionId()
Returns the index of the subpartition where the next buffer locates, or -1 if there is no
buffer available and the subpartition to be consumed is not determined.
|
void |
releaseAllResources() |
void |
resumeConsumption() |
int |
unsynchronizedGetNumberOfQueuedBuffers() |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
notifyPriorityEvent, notifyRequiredSegmentId
public HsSubpartitionConsumer(BufferAvailabilityListener availabilityListener)
@Nullable public ResultSubpartition.BufferAndBacklog getNextBuffer()
ResultSubpartitionView
Buffer
instance of this queue iterator.
If there is currently no instance available, it will return null
. This might
happen for example when a pipelined queue producer is slower than the consumer or a spilled
queue needs to read in more data.
Important: The consumer has to make sure that each buffer instance will
eventually be recycled with Buffer.recycleBuffer()
after it has been consumed.
getNextBuffer
in interface ResultSubpartitionView
public void notifyDataAvailable()
HsSubpartitionConsumerInternalOperations
notifyDataAvailable
in interface HsSubpartitionConsumerInternalOperations
notifyDataAvailable
in interface ResultSubpartitionView
public ResultSubpartitionView.AvailabilityWithBacklog getAvailabilityAndBacklog(boolean isCreditAvailable)
ResultSubpartitionView
getAvailabilityAndBacklog
in interface ResultSubpartitionView
isCreditAvailable
- the availability of credits for this ResultSubpartitionView
.public void releaseAllResources() throws IOException
releaseAllResources
in interface ResultSubpartitionView
IOException
public boolean isReleased()
isReleased
in interface ResultSubpartitionView
public int getConsumingOffset(boolean withLock)
HsSubpartitionConsumerInternalOperations
getConsumingOffset
in interface HsSubpartitionConsumerInternalOperations
withLock
- If true, read the consuming offset outside the guarding of lock. This is
sometimes desired to avoid lock contention, if the caller does not depend on any other
states to change atomically with the consuming offset.public Throwable getFailureCause()
ResultSubpartitionView
ResultSubpartitionView
can decide whether the failure cause should be reported to
consumer as failure (primary failure) or ProducerFailedException
(secondary failure).
Secondary failure can be reported only if producer (upstream task) is guaranteed to failover.
BEWARE: Incorrectly reporting failure cause as primary failure, can hide the root cause of the failure from the user.
getFailureCause
in interface ResultSubpartitionView
public void resumeConsumption()
resumeConsumption
in interface ResultSubpartitionView
public void acknowledgeAllDataProcessed()
acknowledgeAllDataProcessed
in interface ResultSubpartitionView
public int unsynchronizedGetNumberOfQueuedBuffers()
unsynchronizedGetNumberOfQueuedBuffers
in interface ResultSubpartitionView
public int getNumberOfQueuedBuffers()
getNumberOfQueuedBuffers
in interface ResultSubpartitionView
public void notifyNewBufferSize(int newBufferSize)
notifyNewBufferSize
in interface ResultSubpartitionView
public int peekNextBufferSubpartitionId()
ResultSubpartitionView
peekNextBufferSubpartitionId
in interface ResultSubpartitionView
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.