org.apache.flink.runtime.io.network.buffer

Class BufferConsumerWithPartialRecordLength

java.lang.Object

org.apache.flink.runtime.io.network.buffer.BufferConsumerWithPartialRecordLength

@NotThreadSafe
public class BufferConsumerWithPartialRecordLength
extends Object

BufferConsumer with partial record length if a record is spanning over buffers

`partialRecordLength` is the length of bytes to skip in order to start with a complete record, from position index 0 of the underlying MemorySegment. `partialRecordLength` is used in approximate local recovery to find the start position of a complete record on a BufferConsumer, so called `partial record clean-up`.

Partial records happen if a record can not fit into one buffer, then the remaining part of the same record is put into the next buffer. Hence partial records only exist at the beginning of a buffer. Partial record clean-up is needed in the mode of approximate local recovery. If a record is spanning over multiple buffers, and the first (several) buffers have got lost due to the failure of the receiver task, the remaining data belonging to the same record in transition should be cleaned up.

If partialRecordLength == 0, the buffer starts with a complete record

If partialRecordLength > 0, the buffer starts with a partial record, its length = partialRecordLength

If partialRecordLength < 0, partialRecordLength is undefined. It is currently used in

Constructor Summary

Constructors

Constructor and Description
BufferConsumerWithPartialRecordLength(BufferConsumer bufferConsumer, int partialRecordLength)

Method Summary

Modifier and Type	Method and Description
Buffer	build()
boolean	cleanupPartialRecord()
BufferConsumer	getBufferConsumer()
int	getPartialRecordLength()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BufferConsumerWithPartialRecordLength

public BufferConsumerWithPartialRecordLength(BufferConsumer bufferConsumer,
                                             int partialRecordLength)

Method Detail

getBufferConsumer

public BufferConsumer getBufferConsumer()

getPartialRecordLength

public int getPartialRecordLength()

build

public Buffer build()

cleanupPartialRecord

public boolean cleanupPartialRecord()