org.apache.flink.runtime.io.network.api.serialization

Class SpillingAdaptiveSpanningRecordDeserializer<T extends IOReadableWritable>

java.lang.Object

org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer<T>

Type Parameters:

T - The type of the record to be deserialized.

All Implemented Interfaces:

RecordDeserializer<T>

public class SpillingAdaptiveSpanningRecordDeserializer<T extends IOReadableWritable>
extends Object
implements RecordDeserializer<T>

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.network.api.serialization.RecordDeserializer

RecordDeserializer.DeserializationResult

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_FILE_BUFFER_SIZE
static int	DEFAULT_THRESHOLD_FOR_SPILLING

Constructor Summary

Constructors

Constructor and Description
SpillingAdaptiveSpanningRecordDeserializer(String[] tmpDirectories)
SpillingAdaptiveSpanningRecordDeserializer(String[] tmpDirectories, int thresholdForSpilling, int fileBufferSize)

Method Summary

Modifier and Type	Method and Description
void	clear()
RecordDeserializer.DeserializationResult	getNextRecord(T target)
CloseableIterator<Buffer>	getUnconsumedBuffer() Gets the unconsumed buffer which needs to be persisted in unaligned checkpoint scenario.
void	setNextBuffer(Buffer buffer)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_THRESHOLD_FOR_SPILLING

public static final int DEFAULT_THRESHOLD_FOR_SPILLING

See Also:

Constant Field Values

DEFAULT_FILE_BUFFER_SIZE

public static final int DEFAULT_FILE_BUFFER_SIZE

See Also:

Constant Field Values

Constructor Detail

SpillingAdaptiveSpanningRecordDeserializer

public SpillingAdaptiveSpanningRecordDeserializer(String[] tmpDirectories)

SpillingAdaptiveSpanningRecordDeserializer

public SpillingAdaptiveSpanningRecordDeserializer(String[] tmpDirectories,
                                                  int thresholdForSpilling,
                                                  int fileBufferSize)

Method Detail

setNextBuffer

public void setNextBuffer(Buffer buffer)
                   throws IOException

Specified by:

setNextBuffer in interface RecordDeserializer<T extends IOReadableWritable>

Throws:

IOException

getUnconsumedBuffer

public CloseableIterator<Buffer> getUnconsumedBuffer()
                                              throws IOException

Description copied from interface: RecordDeserializer

Gets the unconsumed buffer which needs to be persisted in unaligned checkpoint scenario.

Note that the unconsumed buffer might be null if the whole buffer was already consumed before and there are no partial length or data remained in the end of buffer.

Specified by:

getUnconsumedBuffer in interface RecordDeserializer<T extends IOReadableWritable>

Throws:

IOException

getNextRecord

public RecordDeserializer.DeserializationResult getNextRecord(T target)
                                                       throws IOException

Specified by:

getNextRecord in interface RecordDeserializer<T extends IOReadableWritable>

Throws:

IOException

clear

public void clear()

Specified by:

clear in interface RecordDeserializer<T extends IOReadableWritable>