org.apache.flink.runtime.io.disk.iomanager

Class HeaderlessChannelReaderInputView

java.lang.Object

org.apache.flink.runtime.memory.AbstractPagedInputView

org.apache.flink.runtime.io.disk.iomanager.AbstractChannelReaderInputView

org.apache.flink.runtime.io.disk.iomanager.ChannelReaderInputView

org.apache.flink.runtime.io.disk.iomanager.HeaderlessChannelReaderInputView

All Implemented Interfaces:

DataInput, DataInputView

public class HeaderlessChannelReaderInputView
extends ChannelReaderInputView

A DataInputView that is backed by a BlockChannelReader, making it effectively a data input stream. This view is similar to the ChannelReaderInputView, but does not expect a header for each block, giving a direct stream abstraction over sequence of written blocks. It therefore requires specification of the number of blocks and the number of bytes in the last block.

Field Summary

Fields inherited from class org.apache.flink.runtime.io.disk.iomanager.ChannelReaderInputView

numRequestsRemaining, reader

Fields inherited from class org.apache.flink.runtime.memory.AbstractPagedInputView

headerLength

Constructor Summary

Constructors

Constructor and Description
HeaderlessChannelReaderInputView(BlockChannelReader<MemorySegment> reader, List<MemorySegment> memory, int numBlocks, int numBytesInLastBlock, boolean waitForFirstBlock) Creates a new channel reader that reads from the given channel, expecting a specified number of blocks in the channel, and returns only a specified number of bytes from the last block.
HeaderlessChannelReaderInputView(BlockChannelReader<MemorySegment> reader, List<MemorySegment> memory, int numBlocks, int numBytesInLastBlock, boolean waitForFirstBlock, long offset)

Method Summary

Modifier and Type	Method and Description
void	advance() Advances the view to the next memory segment.
protected int	getLimitForSegment(MemorySegment segment) Gets the limit for reading bytes from the given memory segment.
protected MemorySegment	nextSegment(MemorySegment current) Gets the next segment from the asynchronous block reader.

Methods inherited from class org.apache.flink.runtime.io.disk.iomanager.ChannelReaderInputView

close, getChannel, isClosed, sendReadRequest, waitForFirstBlock

Methods inherited from class org.apache.flink.runtime.memory.AbstractPagedInputView

clear, doAdvance, getCurrentPositionInSegment, getCurrentSegment, getCurrentSegmentLimit, getHeaderLength, read, read, readBoolean, readByte, readChar, readDouble, readFloat, readFully, readFully, readInt, readLine, readLong, readShort, readUnsignedByte, readUnsignedShort, readUTF, seekInput, skipBytes, skipBytesToRead

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HeaderlessChannelReaderInputView

public HeaderlessChannelReaderInputView(BlockChannelReader<MemorySegment> reader,
                                        List<MemorySegment> memory,
                                        int numBlocks,
                                        int numBytesInLastBlock,
                                        boolean waitForFirstBlock)
                                 throws IOException

Creates a new channel reader that reads from the given channel, expecting a specified number of blocks in the channel, and returns only a specified number of bytes from the last block.

WARNING: If the number of blocks given here is higher than the number of blocks in the channel, then the last blocks will not be filled by the reader and will contain undefined data.

Parameters:

reader - The reader that reads the data from disk back into memory.

memory - A list of memory segments that the reader uses for reading the data in. If the list contains more than one segment, the reader will asynchronously pre-fetch blocks ahead.

numBlocks - The number of blocks this channel will read.

numBytesInLastBlock - The number of valid bytes in the last block.

waitForFirstBlock - A flag indicating weather this constructor call should block until the first block has returned from the asynchronous I/O reader.

Throws:

IOException - Thrown, if the read requests for the first blocks fail to be served by the reader.

HeaderlessChannelReaderInputView

public HeaderlessChannelReaderInputView(BlockChannelReader<MemorySegment> reader,
                                        List<MemorySegment> memory,
                                        int numBlocks,
                                        int numBytesInLastBlock,
                                        boolean waitForFirstBlock,
                                        long offset)
                                 throws IOException

Throws:

IOException

Method Detail

nextSegment

protected MemorySegment nextSegment(MemorySegment current)
                             throws IOException

Description copied from class: ChannelReaderInputView

Gets the next segment from the asynchronous block reader. If more requests are to be issued, the method first sends a new request with the current memory segment. If no more requests are pending, the method adds the segment to the readers return queue, which thereby effectively collects all memory segments. Secondly, the method fetches the next non-consumed segment returned by the reader. If no further segments are available, this method thrown an EOFException.

Overrides:

nextSegment in class ChannelReaderInputView

Parameters:

current - The memory segment used for the next request.

Returns:

The memory segment to read from next.

Throws:

EOFException - Thrown, if no further segments are available.

IOException - Thrown, if an I/O error occurred while reading

See Also:

AbstractPagedInputView.nextSegment(org.apache.flink.core.memory.MemorySegment)

advance

public void advance()
             throws IOException

Description copied from class: AbstractPagedInputView

Advances the view to the next memory segment. The reading will continue after the header of the next segment. This method uses AbstractPagedInputView.nextSegment(MemorySegment) and AbstractPagedInputView.getLimitForSegment(MemorySegment) to get the next segment and set its limit.

Overrides:

advance in class AbstractPagedInputView

Throws:

IOException - Thrown, if the next segment could not be obtained.

See Also:

AbstractPagedInputView.nextSegment(MemorySegment), AbstractPagedInputView.getLimitForSegment(MemorySegment)

getLimitForSegment

protected int getLimitForSegment(MemorySegment segment)

Description copied from class: AbstractPagedInputView

Gets the limit for reading bytes from the given memory segment. This method must return the position of the byte after the last valid byte in the given memory segment. When the position returned by this method is reached, the view will attempt to switch to the next memory segment.

Overrides:

getLimitForSegment in class ChannelReaderInputView

Parameters:

segment - The segment to determine the limit for.

Returns:

The limit for the given memory segment.