org.apache.flink.runtime.io.disk.iomanager

Class ChannelReaderInputView

java.lang.Object

org.apache.flink.runtime.memory.AbstractPagedInputView

org.apache.flink.runtime.io.disk.iomanager.ChannelReaderInputView

All Implemented Interfaces:

DataInput, DataInputView

Direct Known Subclasses:

HeaderlessChannelReaderInputView

public class ChannelReaderInputView
extends AbstractPagedInputView

A DataInputView that is backed by a BlockChannelReader, making it effectively a data input stream. The view reads it data in blocks from the underlying channel. The view can only read data that has been written by a ChannelWriterOutputView, due to block formatting.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	numRequestsRemaining
protected BlockChannelReader<MemorySegment>	reader

Fields inherited from class org.apache.flink.runtime.memory.AbstractPagedInputView

headerLength

Constructor Summary

Constructors

Constructor and Description
ChannelReaderInputView(BlockChannelReader<MemorySegment> reader, List<MemorySegment> memory, boolean waitForFirstBlock) Creates a new channel reader that reads from the given channel until the last block (as marked by a ChannelWriterOutputView) is found.
ChannelReaderInputView(BlockChannelReader<MemorySegment> reader, List<MemorySegment> memory, int numBlocks, boolean waitForFirstBlock) Creates a new channel reader that reads from the given channel, expecting a specified number of blocks in the channel.

Method Summary

Modifier and Type	Method and Description
List<MemorySegment>	close() Closes this InputView, closing the underlying reader and returning all memory segments.
protected int	getLimitForSegment(MemorySegment segment) Gets the limit for reading bytes from the given memory segment.
boolean	isClosed()
protected MemorySegment	nextSegment(MemorySegment current) Gets the next segment from the asynchronous block reader.
protected void	sendReadRequest(MemorySegment seg) Sends a new read requests, if further requests remain.
void	waitForFirstBlock()

Methods inherited from class org.apache.flink.runtime.memory.AbstractPagedInputView

advance, clear, getCurrentPositionInSegment, getCurrentSegment, getCurrentSegmentLimit, read, read, readBoolean, readByte, readChar, readDouble, readFloat, readFully, readFully, readInt, readLine, readLong, readShort, readUnsignedByte, readUnsignedShort, readUTF, seekInput, skipBytes, skipBytesToRead

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

reader

protected final BlockChannelReader<MemorySegment> reader

numRequestsRemaining

protected int numRequestsRemaining

Constructor Detail

ChannelReaderInputView

public ChannelReaderInputView(BlockChannelReader<MemorySegment> reader,
                              List<MemorySegment> memory,
                              boolean waitForFirstBlock)
                       throws IOException

Creates a new channel reader that reads from the given channel until the last block (as marked by a ChannelWriterOutputView) is found.

Parameters:

reader - The reader that reads the data from disk back into memory.

memory - A list of memory segments that the reader uses for reading the data in. If the list contains more than one segment, the reader will asynchronously pre-fetch blocks ahead.

waitForFirstBlock - A flag indicating weather this constructor call should block until the first block has returned from the asynchronous I/O reader.

Throws:

IOException - Thrown, if the read requests for the first blocks fail to be served by the reader.

ChannelReaderInputView

public ChannelReaderInputView(BlockChannelReader<MemorySegment> reader,
                              List<MemorySegment> memory,
                              int numBlocks,
                              boolean waitForFirstBlock)
                       throws IOException

Creates a new channel reader that reads from the given channel, expecting a specified number of blocks in the channel.

WARNING: The reader will lock if the number of blocks given here is actually lower than the actual number of blocks in the channel.

Parameters:

reader - The reader that reads the data from disk back into memory.

memory - A list of memory segments that the reader uses for reading the data in. If the list contains more than one segment, the reader will asynchronously pre-fetch blocks ahead.

numBlocks - The number of blocks this channel will read. If this value is given, the reader avoids issuing pre-fetch requests for blocks beyond the channel size.

waitForFirstBlock - A flag indicating weather this constructor call should block until the first block has returned from the asynchronous I/O reader.

Throws:

IOException - Thrown, if the read requests for the first blocks fail to be served by the reader.

Method Detail

waitForFirstBlock

public void waitForFirstBlock()
                       throws IOException

Throws:

IOException

isClosed

public boolean isClosed()

close

public List<MemorySegment> close()
                          throws IOException

Closes this InputView, closing the underlying reader and returning all memory segments.

Returns:

A list containing all memory segments originally supplied to this view.

Throws:

IOException - Thrown, if the underlying reader could not be properly closed.

nextSegment

protected MemorySegment nextSegment(MemorySegment current)
                             throws IOException

Gets the next segment from the asynchronous block reader. If more requests are to be issued, the method first sends a new request with the current memory segment. If no more requests are pending, the method adds the segment to the readers return queue, which thereby effectively collects all memory segments. Secondly, the method fetches the next non-consumed segment returned by the reader. If no further segments are available, this method thrown an EOFException.

Specified by:

nextSegment in class AbstractPagedInputView

Parameters:

current - The memory segment used for the next request.

Returns:

The memory segment to read from next.

Throws:

EOFException - Thrown, if no further segments are available.

IOException - Thrown, if an I/O error occurred while reading

See Also:

AbstractPagedInputView.nextSegment(org.apache.flink.core.memory.MemorySegment)

getLimitForSegment

protected int getLimitForSegment(MemorySegment segment)

Description copied from class: AbstractPagedInputView

Gets the limit for reading bytes from the given memory segment. This method must return the position of the byte after the last valid byte in the given memory segment. When the position returned by this method is reached, the view will attempt to switch to the next memory segment.

Specified by:

getLimitForSegment in class AbstractPagedInputView

Parameters:

segment - The segment to determine the limit for.

Returns:

The limit for the given memory segment.

sendReadRequest

protected void sendReadRequest(MemorySegment seg)
                        throws IOException

Sends a new read requests, if further requests remain. Otherwise, this method adds the segment directly to the readers return queue.

Parameters:

seg - The segment to use for the read request.

Throws:

IOException - Thrown, if the reader is in error.