T
- the type of the record that can be emitted with this record writerpublic abstract class RecordWriter<T extends IOReadableWritable> extends Object implements AvailabilityProvider
The RecordWriter wraps the runtime's ResultPartitionWriter
and takes care of
serializing records into buffers.
Important: it is necessary to call flushAll()
after
all records have been written with emit(IOReadableWritable)
. This
ensures that all produced records are written to the output stream (incl.
partially filled ones).
AvailabilityProvider.AvailabilityHelper
Modifier and Type | Field and Description |
---|---|
static String |
DEFAULT_OUTPUT_FLUSH_THREAD_NAME
Default name for the output flush thread, if no name with a task reference is given.
|
protected int |
numberOfChannels |
protected Random |
rng |
protected RecordSerializer<T> |
serializer |
protected ResultPartitionWriter |
targetPartition |
AVAILABLE
Modifier and Type | Method and Description |
---|---|
abstract void |
broadcastEmit(T record)
This is used to broadcast streaming Watermarks in-band with records.
|
void |
broadcastEvent(AbstractEvent event) |
protected void |
checkErroneous() |
abstract void |
clearBuffers()
Closes the
BufferBuilder s for all the channels. |
void |
close()
Closes the writer.
|
protected boolean |
copyFromSerializerToTargetChannel(int targetChannel) |
abstract void |
emit(T record)
This is used to send regular records.
|
protected void |
emit(T record,
int targetChannel) |
protected void |
finishBufferBuilder(BufferBuilder bufferBuilder) |
void |
flushAll() |
protected void |
flushTargetPartition(int targetChannel) |
CompletableFuture<?> |
getAvailableFuture() |
abstract void |
randomEmit(T record)
This is used to send LatencyMarks to a random target channel.
|
void |
setMetricGroup(TaskIOMetricGroup metrics)
Sets the metric group for this RecordWriter.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
isApproximatelyAvailable, isAvailable
@VisibleForTesting public static final String DEFAULT_OUTPUT_FLUSH_THREAD_NAME
protected final ResultPartitionWriter targetPartition
protected final int numberOfChannels
protected final RecordSerializer<T extends IOReadableWritable> serializer
protected final Random rng
protected void emit(T record, int targetChannel) throws IOException, InterruptedException
IOException
InterruptedException
protected boolean copyFromSerializerToTargetChannel(int targetChannel) throws IOException, InterruptedException
targetChannel
- IOException
InterruptedException
public void broadcastEvent(AbstractEvent event) throws IOException
IOException
public void flushAll()
protected void flushTargetPartition(int targetChannel)
public void setMetricGroup(TaskIOMetricGroup metrics)
protected void finishBufferBuilder(BufferBuilder bufferBuilder)
public CompletableFuture<?> getAvailableFuture()
getAvailableFuture
in interface AvailabilityProvider
public abstract void emit(T record) throws IOException, InterruptedException
IOException
InterruptedException
public abstract void randomEmit(T record) throws IOException, InterruptedException
IOException
InterruptedException
public abstract void broadcastEmit(T record) throws IOException, InterruptedException
IOException
InterruptedException
public abstract void clearBuffers()
BufferBuilder
s for all the channels.public void close()
protected void checkErroneous() throws IOException
IOException
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.