T
- The type of the value in incoming StreamRecords
.K
- The type of the key.public final class SortingDataInput<T,K> extends Object implements StreamTaskInput<T>
StreamTaskInput
which sorts in the incoming records from a chained input. It postpones
emitting the records until it receives InputStatus.END_OF_INPUT
from the chained input.
After it is done it emits a single record at a time from the sorter.
The sorter uses binary comparison of keys, which are extracted and serialized when received
from the chained input. Moreover the timestamps of incoming records are used for secondary
ordering. For the comparison it uses either FixedLengthByteKeyComparator
if the length of
the serialized key is constant, or VariableLengthByteKeyComparator
otherwise.
Watermarks, stream statuses, nor latency markers are propagated downstream as they do not make sense with buffered records. The input emits the largest watermark seen after all records.
PushingAsyncDataInput.DataOutput<T>
AvailabilityProvider.AvailabilityHelper
UNSPECIFIED
AVAILABLE
Constructor and Description |
---|
SortingDataInput(StreamTaskInput<T> wrappedInput,
TypeSerializer<T> typeSerializer,
TypeSerializer<K> keySerializer,
KeySelector<T,K> keySelector,
MemoryManager memoryManager,
IOManager ioManager,
boolean objectReuse,
double managedMemoryFraction,
Configuration jobConfiguration,
AbstractInvokable containingTask) |
Modifier and Type | Method and Description |
---|---|
void |
close() |
InputStatus |
emitNext(PushingAsyncDataInput.DataOutput<T> output)
Pushes the next element to the output from current data input, and returns the input status
to indicate whether there are more available data in current input.
|
CompletableFuture<?> |
getAvailableFuture() |
int |
getInputIndex()
Returns the input index of this input.
|
CompletableFuture<Void> |
prepareSnapshot(ChannelStateWriter channelStateWriter,
long checkpointId)
Prepares to spill the in-flight input buffers as checkpoint snapshot.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
and, isApproximatelyAvailable, isAvailable, or
public SortingDataInput(StreamTaskInput<T> wrappedInput, TypeSerializer<T> typeSerializer, TypeSerializer<K> keySerializer, KeySelector<T,K> keySelector, MemoryManager memoryManager, IOManager ioManager, boolean objectReuse, double managedMemoryFraction, Configuration jobConfiguration, AbstractInvokable containingTask)
public int getInputIndex()
StreamTaskInput
getInputIndex
in interface StreamTaskInput<T>
public CompletableFuture<Void> prepareSnapshot(ChannelStateWriter channelStateWriter, long checkpointId)
StreamTaskInput
prepareSnapshot
in interface StreamTaskInput<T>
public void close() throws IOException
close
in interface Closeable
close
in interface AutoCloseable
IOException
public InputStatus emitNext(PushingAsyncDataInput.DataOutput<T> output) throws Exception
PushingAsyncDataInput
This method should be non blocking.
emitNext
in interface PushingAsyncDataInput<T>
Exception
public CompletableFuture<?> getAvailableFuture()
getAvailableFuture
in interface AvailabilityProvider
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.