Modifier and Type | Method and Description |
---|---|
void |
SequentialChannelStateReader.readOutputData(ResultPartitionWriter[] writers,
boolean notifyAndBlockOnCompletion) |
void |
SequentialChannelStateReaderImpl.readOutputData(ResultPartitionWriter[] writers,
boolean notifyAndBlockOnCompletion) |
Modifier and Type | Method and Description |
---|---|
ResultPartitionWriter[] |
Environment.getAllWriters() |
ResultPartitionWriter |
Environment.getWriter(int index) |
Modifier and Type | Method and Description |
---|---|
void |
NettyShuffleEnvironment.registerLegacyNetworkMetrics(MetricGroup metricGroup,
ResultPartitionWriter[] producedPartitions,
InputGate[] inputGates)
Deprecated.
should be removed in future
|
Modifier and Type | Field and Description |
---|---|
protected ResultPartitionWriter |
RecordWriter.targetPartition |
Modifier and Type | Method and Description |
---|---|
RecordWriter<T> |
RecordWriterBuilder.build(ResultPartitionWriter writer) |
Modifier and Type | Method and Description |
---|---|
static void |
NettyShuffleMetricFactory.registerLegacyNetworkMetrics(boolean isDetailedMetrics,
MetricGroup metricGroup,
ResultPartitionWriter[] producedPartitions,
InputGate[] inputGates)
Deprecated.
should be removed in future
|
Modifier and Type | Class and Description |
---|---|
class |
BoundedBlockingResultPartition
A output data result of an individual task (one partition of an intermediate result), produced
and communicated in a batch manner: The result must be produced completely before it can be
consumed.
|
class |
BufferWritingResultPartition
A
ResultPartition which writes buffers directly to ResultSubpartition s. |
class |
PipelinedResultPartition
A result output of a task, pipelined (streamed) to the receivers.
|
class |
ResultPartition
A result partition for data produced by a single task.
|
class |
SortMergeResultPartition
SortMergeResultPartition appends records and events to DataBuffer and after the
DataBuffer is full, all data in the DataBuffer will be copied and spilled to a
PartitionedFile in subpartition index order sequentially. |
Modifier and Type | Class and Description |
---|---|
class |
HsResultPartition
HsResultPartition appends records and events to HsMemoryDataManager , the shuffle
data maybe spilled to disk according to the HsSpillingStrategy , and the downstream can
consume data from memory or disk. |
Modifier and Type | Class and Description |
---|---|
class |
TieredResultPartition
TieredResultPartition appends records and events to the tiered storage, which supports
the upstream dynamically switches storage tier for writing shuffle data, and the downstream will
read data from the relevant tier. |
Modifier and Type | Interface and Description |
---|---|
interface |
ShuffleEnvironment<P extends ResultPartitionWriter,G extends IndexedInputGate>
Interface for the implementation of shuffle service local environment.
|
interface |
ShuffleServiceFactory<SD extends ShuffleDescriptor,P extends ResultPartitionWriter,G extends IndexedInputGate>
Interface for shuffle service factory implementations.
|
Modifier and Type | Method and Description |
---|---|
ResultPartitionWriter[] |
RuntimeEnvironment.getAllWriters() |
ResultPartitionWriter |
RuntimeEnvironment.getWriter(int index) |
Modifier and Type | Method and Description |
---|---|
static void |
Task.setupPartitionsAndGates(ResultPartitionWriter[] producedPartitions,
InputGate[] inputGates) |
Constructor and Description |
---|
RuntimeEnvironment(JobID jobId,
JobType jobType,
JobVertexID jobVertexId,
ExecutionAttemptID executionId,
ExecutionConfig executionConfig,
JobInfo jobInfo,
TaskInfo taskInfo,
Configuration jobConfiguration,
Configuration taskConfiguration,
UserCodeClassLoader userCodeClassLoader,
MemoryManager memManager,
SharedResources sharedResources,
IOManager ioManager,
BroadcastVariableManager bcVarManager,
TaskStateManager taskStateManager,
GlobalAggregateManager aggregateManager,
AccumulatorRegistry accumulatorRegistry,
TaskKvStateRegistry kvStateRegistry,
InputSplitProvider splitProvider,
Map<String,Future<Path>> distCacheEntries,
ResultPartitionWriter[] writers,
IndexedInputGate[] inputGates,
TaskEventDispatcher taskEventDispatcher,
CheckpointResponder checkpointResponder,
TaskOperatorEventGateway operatorEventGateway,
TaskManagerRuntimeInfo taskManagerInfo,
TaskMetricGroup metrics,
Task containingTask,
ExternalResourceInfoProvider externalResourceInfoProvider,
ChannelStateWriteRequestExecutorFactory channelStateExecutorFactory,
TaskManagerActions taskManagerActions) |
Modifier and Type | Method and Description |
---|---|
ResultPartitionWriter[] |
SavepointEnvironment.getAllWriters() |
ResultPartitionWriter |
SavepointEnvironment.getWriter(int index) |
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.