Interface | Description |
---|---|
HsDataView |
A view for
HsSubpartitionView to find out what data exists in memory or disk and polling
the data. |
HsFileDataIndex |
Index of spilled data.
|
HsMemoryDataManagerOperation |
This interface is used by
HsSubpartitionMemoryDataManager to operate HsMemoryDataManager . |
HsSpillingInfoProvider |
This component is responsible for providing some information needed for the spill decision.
|
HsSpillingStrategy |
Spilling strategy for hybrid shuffle mode.
|
HsSubpartitionFileReader |
This component is responsible for reading data from disk for a specific subpartition.
|
HsSubpartitionFileReader.Factory |
Factory to create
HsSubpartitionFileReader . |
HsSubpartitionViewInternalOperations |
Operations provided by HsSubpartitionView that are used by other internal components of hybrid
result partition.
|
Class | Description |
---|---|
BufferIndexAndChannel |
Integrate the buffer index and the channel id which it belongs.
|
BufferWithIdentity |
Integrate the buffer with index and the channel information to which it belongs.
|
HsBufferContext |
This class maintains the buffer's reference count and its status for hybrid shuffle mode.
|
HsFileDataIndex.ReadableRegion |
Represents a series of physically continuous buffers in the file, which are readable, from
the same subpartition, and has sequential buffer index.
|
HsFileDataIndex.SpilledBuffer |
Represents a spilled buffer.
|
HsFileDataIndexImpl |
Default implementation of
HsFileDataIndex . |
HsFileDataManager |
File data manager for HsResultPartition, which schedules
HsSubpartitionFileReaderImpl for
loading data w.r.t. |
HsFullSpillingStrategy |
A special implementation of
HsSpillingStrategy that spilled all buffers to disk. |
HsMemoryDataManager |
This class is responsible for managing data in memory.
|
HsMemoryDataSpiller |
This component is responsible for asynchronously writing in-memory data to disk.
|
HsOutputMetrics |
All metrics that
HsResultPartition needs to count, except numBytesProduced. |
HsResultPartition |
HsResultPartition appends records and events to HsMemoryDataManager , the shuffle
data maybe spilled to disk according to the HsSpillingStrategy , and the downstream can
consume data from memory or disk. |
HsSelectiveSpillingStrategy |
A special implementation of
HsSpillingStrategy that reduce disk writes as much as
possible. |
HsSpillingStrategy.Decision |
This class represents the spill and release decision made by
HsSpillingStrategy , in
other words, which data is to be spilled and which data is to be released. |
HsSpillingStrategyUtils |
Utilities for
HsSpillingStrategy . |
HsSubpartitionFileReaderImpl |
Default implementation of
HsSubpartitionFileReader . |
HsSubpartitionFileReaderImpl.BufferIndexOrError |
Indicates a buffer with index or an error.
|
HsSubpartitionFileReaderImpl.Factory |
Factory of
HsSubpartitionFileReader . |
HsSubpartitionMemoryDataManager |
This class is responsible for managing the data in a single subpartition.
|
HsSubpartitionView |
The read view of HsResultPartition, data can be read from memory or disk.
|
HybridShuffleConfiguration |
Configuration for hybrid shuffle mode.
|
HybridShuffleConfiguration.Builder |
Builder for
HybridShuffleConfiguration . |
Enum | Description |
---|---|
HsSpillingInfoProvider.ConsumeStatus |
This enum represents buffer status of consume in hybrid shuffle mode.
|
HsSpillingInfoProvider.SpillStatus |
This enum represents buffer status of spill in hybrid shuffle mode.
|
HybridShuffleConfiguration.SpillingStrategyType |
Type of
HsSpillingStrategy . |
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.