Modifier and Type | Method and Description |
---|---|
static OpaqueMemoryResource<org.apache.flink.contrib.streaming.state.RocksDBSharedResources> |
RocksDBOperationUtils.allocateSharedCachesIfConfigured(RocksDBMemoryConfiguration memoryConfig,
MemoryManager memoryManager,
org.slf4j.Logger logger) |
Modifier and Type | Method and Description |
---|---|
MemoryManager |
Environment.getMemoryManager()
Returns the current
MemoryManager . |
Constructor and Description |
---|
FileChannelInputView(BlockChannelReader<MemorySegment> reader,
MemoryManager memManager,
List<MemorySegment> memory,
int sizeOfLastBlock) |
FileChannelOutputView(BlockChannelWriter<MemorySegment> writer,
MemoryManager memManager,
List<MemorySegment> memory,
int segmentSize) |
SeekableFileChannelInputView(IOManager ioManager,
FileIOChannel.ID channelId,
MemoryManager memManager,
List<MemorySegment> memory,
int sizeOfLastBlock) |
Modifier and Type | Method and Description |
---|---|
static MemoryManager |
MemoryManager.create(long memorySize,
int pageSize)
Creates a memory manager with the given capacity and given page size.
|
Modifier and Type | Method and Description |
---|---|
MemoryManager |
BatchTask.getMemoryManager() |
MemoryManager |
TaskContext.getMemoryManager() |
Constructor and Description |
---|
TempBarrier(AbstractInvokable owner,
MutableObjectIterator<T> input,
TypeSerializerFactory<T> serializerFactory,
MemoryManager memManager,
IOManager ioManager,
int numPages) |
Modifier and Type | Method and Description |
---|---|
<BT,PT> MutableHashTable<BT,PT> |
NonReusingBuildSecondReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
HashJoinIteratorBase.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBloomFilters) |
<BT,PT> MutableHashTable<BT,PT> |
NonReusingBuildFirstReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
ReusingBuildSecondReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
ReusingBuildFirstReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
Constructor and Description |
---|
NonReusingBuildFirstHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildFirstReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildSecondHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildSecondReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildFirstHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildFirstReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildSecondHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildSecondReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
Modifier and Type | Field and Description |
---|---|
protected MemoryManager |
SpillingResettableMutableObjectIterator.memoryManager |
protected MemoryManager |
SpillingResettableIterator.memoryManager |
Modifier and Type | Field and Description |
---|---|
protected MemoryManager |
UnilateralSortMerger.SpillingThread.memManager |
protected MemoryManager |
UnilateralSortMerger.memoryManager
The memory manager through which memory is allocated and released.
|
Modifier and Type | Method and Description |
---|---|
protected UnilateralSortMerger.ThreadBase<E> |
UnilateralSortMerger.getSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memoryManager,
IOManager ioManager,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxFileHandles) |
protected UnilateralSortMerger.ThreadBase<E> |
CombiningUnilateralSortMerger.getSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memoryManager,
IOManager ioManager,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxFileHandles) |
Constructor and Description |
---|
AbstractMergeInnerJoinIterator(MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
AbstractMergeIterator(MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
AbstractMergeOuterJoinIterator(OuterJoinOperatorBase.OuterJoinType outerJoinType,
MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
CombiningSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memManager,
IOManager ioManager,
TypeSerializer<E> serializer,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxNumFileHandles,
boolean objectReuseEnabled) |
CombiningUnilateralSortMerger(GroupCombineFunction<E,E> combineStub,
MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
CombiningUnilateralSortMerger(GroupCombineFunction<E,E> combineStub,
MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
LargeRecordHandler(TypeSerializer<T> serializer,
TypeComparator<T> comparator,
IOManager ioManager,
MemoryManager memManager,
List<MemorySegment> memory,
AbstractInvokable memoryOwner,
int maxFilehandles) |
NonReusingMergeInnerJoinIterator(MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
NonReusingMergeOuterJoinIterator(OuterJoinOperatorBase.OuterJoinType outerJoinType,
MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
ReusingMergeInnerJoinIterator(MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
ReusingMergeOuterJoinIterator(OuterJoinOperatorBase.OuterJoinType outerJoinType,
MutableObjectIterator<T1> input1,
MutableObjectIterator<T2> input2,
TypeSerializer<T1> serializer1,
TypeComparator<T1> comparator1,
TypeSerializer<T2> serializer2,
TypeComparator<T2> comparator2,
TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
SpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memManager,
IOManager ioManager,
TypeSerializer<E> serializer,
TypeComparator<E> comparator,
List<MemorySegment> sortReadMemory,
List<MemorySegment> writeMemory,
int maxNumFileHandles)
Creates the spilling thread.
|
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean noSpillingMemory,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
List<MemorySegment> memory,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
List<MemorySegment> memory,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean noSpillingMemory,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
List<MemorySegment> memory,
IOManager ioManager,
MutableObjectIterator<E> input,
AbstractInvokable parentTask,
TypeSerializerFactory<E> serializerFactory,
TypeComparator<E> comparator,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean noSpillingMemory,
boolean handleLargeRecords,
boolean objectReuseEnabled,
InMemorySorterFactory<E> inMemorySorterFactory) |
Modifier and Type | Method and Description |
---|---|
MemoryManager |
TaskSlot.getMemoryManager() |
MemoryManager |
TaskSlotTableImpl.getTaskMemoryManager(AllocationID allocationID) |
MemoryManager |
TaskSlotTable.getTaskMemoryManager(AllocationID allocationID)
Get the memory manager of the slot allocated for the task.
|
Modifier and Type | Method and Description |
---|---|
MemoryManager |
RuntimeEnvironment.getMemoryManager() |
Constructor and Description |
---|
RuntimeEnvironment(JobID jobId,
JobVertexID jobVertexId,
ExecutionAttemptID executionId,
ExecutionConfig executionConfig,
TaskInfo taskInfo,
Configuration jobConfiguration,
Configuration taskConfiguration,
ClassLoader userCodeClassLoader,
MemoryManager memManager,
IOManager ioManager,
BroadcastVariableManager bcVarManager,
TaskStateManager taskStateManager,
GlobalAggregateManager aggregateManager,
AccumulatorRegistry accumulatorRegistry,
TaskKvStateRegistry kvStateRegistry,
InputSplitProvider splitProvider,
Map<String,Future<Path>> distCacheEntries,
ResultPartitionWriter[] writers,
InputGate[] inputGates,
TaskEventDispatcher taskEventDispatcher,
CheckpointResponder checkpointResponder,
TaskManagerRuntimeInfo taskManagerInfo,
TaskMetricGroup metrics,
Task containingTask) |
Task(JobInformation jobInformation,
TaskInformation taskInformation,
ExecutionAttemptID executionAttemptID,
AllocationID slotAllocationId,
int subtaskIndex,
int attemptNumber,
Collection<ResultPartitionDeploymentDescriptor> resultPartitionDeploymentDescriptors,
Collection<InputGateDeploymentDescriptor> inputGateDeploymentDescriptors,
int targetSlotNumber,
MemoryManager memManager,
IOManager ioManager,
ShuffleEnvironment<?,?> shuffleEnvironment,
KvStateService kvStateService,
BroadcastVariableManager bcVarManager,
TaskEventDispatcher taskEventDispatcher,
TaskStateManager taskStateManager,
TaskManagerActions taskManagerActions,
InputSplitProvider inputSplitProvider,
CheckpointResponder checkpointResponder,
GlobalAggregateManager aggregateManager,
BlobCacheService blobService,
LibraryCacheManager libraryCache,
FileCache fileCache,
TaskManagerRuntimeInfo taskManagerConfig,
TaskMetricGroup metricGroup,
ResultPartitionConsumableNotifier resultPartitionConsumableNotifier,
PartitionProducerStateChecker partitionProducerStateChecker,
Executor executor)
IMPORTANT: This constructor may not start any work that would need to
be undone in the case of a failing task deployment.
|
Modifier and Type | Method and Description |
---|---|
MemoryManager |
SavepointEnvironment.getMemoryManager() |
Constructor and Description |
---|
BaseHybridHashTable(Configuration conf,
Object owner,
MemoryManager memManager,
long reservedMemorySize,
IOManager ioManager,
int avgRecordLen,
long buildRowCount,
boolean tryDistinctBuildRow) |
BinaryHashTable(Configuration conf,
Object owner,
AbstractRowSerializer buildSideSerializer,
AbstractRowSerializer probeSideSerializer,
Projection<BaseRow,BinaryRow> buildSideProjection,
Projection<BaseRow,BinaryRow> probeSideProjection,
MemoryManager memManager,
long reservedMemorySize,
IOManager ioManager,
int avgRecordLen,
long buildRowCount,
boolean useBloomFilters,
HashJoinType type,
JoinCondition condFunc,
boolean reverseJoin,
boolean[] filterNulls,
boolean tryDistinctBuildRow) |
LongHybridHashTable(Configuration conf,
Object owner,
BinaryRowSerializer buildSideSerializer,
BinaryRowSerializer probeSideSerializer,
MemoryManager memManager,
long reservedMemorySize,
IOManager ioManager,
int avgRecordLen,
long buildRowCount) |
Constructor and Description |
---|
BytesHashMap(Object owner,
MemoryManager memoryManager,
long memorySize,
LogicalType[] keyTypes,
LogicalType[] valueTypes) |
BytesHashMap(Object owner,
MemoryManager memoryManager,
long memorySize,
LogicalType[] keyTypes,
LogicalType[] valueTypes,
boolean inferBucketMemory) |
Constructor and Description |
---|
BinaryExternalSorter(Object owner,
MemoryManager memoryManager,
long reservedMemorySize,
IOManager ioManager,
AbstractRowSerializer<BaseRow> inputSerializer,
BinaryRowSerializer serializer,
NormalizedKeyComputer normalizedKeyComputer,
RecordComparator comparator,
Configuration conf) |
BinaryExternalSorter(Object owner,
MemoryManager memoryManager,
long reservedMemorySize,
IOManager ioManager,
AbstractRowSerializer<BaseRow> inputSerializer,
BinaryRowSerializer serializer,
NormalizedKeyComputer normalizedKeyComputer,
RecordComparator comparator,
Configuration conf,
float startSpillingFraction) |
Constructor and Description |
---|
ResettableExternalBuffer(MemoryManager memoryManager,
IOManager ioManager,
List<MemorySegment> memory,
AbstractRowSerializer serializer,
boolean isRowAllInFixedPart) |
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.