Modifier and Type | Method and Description |
---|---|
static OpaqueMemoryResource<org.apache.flink.contrib.streaming.state.RocksDBSharedResources> |
RocksDBOperationUtils.allocateSharedCachesIfConfigured(RocksDBMemoryConfiguration memoryConfig,
MemoryManager memoryManager,
double memoryFraction,
org.slf4j.Logger logger) |
Modifier and Type | Method and Description |
---|---|
MemoryManager |
Environment.getMemoryManager()
Returns the current
MemoryManager . |
Constructor and Description |
---|
FileChannelInputView(BlockChannelReader<MemorySegment> reader,
MemoryManager memManager,
List<MemorySegment> memory,
int sizeOfLastBlock) |
FileChannelOutputView(BlockChannelWriter<MemorySegment> writer,
MemoryManager memManager,
List<MemorySegment> memory,
int segmentSize) |
SeekableFileChannelInputView(IOManager ioManager,
FileIOChannel.ID channelId,
MemoryManager memManager,
List<MemorySegment> memory,
int sizeOfLastBlock) |
Modifier and Type | Method and Description |
---|---|
static MemoryManager |
MemoryManager.create(long memorySize,
int pageSize)
Creates a memory manager with the given capacity and given page size.
|
Modifier and Type | Method and Description |
---|---|
MemoryManager |
TaskContext.getMemoryManager() |
MemoryManager |
BatchTask.getMemoryManager() |
Constructor and Description |
---|
TempBarrier(AbstractInvokable owner,
MutableObjectIterator<T> input,
TypeSerializerFactory<T> serializerFactory,
MemoryManager memManager,
IOManager ioManager,
int numPages,
List<MemorySegment> preAllocated) |
Modifier and Type | Method and Description |
---|---|
<BT,PT> MutableHashTable<BT,PT> |
ReusingBuildFirstReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
NonReusingBuildFirstReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
HashJoinIteratorBase.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBloomFilters) |
<BT,PT> MutableHashTable<BT,PT> |
ReusingBuildSecondReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
NonReusingBuildSecondReOpenableHashJoinIterator.getHashJoin(TypeSerializer<BT> buildSideSerializer,
TypeComparator<BT> buildSideComparator,
TypeSerializer<PT> probeSideSerializer,
TypeComparator<PT> probeSideComparator,
TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
Constructor and Description |
---|
NonReusingBuildFirstHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildFirstReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildSecondHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildSecondReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildFirstHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildFirstReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildSecondHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildSecondReOpenableHashJoinIterator(MutableObjectIterator<V1> firstInput,
MutableObjectIterator<V2> secondInput,
TypeSerializer<V1> serializer1,
TypeComparator<V1> comparator1,
TypeSerializer<V2> serializer2,
TypeComparator<V2> comparator2,
TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
Modifier and Type | Field and Description |
---|---|
protected MemoryManager |
SpillingResettableMutableObjectIterator.memoryManager |
protected MemoryManager |
SpillingResettableIterator.memoryManager |
Modifier and Type | Method and Description |
---|---|
static <E> ExternalSorterBuilder<E> |
ExternalSorter.newBuilder(MemoryManager memoryManager,
AbstractInvokable parentTask,
TypeSerializer<E> serializer,
TypeComparator<E> comparator)
Creates a builder for the
ExternalSorter . |
Modifier and Type | Method and Description |
---|---|
MemoryManager |
TaskSlot.getMemoryManager() |
MemoryManager |
TaskSlotTableImpl.getTaskMemoryManager(AllocationID allocationID) |
MemoryManager |
TaskSlotTable.getTaskMemoryManager(AllocationID allocationID)
Get the memory manager of the slot allocated for the task.
|
Modifier and Type | Method and Description |
---|---|
MemoryManager |
RuntimeEnvironment.getMemoryManager() |
Constructor and Description |
---|
RuntimeEnvironment(JobID jobId,
JobVertexID jobVertexId,
ExecutionAttemptID executionId,
ExecutionConfig executionConfig,
TaskInfo taskInfo,
Configuration jobConfiguration,
Configuration taskConfiguration,
UserCodeClassLoader userCodeClassLoader,
MemoryManager memManager,
IOManager ioManager,
BroadcastVariableManager bcVarManager,
TaskStateManager taskStateManager,
GlobalAggregateManager aggregateManager,
AccumulatorRegistry accumulatorRegistry,
TaskKvStateRegistry kvStateRegistry,
InputSplitProvider splitProvider,
Map<String,Future<Path>> distCacheEntries,
ResultPartitionWriter[] writers,
IndexedInputGate[] inputGates,
TaskEventDispatcher taskEventDispatcher,
CheckpointResponder checkpointResponder,
TaskOperatorEventGateway operatorEventGateway,
TaskManagerRuntimeInfo taskManagerInfo,
TaskMetricGroup metrics,
Task containingTask,
ExternalResourceInfoProvider externalResourceInfoProvider) |
Task(JobInformation jobInformation,
TaskInformation taskInformation,
ExecutionAttemptID executionAttemptID,
AllocationID slotAllocationId,
int subtaskIndex,
int attemptNumber,
List<ResultPartitionDeploymentDescriptor> resultPartitionDeploymentDescriptors,
List<InputGateDeploymentDescriptor> inputGateDeploymentDescriptors,
int targetSlotNumber,
MemoryManager memManager,
IOManager ioManager,
ShuffleEnvironment<?,?> shuffleEnvironment,
KvStateService kvStateService,
BroadcastVariableManager bcVarManager,
TaskEventDispatcher taskEventDispatcher,
ExternalResourceInfoProvider externalResourceInfoProvider,
TaskStateManager taskStateManager,
TaskManagerActions taskManagerActions,
InputSplitProvider inputSplitProvider,
CheckpointResponder checkpointResponder,
TaskOperatorEventGateway operatorCoordinatorEventGateway,
GlobalAggregateManager aggregateManager,
LibraryCacheManager.ClassLoaderHandle classLoaderHandle,
FileCache fileCache,
TaskManagerRuntimeInfo taskManagerConfig,
TaskMetricGroup metricGroup,
ResultPartitionConsumableNotifier resultPartitionConsumableNotifier,
PartitionProducerStateChecker partitionProducerStateChecker,
Executor executor)
IMPORTANT: This constructor may not start any work that would need to be undone in the
case of a failing task deployment.
|
Modifier and Type | Method and Description |
---|---|
MemoryManager |
SavepointEnvironment.getMemoryManager() |
Modifier and Type | Method and Description |
---|---|
static <K> MultiInputSortingDataInput.SelectableSortingInputs |
MultiInputSortingDataInput.wrapInputs(AbstractInvokable containingTask,
StreamTaskInput<Object>[] sortingInputs,
KeySelector<Object,K>[] keySelectors,
TypeSerializer<Object>[] inputSerializers,
TypeSerializer<K> keySerializer,
StreamTaskInput<Object>[] passThroughInputs,
MemoryManager memoryManager,
IOManager ioManager,
boolean objectReuse,
double managedMemoryFraction,
Configuration jobConfiguration) |
Constructor and Description |
---|
SortingDataInput(StreamTaskInput<T> wrappedInput,
TypeSerializer<T> typeSerializer,
TypeSerializer<K> keySerializer,
KeySelector<T,K> keySelector,
MemoryManager memoryManager,
IOManager ioManager,
boolean objectReuse,
double managedMemoryFraction,
Configuration jobConfiguration,
AbstractInvokable containingTask) |
Constructor and Description |
---|
BeamDataStreamPythonFunctionRunner(String taskName,
PythonEnvironmentManager environmentManager,
TypeInformation inputType,
TypeInformation outputType,
String functionUrn,
FlinkFnApi.UserDefinedDataStreamFunction userDefinedDataStreamFunction,
String coderUrn,
Map<String,String> jobOptions,
FlinkMetricContainer flinkMetricContainer,
KeyedStateBackend stateBackend,
TypeSerializer keySerializer,
MemoryManager memoryManager,
double managedMemoryFraction) |
BeamPythonFunctionRunner(String taskName,
PythonEnvironmentManager environmentManager,
String functionUrn,
Map<String,String> jobOptions,
FlinkMetricContainer flinkMetricContainer,
KeyedStateBackend keyedStateBackend,
TypeSerializer keySerializer,
MemoryManager memoryManager,
double managedMemoryFraction) |
Modifier and Type | Method and Description |
---|---|
static <IN1,IN2> StreamTwoInputProcessor<IN1,IN2> |
StreamTwoInputProcessorFactory.create(AbstractInvokable ownerTask,
CheckpointedInputGate[] checkpointedInputGates,
IOManager ioManager,
MemoryManager memoryManager,
TaskIOMetricGroup taskIOMetricGroup,
StreamStatusMaintainer streamStatusMaintainer,
TwoInputStreamOperator<IN1,IN2,?> streamOperator,
WatermarkGauge input1WatermarkGauge,
WatermarkGauge input2WatermarkGauge,
BoundedMultiInput endOfInputAware,
StreamConfig streamConfig,
Configuration taskManagerConfig,
Configuration jobConfig,
ExecutionConfig executionConfig,
ClassLoader userClassloader,
Counter numRecordsIn) |
static StreamMultipleInputProcessor |
StreamMultipleInputProcessorFactory.create(AbstractInvokable ownerTask,
CheckpointedInputGate[] checkpointedInputGates,
StreamConfig.InputConfig[] configuredInputs,
IOManager ioManager,
MemoryManager memoryManager,
TaskIOMetricGroup ioMetricGroup,
Counter mainOperatorRecordsIn,
StreamStatusMaintainer streamStatusMaintainer,
MultipleInputStreamOperator<?> mainOperator,
WatermarkGauge[] inputWatermarkGauges,
StreamConfig streamConfig,
Configuration taskManagerConfig,
Configuration jobConfig,
ExecutionConfig executionConfig,
ClassLoader userClassloader,
OperatorChain<?,?> operatorChain) |
Constructor and Description |
---|
BaseHybridHashTable(Configuration conf,
Object owner,
MemoryManager memManager,
long reservedMemorySize,
IOManager ioManager,
int avgRecordLen,
long buildRowCount,
boolean tryDistinctBuildRow) |
BinaryHashTable(Configuration conf,
Object owner,
AbstractRowDataSerializer buildSideSerializer,
AbstractRowDataSerializer probeSideSerializer,
Projection<RowData,BinaryRowData> buildSideProjection,
Projection<RowData,BinaryRowData> probeSideProjection,
MemoryManager memManager,
long reservedMemorySize,
IOManager ioManager,
int avgRecordLen,
long buildRowCount,
boolean useBloomFilters,
HashJoinType type,
JoinCondition condFunc,
boolean reverseJoin,
boolean[] filterNulls,
boolean tryDistinctBuildRow) |
LongHybridHashTable(Configuration conf,
Object owner,
BinaryRowDataSerializer buildSideSerializer,
BinaryRowDataSerializer probeSideSerializer,
MemoryManager memManager,
long reservedMemorySize,
IOManager ioManager,
int avgRecordLen,
long buildRowCount) |
Constructor and Description |
---|
BytesHashMap(Object owner,
MemoryManager memoryManager,
long memorySize,
LogicalType[] keyTypes,
LogicalType[] valueTypes) |
BytesHashMap(Object owner,
MemoryManager memoryManager,
long memorySize,
LogicalType[] keyTypes,
LogicalType[] valueTypes,
boolean inferBucketMemory) |
Constructor and Description |
---|
BinaryExternalSorter(Object owner,
MemoryManager memoryManager,
long reservedMemorySize,
IOManager ioManager,
AbstractRowDataSerializer<RowData> inputSerializer,
BinaryRowDataSerializer serializer,
NormalizedKeyComputer normalizedKeyComputer,
RecordComparator comparator,
Configuration conf) |
BinaryExternalSorter(Object owner,
MemoryManager memoryManager,
long reservedMemorySize,
IOManager ioManager,
AbstractRowDataSerializer<RowData> inputSerializer,
BinaryRowDataSerializer serializer,
NormalizedKeyComputer normalizedKeyComputer,
RecordComparator comparator,
Configuration conf,
float startSpillingFraction) |
Constructor and Description |
---|
BeamTableStatefulPythonFunctionRunner(String taskName,
PythonEnvironmentManager environmentManager,
RowType inputType,
RowType outputType,
String functionUrn,
FlinkFnApi.UserDefinedAggregateFunctions userDefinedFunctions,
String coderUrn,
Map<String,String> jobOptions,
FlinkMetricContainer flinkMetricContainer,
KeyedStateBackend keyedStateBackend,
TypeSerializer keySerializer,
MemoryManager memoryManager,
double managedMemoryFraction) |
BeamTableStatelessPythonFunctionRunner(String taskName,
PythonEnvironmentManager environmentManager,
RowType inputType,
RowType outputType,
String functionUrn,
FlinkFnApi.UserDefinedFunctions userDefinedFunctions,
String coderUrn,
Map<String,String> jobOptions,
FlinkMetricContainer flinkMetricContainer,
MemoryManager memoryManager,
double managedMemoryFraction) |
Constructor and Description |
---|
LazyMemorySegmentPool(Object owner,
MemoryManager memoryManager,
int maxPages) |
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.