public class TaskDeploymentDescriptorFactory extends Object
Modifier and Type | Class and Description |
---|---|
static class |
TaskDeploymentDescriptorFactory.PartitionLocationConstraint
Defines whether the partition's location must be known at deployment time or can be unknown
and, therefore, updated later.
|
static class |
TaskDeploymentDescriptorFactory.ShuffleDescriptorAndIndex
This class represents the shuffle descriptor with it index in
ConsumedPartitionGroup . |
static class |
TaskDeploymentDescriptorFactory.ShuffleDescriptorGroup
A set of shuffle descriptors that will be serialized together.
|
Modifier and Type | Field and Description |
---|---|
static ConfigOption<Integer> |
OFFLOAD_SHUFFLE_DESCRIPTORS_THRESHOLD
This is an expert option, that we do not want to expose in the documentation.
|
Constructor and Description |
---|
TaskDeploymentDescriptorFactory(Either<SerializedValue<JobInformation>,PermanentBlobKey> jobInformationOrBlobKey,
JobID jobID,
TaskDeploymentDescriptorFactory.PartitionLocationConstraint partitionDeploymentConstraint,
BlobWriter blobWriter,
boolean nonFinishedHybridPartitionShouldBeUnknown,
int offloadShuffleDescriptorsThreshold) |
Modifier and Type | Method and Description |
---|---|
TaskDeploymentDescriptor |
createDeploymentDescriptor(Execution execution,
AllocationID allocationID,
JobManagerTaskRestore taskRestore,
Collection<ResultPartitionDeploymentDescriptor> producedPartitions) |
static ShuffleDescriptor |
getConsumedPartitionShuffleDescriptor(IntermediateResultPartition consumedPartition,
TaskDeploymentDescriptorFactory.PartitionLocationConstraint partitionDeploymentConstraint,
boolean nonFinishedHybridPartitionShouldBeUnknown) |
TaskDeploymentDescriptor.MaybeOffloaded<JobInformation> |
getSerializedJobInformation() |
@Experimental public static final ConfigOption<Integer> OFFLOAD_SHUFFLE_DESCRIPTORS_THRESHOLD
public TaskDeploymentDescriptorFactory(Either<SerializedValue<JobInformation>,PermanentBlobKey> jobInformationOrBlobKey, JobID jobID, TaskDeploymentDescriptorFactory.PartitionLocationConstraint partitionDeploymentConstraint, BlobWriter blobWriter, boolean nonFinishedHybridPartitionShouldBeUnknown, int offloadShuffleDescriptorsThreshold)
public TaskDeploymentDescriptor.MaybeOffloaded<JobInformation> getSerializedJobInformation()
public TaskDeploymentDescriptor createDeploymentDescriptor(Execution execution, AllocationID allocationID, @Nullable JobManagerTaskRestore taskRestore, Collection<ResultPartitionDeploymentDescriptor> producedPartitions) throws IOException, ClusterDatasetCorruptedException
public static ShuffleDescriptor getConsumedPartitionShuffleDescriptor(IntermediateResultPartition consumedPartition, TaskDeploymentDescriptorFactory.PartitionLocationConstraint partitionDeploymentConstraint, boolean nonFinishedHybridPartitionShouldBeUnknown)
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.