Modifier and Type | Method and Description |
---|---|
static Map<String,Object> |
AccumulatorHelper.deserializeAccumulators(Map<String,SerializedValue<Object>> serializedAccumulators,
ClassLoader loader)
Takes the serialized accumulator results and tries to deserialize them using the provided
class loader.
|
Modifier and Type | Method and Description |
---|---|
Map<String,SerializedValue<Object>> |
SerializedJobExecutionResult.getSerializedAccumulatorResults() |
Constructor and Description |
---|
SerializedJobExecutionResult(JobID jobID,
long netRuntime,
Map<String,SerializedValue<Object>> accumulators)
Creates a new SerializedJobExecutionResult.
|
Modifier and Type | Method and Description |
---|---|
SerializedValue<JobInformation> |
TaskDeploymentDescriptor.getSerializedJobInformation()
Return the sub task's serialized job information.
|
SerializedValue<TaskInformation> |
TaskDeploymentDescriptor.getSerializedTaskInformation()
Return the sub task's serialized task information.
|
Constructor and Description |
---|
TaskDeploymentDescriptor(SerializedValue<JobInformation> serializedJobInformation,
SerializedValue<TaskInformation> serializedTaskInformation,
ExecutionAttemptID executionAttemptId,
AllocationID allocationId,
int subtaskIndex,
int attemptNumber,
int targetSlotNumber,
TaskStateHandles taskStateHandles,
Collection<ResultPartitionDeploymentDescriptor> resultPartitionDeploymentDescriptors,
Collection<InputGateDeploymentDescriptor> inputGateDeploymentDescriptors) |
TaskDeploymentDescriptor(SerializedValue<JobInformation> serializedJobInformation,
SerializedValue<TaskInformation> serializedTaskInformation,
ExecutionAttemptID executionAttemptId,
AllocationID allocationId,
int subtaskIndex,
int attemptNumber,
int targetSlotNumber,
TaskStateHandles taskStateHandles,
Collection<ResultPartitionDeploymentDescriptor> resultPartitionDeploymentDescriptors,
Collection<InputGateDeploymentDescriptor> inputGateDeploymentDescriptors) |
Modifier and Type | Method and Description |
---|---|
SerializedValue<ExecutionConfig> |
JobInformation.getSerializedExecutionConfig() |
SerializedValue<JobInformation> |
ExecutionGraph.getSerializedJobInformation() |
SerializedValue<TaskInformation> |
ExecutionJobVertex.getSerializedTaskInformation() |
Modifier and Type | Method and Description |
---|---|
Map<String,SerializedValue<Object>> |
ExecutionGraph.getAccumulatorsSerialized()
Gets a serialized accumulator map.
|
Map<String,SerializedValue<Object>> |
ArchivedExecutionGraph.getAccumulatorsSerialized() |
Map<String,SerializedValue<Object>> |
AccessExecutionGraph.getAccumulatorsSerialized()
Returns a map containing the serialized values of user-defined accumulators.
|
Constructor and Description |
---|
ExecutionGraph(ScheduledExecutorService futureExecutor,
Executor ioExecutor,
JobID jobId,
String jobName,
Configuration jobConfig,
SerializedValue<ExecutionConfig> serializedConfig,
Time timeout,
RestartStrategy restartStrategy,
FailoverStrategy.Factory failoverStrategyFactory,
List<BlobKey> requiredJarFiles,
List<URL> requiredClasspaths,
SlotProvider slotProvider,
ClassLoader userClassLoader) |
JobInformation(JobID jobId,
String jobName,
SerializedValue<ExecutionConfig> serializedExecutionConfig,
Configuration jobConfiguration,
Collection<BlobKey> requiredJarFileBlobKeys,
Collection<URL> requiredClasspathURLs) |
Constructor and Description |
---|
ArchivedExecutionGraph(JobID jobID,
String jobName,
Map<JobVertexID,ArchivedExecutionJobVertex> tasks,
List<ArchivedExecutionJobVertex> verticesInCreationOrder,
long[] stateTimestamps,
JobStatus state,
String failureCause,
String jsonPlan,
StringifiedAccumulatorResult[] archivedUserAccumulators,
Map<String,SerializedValue<Object>> serializedUserAccumulators,
ArchivedExecutionConfig executionConfig,
boolean isStoppable,
JobCheckpointingSettings jobCheckpointingSettings,
CheckpointStatsSnapshot checkpointStatsSnapshot) |
Modifier and Type | Method and Description |
---|---|
SerializedValue<ExecutionConfig> |
JobGraph.getSerializedExecutionConfig()
Returns the
ExecutionConfig |
Modifier and Type | Method and Description |
---|---|
SerializedValue<StateBackend> |
JobCheckpointingSettings.getDefaultStateBackend() |
SerializedValue<MasterTriggerRestoreHook.Factory[]> |
JobCheckpointingSettings.getMasterHooks() |
Constructor and Description |
---|
JobCheckpointingSettings(List<JobVertexID> verticesToTrigger,
List<JobVertexID> verticesToAcknowledge,
List<JobVertexID> verticesToConfirm,
long checkpointInterval,
long checkpointTimeout,
long minPauseBetweenCheckpoints,
int maxConcurrentCheckpoints,
ExternalizedCheckpointSettings externalizedCheckpointSettings,
SerializedValue<StateBackend> defaultStateBackend,
boolean isExactlyOnce) |
JobCheckpointingSettings(List<JobVertexID> verticesToTrigger,
List<JobVertexID> verticesToAcknowledge,
List<JobVertexID> verticesToConfirm,
long checkpointInterval,
long checkpointTimeout,
long minPauseBetweenCheckpoints,
int maxConcurrentCheckpoints,
ExternalizedCheckpointSettings externalizedCheckpointSettings,
SerializedValue<StateBackend> defaultStateBackend,
SerializedValue<MasterTriggerRestoreHook.Factory[]> masterHooks,
boolean isExactlyOnce) |
JobCheckpointingSettings(List<JobVertexID> verticesToTrigger,
List<JobVertexID> verticesToAcknowledge,
List<JobVertexID> verticesToConfirm,
long checkpointInterval,
long checkpointTimeout,
long minPauseBetweenCheckpoints,
int maxConcurrentCheckpoints,
ExternalizedCheckpointSettings externalizedCheckpointSettings,
SerializedValue<StateBackend> defaultStateBackend,
SerializedValue<MasterTriggerRestoreHook.Factory[]> masterHooks,
boolean isExactlyOnce) |
Modifier and Type | Method and Description |
---|---|
Map<String,SerializedValue<Object>> |
AccumulatorResultsFound.result() |
Constructor and Description |
---|
AccumulatorResultsFound(JobID jobID,
Map<String,SerializedValue<Object>> result) |
Modifier and Type | Method and Description |
---|---|
protected AbstractFetcher<T,?> |
FlinkKafkaConsumer010.createFetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
SerializedValue<AssignerWithPeriodicWatermarks<T>> watermarksPeriodic,
SerializedValue<AssignerWithPunctuatedWatermarks<T>> watermarksPunctuated,
StreamingRuntimeContext runtimeContext,
OffsetCommitMode offsetCommitMode) |
protected AbstractFetcher<T,?> |
FlinkKafkaConsumer010.createFetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
SerializedValue<AssignerWithPeriodicWatermarks<T>> watermarksPeriodic,
SerializedValue<AssignerWithPunctuatedWatermarks<T>> watermarksPunctuated,
StreamingRuntimeContext runtimeContext,
OffsetCommitMode offsetCommitMode) |
protected abstract AbstractFetcher<T,?> |
FlinkKafkaConsumerBase.createFetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> subscribedPartitionsToStartOffsets,
SerializedValue<AssignerWithPeriodicWatermarks<T>> watermarksPeriodic,
SerializedValue<AssignerWithPunctuatedWatermarks<T>> watermarksPunctuated,
StreamingRuntimeContext runtimeContext,
OffsetCommitMode offsetCommitMode)
Creates the fetcher that connect to the Kafka brokers, pulls data, deserialized the
data, and emits it into the data streams.
|
protected abstract AbstractFetcher<T,?> |
FlinkKafkaConsumerBase.createFetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> subscribedPartitionsToStartOffsets,
SerializedValue<AssignerWithPeriodicWatermarks<T>> watermarksPeriodic,
SerializedValue<AssignerWithPunctuatedWatermarks<T>> watermarksPunctuated,
StreamingRuntimeContext runtimeContext,
OffsetCommitMode offsetCommitMode)
Creates the fetcher that connect to the Kafka brokers, pulls data, deserialized the
data, and emits it into the data streams.
|
protected AbstractFetcher<T,?> |
FlinkKafkaConsumer09.createFetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
SerializedValue<AssignerWithPeriodicWatermarks<T>> watermarksPeriodic,
SerializedValue<AssignerWithPunctuatedWatermarks<T>> watermarksPunctuated,
StreamingRuntimeContext runtimeContext,
OffsetCommitMode offsetCommitMode) |
protected AbstractFetcher<T,?> |
FlinkKafkaConsumer09.createFetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
SerializedValue<AssignerWithPeriodicWatermarks<T>> watermarksPeriodic,
SerializedValue<AssignerWithPunctuatedWatermarks<T>> watermarksPunctuated,
StreamingRuntimeContext runtimeContext,
OffsetCommitMode offsetCommitMode) |
protected AbstractFetcher<T,?> |
FlinkKafkaConsumer08.createFetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
SerializedValue<AssignerWithPeriodicWatermarks<T>> watermarksPeriodic,
SerializedValue<AssignerWithPunctuatedWatermarks<T>> watermarksPunctuated,
StreamingRuntimeContext runtimeContext,
OffsetCommitMode offsetCommitMode) |
protected AbstractFetcher<T,?> |
FlinkKafkaConsumer08.createFetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
SerializedValue<AssignerWithPeriodicWatermarks<T>> watermarksPeriodic,
SerializedValue<AssignerWithPunctuatedWatermarks<T>> watermarksPunctuated,
StreamingRuntimeContext runtimeContext,
OffsetCommitMode offsetCommitMode) |
Constructor and Description |
---|
Kafka010Fetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
SerializedValue<AssignerWithPeriodicWatermarks<T>> watermarksPeriodic,
SerializedValue<AssignerWithPunctuatedWatermarks<T>> watermarksPunctuated,
ProcessingTimeService processingTimeProvider,
long autoWatermarkInterval,
ClassLoader userCodeClassLoader,
String taskNameWithSubtasks,
MetricGroup metricGroup,
KeyedDeserializationSchema<T> deserializer,
Properties kafkaProperties,
long pollTimeout,
boolean useMetrics) |
Kafka010Fetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
SerializedValue<AssignerWithPeriodicWatermarks<T>> watermarksPeriodic,
SerializedValue<AssignerWithPunctuatedWatermarks<T>> watermarksPunctuated,
ProcessingTimeService processingTimeProvider,
long autoWatermarkInterval,
ClassLoader userCodeClassLoader,
String taskNameWithSubtasks,
MetricGroup metricGroup,
KeyedDeserializationSchema<T> deserializer,
Properties kafkaProperties,
long pollTimeout,
boolean useMetrics) |
Kafka09Fetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
SerializedValue<AssignerWithPeriodicWatermarks<T>> watermarksPeriodic,
SerializedValue<AssignerWithPunctuatedWatermarks<T>> watermarksPunctuated,
ProcessingTimeService processingTimeProvider,
long autoWatermarkInterval,
ClassLoader userCodeClassLoader,
String taskNameWithSubtasks,
MetricGroup metricGroup,
KeyedDeserializationSchema<T> deserializer,
Properties kafkaProperties,
long pollTimeout,
boolean useMetrics) |
Kafka09Fetcher(SourceFunction.SourceContext<T> sourceContext,
Map<KafkaTopicPartition,Long> assignedPartitionsWithInitialOffsets,
SerializedValue<AssignerWithPeriodicWatermarks<T>> watermarksPeriodic,
SerializedValue<AssignerWithPunctuatedWatermarks<T>> watermarksPunctuated,
ProcessingTimeService processingTimeProvider,
long autoWatermarkInterval,
ClassLoader userCodeClassLoader,
String taskNameWithSubtasks,
MetricGroup metricGroup,
KeyedDeserializationSchema<T> deserializer,
Properties kafkaProperties,
long pollTimeout,
boolean useMetrics) |
Modifier and Type | Method and Description |
---|---|
static <T> SerializedValue<T> |
SerializedValue.fromBytes(byte[] serializedData) |
Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.