Modifier and Type | Interface and Description |
---|---|
interface |
SimpleAccumulator<T extends Serializable>
Similar to Accumulator, but the type of items to add and the result value
must be the same.
|
Modifier and Type | Class and Description |
---|---|
class |
AverageAccumulator
An accumulator that computes the average value.
|
class |
DoubleCounter
An accumulator that sums up
double values. |
class |
DoubleMaximum
An accumulator that finds the maximum
double value. |
class |
DoubleMinimum
An accumulator that finds the minimum
double value. |
class |
Histogram
Histogram accumulator, which builds a histogram in a distributed manner.
|
class |
IntCounter
An accumulator that sums up
Integer values. |
class |
IntMaximum
An accumulator that finds the maximum
integer value. |
class |
IntMinimum
An accumulator that finds the minimum
integer value. |
class |
ListAccumulator<T>
This accumulator stores a collection of objects.
|
class |
LongCounter
An accumulator that sums up
long values. |
class |
LongMaximum
An accumulator that finds the maximum
long value. |
class |
LongMinimum
An accumulator that finds the minimum
long value. |
class |
SerializedListAccumulator<T>
This accumulator stores a collection of objects in serialized form, so that the stored objects
are not affected by modifications to the original objects.
|
Modifier and Type | Method and Description |
---|---|
Accumulator<V,R> |
Accumulator.clone()
Duplicates the accumulator.
|
Accumulator<T,ArrayList<T>> |
ListAccumulator.clone() |
Accumulator<Integer,TreeMap<Integer,Integer>> |
Histogram.clone() |
Modifier and Type | Method and Description |
---|---|
static Map<String,Accumulator<?,?>> |
AccumulatorHelper.copy(Map<String,Accumulator<?,?>> accumulators) |
Modifier and Type | Method and Description |
---|---|
void |
DoubleMinimum.merge(Accumulator<Double,Double> other) |
void |
DoubleCounter.merge(Accumulator<Double,Double> other) |
void |
DoubleMaximum.merge(Accumulator<Double,Double> other) |
void |
AverageAccumulator.merge(Accumulator<Double,Double> other) |
void |
IntMinimum.merge(Accumulator<Integer,Integer> other) |
void |
IntMaximum.merge(Accumulator<Integer,Integer> other) |
void |
IntCounter.merge(Accumulator<Integer,Integer> other) |
void |
Histogram.merge(Accumulator<Integer,TreeMap<Integer,Integer>> other) |
void |
LongMaximum.merge(Accumulator<Long,Long> other) |
void |
LongMinimum.merge(Accumulator<Long,Long> other) |
void |
LongCounter.merge(Accumulator<Long,Long> other) |
void |
SerializedListAccumulator.merge(Accumulator<T,ArrayList<byte[]>> other) |
void |
ListAccumulator.merge(Accumulator<T,ArrayList<T>> other) |
void |
Accumulator.merge(Accumulator<V,R> other)
Used by system internally to merge the collected parts of an accumulator
at the end of the job.
|
Modifier and Type | Method and Description |
---|---|
static void |
AccumulatorHelper.compareAccumulatorTypes(Object name,
Class<? extends Accumulator> first,
Class<? extends Accumulator> second)
Compare both classes and throw
UnsupportedOperationException if
they differ. |
static void |
AccumulatorHelper.compareAccumulatorTypes(Object name,
Class<? extends Accumulator> first,
Class<? extends Accumulator> second)
Compare both classes and throw
UnsupportedOperationException if
they differ. |
static Map<String,Accumulator<?,?>> |
AccumulatorHelper.copy(Map<String,Accumulator<?,?>> accumulators) |
static void |
AccumulatorHelper.mergeInto(Map<String,OptionalFailure<Accumulator<?,?>>> target,
Map<String,Accumulator<?,?>> toMerge)
Merge two collections of accumulators.
|
static void |
AccumulatorHelper.mergeInto(Map<String,OptionalFailure<Accumulator<?,?>>> target,
Map<String,Accumulator<?,?>> toMerge)
Merge two collections of accumulators.
|
static Map<String,OptionalFailure<Object>> |
AccumulatorHelper.toResultMap(Map<String,Accumulator<?,?>> accumulators)
Transform the Map with accumulators into a Map containing only the
results.
|
Modifier and Type | Method and Description |
---|---|
<V,A extends Serializable> |
RuntimeContext.getAccumulator(String name)
Get an existing accumulator object.
|
Modifier and Type | Method and Description |
---|---|
Map<String,Accumulator<?,?>> |
RuntimeContext.getAllAccumulators()
Deprecated.
Use getAccumulator(..) to obtain the value of an accumulator.
|
Modifier and Type | Method and Description |
---|---|
<V,A extends Serializable> |
RuntimeContext.addAccumulator(String name,
Accumulator<V,A> accumulator)
Add this accumulator.
|
Modifier and Type | Method and Description |
---|---|
<V,A extends Serializable> |
AbstractRuntimeUDFContext.getAccumulator(String name) |
Modifier and Type | Method and Description |
---|---|
Map<String,Accumulator<?,?>> |
AbstractRuntimeUDFContext.getAllAccumulators() |
Modifier and Type | Method and Description |
---|---|
<V,A extends Serializable> |
AbstractRuntimeUDFContext.addAccumulator(String name,
Accumulator<V,A> accumulator) |
Constructor and Description |
---|
AbstractRuntimeUDFContext(TaskInfo taskInfo,
ClassLoader userCodeClassLoader,
ExecutionConfig executionConfig,
Map<String,Accumulator<?,?>> accumulators,
Map<String,Future<Path>> cpTasks,
MetricGroup metrics) |
RuntimeUDFContext(TaskInfo taskInfo,
ClassLoader userCodeClassLoader,
ExecutionConfig executionConfig,
Map<String,Future<Path>> cpTasks,
Map<String,Accumulator<?,?>> accumulators,
MetricGroup metrics) |
Modifier and Type | Class and Description |
---|---|
static class |
Utils.ChecksumHashCode
Accumulator of
Utils.ChecksumHashCode . |
Modifier and Type | Method and Description |
---|---|
void |
Utils.ChecksumHashCode.merge(Accumulator<Utils.ChecksumHashCode,Utils.ChecksumHashCode> other) |
Modifier and Type | Class and Description |
---|---|
static class |
EmptyFieldsCountAccumulator.VectorAccumulator
This accumulator maintains a vector of counts.
|
Modifier and Type | Method and Description |
---|---|
Accumulator<Integer,ArrayList<Integer>> |
EmptyFieldsCountAccumulator.VectorAccumulator.clone() |
Modifier and Type | Method and Description |
---|---|
void |
EmptyFieldsCountAccumulator.VectorAccumulator.merge(Accumulator<Integer,ArrayList<Integer>> other) |
Modifier and Type | Method and Description |
---|---|
<V,A extends Serializable> |
AnalyticHelper.addAccumulator(String name,
Accumulator<V,A> accumulator)
Adds an accumulator by prepending the given name with a random string.
|
Modifier and Type | Class and Description |
---|---|
static class |
ChecksumHashCode.Checksum
Wraps checksum and count.
|
Modifier and Type | Method and Description |
---|---|
Accumulator<ChecksumHashCode.Checksum,ChecksumHashCode.Checksum> |
ChecksumHashCode.Checksum.clone() |
Modifier and Type | Method and Description |
---|---|
void |
ChecksumHashCode.Checksum.merge(Accumulator<ChecksumHashCode.Checksum,ChecksumHashCode.Checksum> other) |
Modifier and Type | Method and Description |
---|---|
Map<String,Accumulator<?,?>> |
AccumulatorSnapshot.deserializeUserAccumulators(ClassLoader classLoader)
Gets the user-defined accumulators values.
|
Map<String,Accumulator<?,?>> |
AccumulatorRegistry.getUserMap()
Gets the map for user-defined accumulators.
|
Modifier and Type | Method and Description |
---|---|
static StringifiedAccumulatorResult[] |
StringifiedAccumulatorResult.stringifyAccumulatorResults(Map<String,OptionalFailure<Accumulator<?,?>>> accs)
Flatten a map of accumulator names to Accumulator instances into an array of StringifiedAccumulatorResult values.
|
Constructor and Description |
---|
AccumulatorSnapshot(JobID jobID,
ExecutionAttemptID executionAttemptID,
Map<String,Accumulator<?,?>> userAccumulators) |
Modifier and Type | Method and Description |
---|---|
Map<String,OptionalFailure<Accumulator<?,?>>> |
ExecutionGraph.aggregateUserAccumulators()
Merges all accumulator results from the tasks previously executed in the Executions.
|
Map<String,Accumulator<?,?>> |
Execution.getUserAccumulators() |
Modifier and Type | Method and Description |
---|---|
void |
Execution.setAccumulators(Map<String,Accumulator<?,?>> userAccumulators)
Update accumulators (discarded when the Execution has already been terminated).
|
Modifier and Type | Field and Description |
---|---|
protected Map<String,Accumulator<?,?>> |
BatchTask.accumulatorMap
The accumulator map used in the RuntimeContext.
|
Modifier and Type | Method and Description |
---|---|
static <T> Collector<T> |
BatchTask.initOutputs(AbstractInvokable containingTask,
ClassLoader cl,
TaskConfig config,
List<ChainedDriver<?,?>> chainedTasksTarget,
List<RecordWriter<?>> eventualOutputs,
ExecutionConfig executionConfig,
Map<String,Accumulator<?,?>> accumulatorMap)
Creates a writer for each output.
|
Modifier and Type | Method and Description |
---|---|
void |
ChainedDriver.setup(TaskConfig config,
String taskName,
Collector<OT> outputCollector,
AbstractInvokable parent,
ClassLoader userCodeClassLoader,
ExecutionConfig executionConfig,
Map<String,Accumulator<?,?>> accumulatorMap) |
Constructor and Description |
---|
DistributedRuntimeUDFContext(TaskInfo taskInfo,
ClassLoader userCodeClassLoader,
ExecutionConfig executionConfig,
Map<String,Future<Path>> cpTasks,
Map<String,Accumulator<?,?>> accumulators,
MetricGroup metrics) |
Constructor and Description |
---|
StreamingRuntimeContext(AbstractStreamOperator<?> operator,
Environment env,
Map<String,Accumulator<?,?>> accumulators) |
Modifier and Type | Method and Description |
---|---|
Map<String,Accumulator<?,?>> |
StreamTask.getAccumulatorMap() |
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.