Modifier and Type | Class and Description |
---|---|
interface |
VisibleForTesting
This annotations declares that a function, field, constructor, or entire type, is only visible for
testing purposes.
|
Modifier and Type | Class and Description |
---|---|
class |
ArchivedExecutionConfig
Serializable class which is created when archiving the job.
|
class |
Plan
This class represents Flink programs, in the form of dataflow plans.
|
class |
PlanExecutor
A PlanExecutor executes a Flink program's dataflow plan.
|
class |
TaskInfo
Encapsulates task-specific information: name, index of subtask, parallelism and attempt number.
|
Modifier and Type | Method and Description |
---|---|
ArchivedExecutionConfig |
ExecutionConfig.archive() |
Modifier and Type | Class and Description |
---|---|
class |
AccumulatorHelper |
Modifier and Type | Class and Description |
---|---|
class |
AggregatorRegistry
A registry for iteration
Aggregator s. |
Modifier and Type | Class and Description |
---|---|
class |
CommonRangeBoundaries<T> |
Modifier and Type | Class and Description |
---|---|
class |
CopyingIterator<E> |
class |
CopyingListCollector<T> |
class |
FunctionUtils
Utility class that contains helper methods to work with Flink
Function class. |
class |
ListCollector<T>
A
Collector that puts the collected elements into a given list. |
class |
NoOpFunction |
class |
RuntimeUDFContext
A standalone implementation of the
RuntimeContext , created by runtime UDF operators. |
Modifier and Type | Class and Description |
---|---|
class |
DefaultInputSplitAssigner
This is the default implementation of the
InputSplitAssigner interface. |
class |
GenericCsvInputFormat<OT> |
class |
GlobFilePathFilter
Class for determining if a particular file should be included or excluded
based on a set of include and exclude glob filters.
|
class |
ReplicatingInputSplitAssigner
Assigns each InputSplit to each requesting parallel instance.
|
Modifier and Type | Class and Description |
---|---|
class |
Bzip2InputStreamFactory |
class |
DeflateInflaterInputStreamFactory
Factory for input streams that decompress the "deflate" compression format.
|
class |
GzipInflaterInputStreamFactory
Factory for input streams that decompress the GZIP compression format.
|
interface |
InflaterInputStreamFactory<T extends InputStream>
Creates a new instance of a certain subclass of
InflaterInputStream . |
class |
XZInputStreamFactory |
Modifier and Type | Class and Description |
---|---|
class |
AbstractUdfOperator<OUT,FT extends Function>
Abstract superclass for all contracts that represent actual operators.
|
class |
BinaryOperatorInformation<IN1,IN2,OUT>
A class for holding information about a single input operator, such as input/output TypeInformation.
|
class |
CollectionExecutor
Execution utility for serial, local, collection-based executions of Flink programs.
|
class |
CompilerHints
A class encapsulating compiler hints describing the behavior of the user function.
|
class |
DualInputOperator<IN1,IN2,OUT,FT extends Function>
Abstract operator superclass for for all operators that have two inputs, like "Join", "CoGroup", or "Cross".
|
class |
DualInputSemanticProperties
Container for the semantic properties associated to a dual input operator.
|
class |
GenericDataSinkBase<IN>
Operator for nodes that act as data sinks, storing the data they receive.
|
class |
GenericDataSourceBase<OUT,T extends InputFormat<OUT,?>>
Abstract superclass for data sources in a Pact plan.
|
interface |
IterationOperator |
class |
Keys<T> |
class |
Operator<OUT>
Abstract base class for all operators.
|
class |
OperatorInformation<OUT>
A class for holding information about an operator, such as input/output TypeInformation.
|
class |
Ordering
This class represents an ordering on a set of fields.
|
interface |
SemanticProperties
Container for the semantic properties associated to an operator.
|
class |
SingleInputOperator<IN,OUT,FT extends Function>
Abstract superclass for for all operators that have one input like "map" or "reduce".
|
class |
SingleInputSemanticProperties
Container for the semantic properties associated to a single input operator.
|
class |
UnaryOperatorInformation<IN,OUT>
A class for holding information about a single input operator, such as input/output TypeInformation.
|
class |
Union<T>
This operator represents a Union between two inputs.
|
Modifier and Type | Class and Description |
---|---|
class |
BulkIterationBase<T> |
class |
CoGroupOperatorBase<IN1,IN2,OUT,FT extends CoGroupFunction<IN1,IN2,OUT>> |
class |
CoGroupRawOperatorBase<IN1,IN2,OUT,FT extends CoGroupFunction<IN1,IN2,OUT>> |
class |
CrossOperatorBase<IN1,IN2,OUT,FT extends CrossFunction<IN1,IN2,OUT>> |
class |
DeltaIterationBase<ST,WT>
A DeltaIteration is similar to a
BulkIterationBase ,
but maintains state across the individual iteration steps. |
class |
FilterOperatorBase<T,FT extends FlatMapFunction<T,T>> |
class |
FlatMapOperatorBase<IN,OUT,FT extends FlatMapFunction<IN,OUT>> |
class |
GroupCombineOperatorBase<IN,OUT,FT extends GroupCombineFunction<IN,OUT>>
Base operator for the combineGroup transformation.
|
class |
GroupReduceOperatorBase<IN,OUT,FT extends GroupReduceFunction<IN,OUT>> |
class |
InnerJoinOperatorBase<IN1,IN2,OUT,FT extends FlatJoinFunction<IN1,IN2,OUT>> |
class |
JoinOperatorBase<IN1,IN2,OUT,FT extends FlatJoinFunction<IN1,IN2,OUT>> |
class |
MapOperatorBase<IN,OUT,FT extends MapFunction<IN,OUT>> |
class |
MapPartitionOperatorBase<IN,OUT,FT extends MapPartitionFunction<IN,OUT>> |
class |
OuterJoinOperatorBase<IN1,IN2,OUT,FT extends FlatJoinFunction<IN1,IN2,OUT>> |
class |
PartitionOperatorBase<IN> |
class |
ReduceOperatorBase<T,FT extends ReduceFunction<T>>
Base data flow operator for Reduce user-defined functions.
|
class |
SortPartitionOperatorBase<IN> |
Modifier and Type | Class and Description |
---|---|
class |
FieldList
Immutable ordered list of fields IDs.
|
class |
FieldSet
Immutable unordered collection of fields IDs.
|
class |
JoinHashMap<BT> |
class |
ListKeyGroupedIterator<E>
The KeyValueIterator returns a key and all values that belong to the key (share the same key).
|
class |
TypeComparable<T>
Wrapper that is used to store elements for which we have a TypeComparator in a Hash Map.
|
class |
UserCodeClassWrapper<T>
This holds a class containing user defined code.
|
class |
UserCodeObjectWrapper<T>
This holds an actual object containing user defined code.
|
interface |
UserCodeWrapper<T>
UDf operators can have either a class or an object containing the user
code, this is the common interface to access them.
|
Modifier and Type | Class and Description |
---|---|
class |
GenericPairComparator<T1,T2> |
class |
SameTypePairComparator<T> |
interface |
TypeComparatorFactory<T> |
class |
TypePairComparator<T1,T2>
This interface defines the method required by the runtime to use data types in join-like operations.
|
interface |
TypePairComparatorFactory<T1,T2> |
interface |
TypeSerializerFactory<T> |
class |
TypeSerializerSerializationProxy<T> |
Modifier and Type | Class and Description |
---|---|
class |
BasicTypeComparator<T extends Comparable<T>> |
class |
BigDecComparator
Comparator for comparing BigDecimal values.
|
class |
BigDecSerializer
Serializer for serializing/deserializing BigDecimal values including null values.
|
class |
BigIntComparator
Comparator for comparing BigInteger values.
|
class |
BigIntSerializer
Serializer for serializing/deserializing BigInteger values including null values.
|
class |
BooleanComparator |
class |
BooleanSerializer |
class |
BooleanValueComparator
Specialized comparator for BooleanValue based on CopyableValueComparator.
|
class |
BooleanValueSerializer |
class |
ByteComparator |
class |
ByteSerializer |
class |
ByteValueComparator
Specialized comparator for ByteValue based on CopyableValueComparator.
|
class |
ByteValueSerializer |
class |
CharComparator |
class |
CharSerializer |
class |
CharValueComparator
Specialized comparator for CharValue based on CopyableValueComparator.
|
class |
CharValueSerializer |
class |
DateComparator |
class |
DateSerializer |
class |
DoubleComparator |
class |
DoubleSerializer |
class |
DoubleValueComparator
Specialized comparator for DoubleValue based on CopyableValueComparator.
|
class |
DoubleValueSerializer |
class |
EnumComparator<T extends Enum<T>> |
class |
EnumSerializer<T extends Enum<T>> |
class |
FloatComparator |
class |
FloatSerializer |
class |
FloatValueComparator
Specialized comparator for FloatValue based on CopyableValueComparator.
|
class |
FloatValueSerializer |
class |
GenericArraySerializer<C>
A serializer for arrays of objects.
|
class |
IntComparator |
class |
IntSerializer |
class |
IntValueComparator
Specialized comparator for IntValue based on CopyableValueComparator.
|
class |
IntValueSerializer |
class |
LongComparator |
class |
LongSerializer |
class |
LongValueComparator
Specialized comparator for LongValue based on CopyableValueComparator.
|
class |
LongValueSerializer |
class |
NullValueComparator
Specialized comparator for NullValue based on CopyableValueComparator.
|
class |
NullValueSerializer |
class |
ShortComparator |
class |
ShortSerializer |
class |
ShortValueComparator
Specialized comparator for ShortValue based on CopyableValueComparator.
|
class |
ShortValueSerializer |
class |
SqlDateSerializer |
class |
SqlTimeSerializer |
class |
SqlTimestampComparator
Comparator for comparing Java SQL Timestamps.
|
class |
SqlTimestampSerializer |
class |
StringComparator |
class |
StringSerializer |
class |
StringValueComparator
Specialized comparator for StringValue based on CopyableValueComparator.
|
class |
StringValueSerializer |
class |
TypeSerializerSingleton<T> |
class |
VoidSerializer |
Modifier and Type | Class and Description |
---|---|
class |
ClosureCleaner
The closure cleaner is a utility that tries to truncate the closure (enclosing instance)
of non-static inner classes (created for inline transformation functions).
|
class |
Utils
Utility class that contains helper methods to work with Java APIs.
|
Modifier and Type | Method and Description |
---|---|
static boolean |
ExecutionEnvironment.areExplicitEnvironmentsAllowed()
Checks whether it is currently permitted to explicitly instantiate a LocalEnvironment
or a RemoteEnvironment.
|
Plan |
ExecutionEnvironment.createProgramPlan()
Creates the program's
Plan . |
Plan |
ExecutionEnvironment.createProgramPlan(String jobName)
Creates the program's
Plan . |
Plan |
ExecutionEnvironment.createProgramPlan(String jobName,
boolean clearSinks)
Creates the program's
Plan . |
Modifier and Type | Class and Description |
---|---|
class |
AggregationFunction<T> |
interface |
AggregationFunctionFactory |
class |
MaxAggregationFunction<T extends Comparable<T>> |
class |
MinAggregationFunction<T extends Comparable<T>> |
class |
SumAggregationFunction<T> |
Modifier and Type | Class and Description |
---|---|
class |
FirstReducer<T> |
class |
FormattingMapper<T> |
class |
IdPartitioner |
class |
SampleInCoordinator<T>
SampleInCoordinator wraps the sample logic of the coordinator side (the second phase of
distributed sample algorithm).
|
class |
SampleInPartition<T>
SampleInPartition wraps the sample logic on the partition side (the first phase of distributed
sample algorithm).
|
class |
SampleWithFraction<T>
A map partition function wrapper for sampling algorithms with fraction, the sample algorithm
takes the partition iterator as input.
|
class |
SelectByMaxFunction<T extends Tuple> |
class |
SelectByMinFunction<T extends Tuple> |
class |
SemanticPropUtil
Utility class that contains helper methods to work with
SemanticProperties . |
Modifier and Type | Method and Description |
---|---|
static Set<Annotation> |
FunctionAnnotation.readDualForwardAnnotations(Class<?> udfClass)
Reads the annotations of a user defined function with two inputs and returns semantic properties according to the forwarded fields annotated.
|
static Set<Annotation> |
FunctionAnnotation.readSingleForwardAnnotations(Class<?> udfClass)
Reads the annotations of a user defined function with one input and returns semantic properties according to the forwarded fields annotated.
|
Modifier and Type | Class and Description |
---|---|
class |
HadoopInputFormatCommonBase<T,SPITTYPE extends InputSplit>
A common base for both "mapred" and "mapreduce" Hadoop input formats.
|
class |
HadoopOutputFormatCommonBase<T>
A common base for both "mapred" and "mapreduce" Hadoop output formats.
|
Modifier and Type | Class and Description |
---|---|
class |
HadoopInputFormatBase<K,V,T>
Common base for Java and Scala API for using Hadoop input formats with Flink.
|
class |
HadoopOutputFormatBase<K,V,T>
Common base for the mapred HadoopOutputFormat wrappers.
|
Modifier and Type | Class and Description |
---|---|
class |
HadoopUtils
Utility class to work with Apache Hadoop MapRed classes.
|
Modifier and Type | Class and Description |
---|---|
class |
HadoopInputSplit
A wrapper that represents an input split from the Hadoop mapred API as
a Flink
InputSplit . |
Modifier and Type | Class and Description |
---|---|
class |
HadoopInputFormatBase<K,V,T>
Base class shared between the Java and Scala API of Flink
|
class |
HadoopOutputFormatBase<K,V,T>
Base class shared between the Java and Scala API of Flink
|
Modifier and Type | Class and Description |
---|---|
class |
HadoopUtils
Utility class to work with next generation of Apache Hadoop MapReduce classes.
|
Modifier and Type | Class and Description |
---|---|
class |
CsvInputFormat<OUT> |
class |
PojoCsvInputFormat<OUT> |
class |
TupleCsvInputFormat<OUT> |
Modifier and Type | Class and Description |
---|---|
static class |
AggregateOperator.AggregatingUdf<T extends Tuple> |
class |
BulkIterationResultSet<T> |
class |
CoGroupRawOperator<I1,I2,OUT>
A
DataSet that is the result of a CoGroup transformation. |
static class |
CrossOperator.CrossProjection<I1,I2> |
static class |
CrossOperator.DefaultCrossFunction<T1,T2> |
static class |
CrossOperator.ProjectCrossFunction<T1,T2,R extends Tuple> |
static class |
DistinctOperator.DistinctFunction<T> |
static class |
JoinOperator.DefaultFlatJoinFunction<T1,T2> |
static class |
JoinOperator.DefaultJoin.WrappingFlatJoinFunction<IN1,IN2,OUT> |
static class |
JoinOperator.JoinProjection<I1,I2> |
static class |
JoinOperator.ProjectFlatJoinFunction<T1,T2,R extends Tuple> |
class |
KeyFunctions
This class holds static utilities to append functions that extract and
prune keys.
|
class |
NoOpOperator<IN>
This operator will be ignored during translation.
|
class |
OperatorTranslation |
static class |
ProjectOperator.Projection<T> |
class |
UdfOperatorUtils
Utility class that contains helper methods to work with UDF operators.
|
Modifier and Type | Method and Description |
---|---|
Map<String,DataSet<?>> |
UdfOperator.getBroadcastSets()
Gets the broadcast sets (name and data set) that have been added to context of the UDF.
|
Map<String,DataSet<?>> |
TwoInputUdfOperator.getBroadcastSets() |
Map<String,DataSet<?>> |
SingleInputUdfOperator.getBroadcastSets() |
CrossOperatorBase.CrossHint |
CrossOperator.getCrossHint() |
Partitioner<?> |
PartitionOperator.getCustomPartitioner()
Gets the custom partitioner from this partitioning.
|
Partitioner<?> |
Grouping.getCustomPartitioner()
Gets the custom partitioner to be used for this grouping, or
null , if
none was defined. |
DataSet<T> |
DataSink.getDataSet() |
OutputFormat<T> |
DataSink.getFormat() |
DataSet<T> |
Grouping.getInputDataSet()
Returns the input DataSet of a grouping operation, that is the one before the grouping.
|
InputFormat<OUT,?> |
DataSource.getInputFormat()
Gets the input format that is executed by this data source.
|
JoinOperatorBase.JoinHint |
JoinOperator.getJoinHint()
Gets the JoinHint that describes how the join is executed.
|
JoinType |
JoinOperator.getJoinType()
Gets the JoinType that describes this join operation (e.g.
|
int[] |
DeltaIterationResultSet.getKeyPositions() |
Keys<T> |
Grouping.getKeys() |
protected Keys<I1> |
CoGroupOperator.getKeys1() |
protected Keys<I2> |
CoGroupOperator.getKeys2() |
int |
DeltaIterationResultSet.getMaxIterations() |
Partitioner<?> |
JoinOperator.getPartitioner()
Gets the custom partitioner used by this join, or
null , if none is set. |
SemanticProperties |
UdfOperator.getSemanticProperties()
Gets the semantic properties that have been set for the user-defined functions (UDF).
|
DualInputSemanticProperties |
TwoInputUdfOperator.getSemanticProperties() |
SingleInputSemanticProperties |
SingleInputUdfOperator.getSemanticProperties() |
SingleInputSemanticProperties |
ReduceOperator.getSemanticProperties() |
DualInputSemanticProperties |
JoinOperator.EquiJoin.getSemanticProperties() |
SingleInputSemanticProperties |
GroupReduceOperator.getSemanticProperties() |
SingleInputSemanticProperties |
GroupCombineOperator.getSemanticProperties() |
TypeInformation<T> |
DataSink.getType() |
boolean |
GroupReduceOperator.isCombinable() |
void |
TwoInputUdfOperator.setSemanticProperties(DualInputSemanticProperties properties)
Sets the semantic properties for the user-defined function (UDF).
|
void |
SingleInputUdfOperator.setSemanticProperties(SingleInputSemanticProperties properties)
Sets the semantic properties for the user-defined function (UDF).
|
protected CoGroupOperatorBase<?,?,OUT,?> |
CoGroupOperator.translateToDataFlow(Operator<I1> input1,
Operator<I2> input2) |
protected GroupReduceOperatorBase<IN,IN,GroupReduceFunction<IN,IN>> |
AggregateOperator.translateToDataFlow(Operator<IN> input) |
Modifier and Type | Class and Description |
---|---|
class |
CombineToGroupCombineWrapper<IN,OUT,F extends CombineFunction<IN,IN> & GroupReduceFunction<IN,OUT>>
A wrapper the wraps a function that implements both
CombineFunction and GroupReduceFunction interfaces
and makes it look like a function that implements GroupCombineFunction and GroupReduceFunction to the runtime. |
class |
KeyExtractingMapper<T,K> |
class |
KeyRemovingMapper<T,K> |
class |
PlanBothUnwrappingCoGroupOperator<I1,I2,OUT,K> |
class |
PlanFilterOperator<T> |
class |
PlanLeftUnwrappingCoGroupOperator<I1,I2,OUT,K> |
class |
PlanProjectOperator<T,R extends Tuple> |
class |
PlanRightUnwrappingCoGroupOperator<I1,I2,OUT,K> |
class |
PlanUnwrappingGroupCombineOperator<IN,OUT,K>
A group combine operator that takes 2-tuples (key-value pairs), and applies the group combine operation only
on the unwrapped values.
|
class |
PlanUnwrappingReduceGroupOperator<IN,OUT,K>
A reduce operator that takes 2-tuples (key-value pairs), and applies the group reduce operation only
on the unwrapped values.
|
class |
PlanUnwrappingReduceOperator<T,K>
A reduce operator that takes 2-tuples (key-value pairs), and applies the reduce operation only
on the unwrapped values.
|
class |
PlanUnwrappingSortedGroupCombineOperator<IN,OUT,K1,K2>
A reduce operator that takes 3-tuples (groupKey, sortKey, value), and applies the sorted partial group reduce
operation only on the unwrapped values.
|
class |
PlanUnwrappingSortedReduceGroupOperator<IN,OUT,K1,K2>
A reduce operator that takes 3-tuples (groupKey, sortKey, value), and applies the sorted group reduce
operation only on the unwrapped values.
|
class |
Tuple3UnwrappingIterator<T,K1,K2>
An iterator that reads 3-tuples (groupKey, sortKey, value) and returns only the values (third field).
|
class |
Tuple3WrappingCollector<IN,K1,K2>
Needed to wrap tuples to
Tuple3<groupKey, sortKey, value> for combine method of group reduce with key selector sorting |
class |
TupleLeftUnwrappingJoiner<I1,I2,OUT,K> |
class |
TupleRightUnwrappingJoiner<I1,I2,OUT,K> |
class |
TupleUnwrappingIterator<T,K>
An iterator that reads 2-tuples (key value pairs) and returns only the values (second field).
|
class |
TupleUnwrappingJoiner<I1,I2,OUT,K> |
class |
TupleWrappingCollector<IN,K>
Needed to wrap tuples to
Tuple2<key, value> pairs for combine method of group reduce with key selector function |
class |
TwoKeyExtractingMapper<T,K1,K2> |
class |
WrappingFunction<T extends Function> |
Modifier and Type | Class and Description |
---|---|
class |
BernoulliSampler<T>
A sampler implementation built upon a Bernoulli trail.
|
class |
DistributedRandomSampler<T>
For sampling with fraction, the sample algorithms are natively distributed, while it's not
true for fixed size sample algorithms.
|
class |
IntermediateSampleData<T>
The data structure which is transferred between partitions and the coordinator for distributed
random sampling.
|
class |
PoissonSampler<T>
A sampler implementation based on the Poisson Distribution.
|
class |
RandomSampler<T>
A data sample is a set of data selected from a statistical population by a defined procedure.
|
class |
ReservoirSamplerWithoutReplacement<T>
A simple in memory implementation of Reservoir Sampling without replacement, and with only one
pass through the input iteration whose size is unpredictable.
|
class |
ReservoirSamplerWithReplacement<T>
A simple in memory implementation of Reservoir Sampling with replacement and with only one pass
through the input iteration whose size is unpredictable.
|
Modifier and Type | Class and Description |
---|---|
class |
ModifiedASMAnalyzer
Modified version of ASMs Analyzer.
|
class |
ModifiedASMFrame
Modified version of ASMs Frame.
|
class |
NestedMethodAnalyzer
Extends ASM's BasicInterpreter.
|
class |
TaggedValue
Extension of ASM's BasicValue that allows to assign "tags"
to values and add additional information depending on the tag to the Value.
|
class |
UdfAnalyzer
Implements a Static Code Analyzer (SCA) that uses the ASM framework
for interpreting Java bytecode of Flink UDFs.
|
class |
UdfAnalyzerUtils
Utility class to work with
UdfAnalyzer |
Modifier and Type | Interface and Description |
---|---|
interface |
Aggregator<T,R>
Generic interface for aggregation
|
class |
BooleanSummaryAggregator |
class |
CompensatedSum
Used to calculate sums using the Kahan summation algorithm
The Kahan summation algorithm (also known as compensated summation) reduces the numerical errors that
occur when adding a sequence of finite precision floating point numbers.
|
class |
DoubleSummaryAggregator
Aggregator that can handle Double types
|
class |
FloatSummaryAggregator
Aggregator that can handle Float types
|
class |
IntegerSummaryAggregator
Aggregator that can handle Integer types
|
class |
LongSummaryAggregator
Aggregator that can handle Long types
|
class |
NumericSummaryAggregator<T extends Number>
Generic aggregator for all numeric types creates a summary of a column of numbers.
|
class |
ObjectSummaryAggregator |
class |
ShortSummaryAggregator
Aggregator that can handle Short types
|
class |
StringSummaryAggregator |
class |
SummaryAggregatorFactory
Factory for creating Summary Aggregators
|
class |
TupleSummaryAggregator<R extends Tuple>
Aggregate tuples using an array of aggregators, one for each "column" or position within the Tuple.
|
class |
ValueSummaryAggregator<VT extends Value,PT,R,A extends Aggregator<PT,R>>
This is a generic Aggregator for Value types like StringValue, DoubleValue, etc.
|
Modifier and Type | Class and Description |
---|---|
class |
PojoField
Represent a field definition for
PojoTypeInfo type of objects. |
class |
TypeExtractionException
Type extraction always contains some uncertainty due to unpredictable JVM differences
between vendors or versions.
|
class |
TypeExtractionUtils |
Modifier and Type | Method and Description |
---|---|
static <OUT> TypeInfoFactory<OUT> |
TypeExtractor.getTypeInfoFactory(Type t)
Returns the type information factory for a type using the factory registry or annotations.
|
Modifier and Type | Class and Description |
---|---|
class |
AvroSerializer<T>
General purpose serialization.
|
class |
CopyableValueComparator<T extends CopyableValue<T> & Comparable<T>>
Comparator for all Value types that extend Key
|
class |
CopyableValueSerializer<T extends CopyableValue<T>> |
class |
DataInputDecoder |
class |
DataInputViewStream
An input stream that draws its data from a
DataInputView . |
class |
DataOutputEncoder |
class |
DataOutputViewStream |
class |
EitherSerializer<L,R>
A
TypeSerializer for the Either type of the Java class. |
class |
FieldSerializer
This class is for the serialization of java.lang.reflect.Field, which doesn't implement Serializable, therefore
readObject/writeObject need to be implemented in classes where there is a field of type java.lang.reflect.Field.
|
class |
GenericTypeComparator<T extends Comparable<T>>
TypeComparator for all types that extend Comparable.
|
class |
KryoUtils
Convenience methods for Kryo
|
class |
NoFetchingInput |
class |
NullAwareComparator<T>
Null-aware comparator that wraps a comparator which does not support null references.
|
class |
NullMaskUtils |
class |
PojoComparator<T> |
class |
PojoSerializer<T> |
class |
RowComparator
Comparator for
Row |
class |
RowSerializer
Serializer for
Row . |
class |
RuntimeComparatorFactory<T> |
class |
RuntimePairComparatorFactory<T1,T2> |
class |
RuntimeSerializerFactory<T> |
class |
Tuple0Serializer |
class |
TupleComparator<T extends Tuple> |
class |
TupleComparatorBase<T> |
class |
TupleSerializer<T extends Tuple> |
class |
TupleSerializerBase<T> |
class |
ValueComparator<T extends Value & Comparable<T>>
Comparator for all Value types that extend Key
|
class |
ValueSerializer<T extends Value>
Serializer for
Value types. |
Modifier and Type | Class and Description |
---|---|
class |
Serializers
Class containing utilities for the serializers of the Flink Runtime.
|
Modifier and Type | Class and Description |
---|---|
static class |
ScalaAggregateOperator.AggregatingUdf<T extends scala.Product> |
Modifier and Type | Class and Description |
---|---|
class |
GlobalConfiguration
Global configuration object for Flink.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractMultiFSDataInputStream
Abstract base class for wrappers over multiple
FSDataInputStream , which gives a contiguous view on all inner
streams and makes them look like a single stream, in which we can read, seek, etc. |
class |
ClosingFSDataInputStream
This class is a
WrappingProxy for FSDataInputStream that is used to
implement a safety net against unclosed streams. |
class |
ClosingFSDataOutputStream
This class is a
WrappingProxy for FSDataOutputStream that is used to
implement a safety net against unclosed streams. |
class |
FSDataInputStreamWrapper
Simple forwarding wrapper around
FSDataInputStream |
class |
FSDataOutputStreamWrapper
Simple forwarding wrapper around
FSDataInputStream |
class |
SafetyNetCloseableRegistry
This implementation of an
AbstractCloseableRegistry registers WrappingProxyCloseable . |
class |
SafetyNetWrapperFileSystem
This is a
WrappingProxy around FileSystem which (i) wraps all opened streams as
ClosingFSDataInputStream or ClosingFSDataOutputStream and (ii) registers them to
a SafetyNetCloseableRegistry . |
interface |
WrappingProxyCloseable<T extends Closeable>
WrappingProxy for Closeable that is also closeable. |
Modifier and Type | Method and Description |
---|---|
static void |
FileSystem.closeAndDisposeFileSystemCloseableRegistryForThread()
Create a SafetyNetCloseableRegistry for a Task.
|
static void |
FileSystem.createAndSetFileSystemCloseableRegistryForThread()
Create a SafetyNetCloseableRegistry for a Task.
|
static FileSystem |
FileSystem.getUnguardedFileSystem(URI uri) |
Modifier and Type | Class and Description |
---|---|
class |
LocalBlockLocation
Implementation of the
BlockLocation interface for a local file system. |
class |
LocalDataInputStream
The
LocalDataInputStream class is a wrapper class for a data
input stream to the local file system. |
class |
LocalDataOutputStream
The
LocalDataOutputStream class is a wrapper class for a data
output stream to the local file system. |
class |
LocalFileStatus
The class
LocalFileStatus provides an implementation of the FileStatus interface
for the local file system. |
class |
LocalFileSystem
The class
LocalFile provides an implementation of the FileSystem interface
for the local file system of the machine where the JVM runs. |
Modifier and Type | Class and Description |
---|---|
class |
ByteArrayOutputStreamWithPos
Un-synchronized stream similar to Java's ByteArrayOutputStream that also exposes the current position.
|
class |
HeapMemorySegment
This class represents a piece of heap memory managed by Flink.
|
class |
HybridMemorySegment
This class represents a piece of memory managed by Flink.
|
class |
MemorySegment
This class represents a piece of memory managed by Flink.
|
class |
MemorySegmentFactory
A factory for memory segments.
|
interface |
MemorySegmentSource
Interface describing entities that can provide memory segments.
|
class |
MemoryType
The class of memory, such as heap or off-heap.
|
class |
MemoryUtils
Utility class for memory operations.
|
Modifier and Type | Class and Description |
---|---|
class |
MigrationKeyGroupStateHandle
Deprecated.
|
class |
MigrationStreamStateHandle
Deprecated.
|
Modifier and Type | Class and Description |
---|---|
class |
StreamTaskState
Deprecated.
|
class |
StreamTaskStateList
Deprecated.
|
Modifier and Type | Class and Description |
---|---|
class |
SavepointV1Serializer
Serializer for
SavepointV1 instances. |
Modifier and Type | Class and Description |
---|---|
class |
ExternalizedCheckpointSettings
Grouped settings for externalized checkpoints.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractMetricGroup<A extends AbstractMetricGroup<?>>
Abstract
MetricGroup that contains key functionality for adding metrics and groups. |
class |
ComponentMetricGroup<P extends AbstractMetricGroup<?>>
Abstract
MetricGroup for system components (e.g.,
TaskManager, Job, Task, Operator). |
class |
GenericMetricGroup
A simple named
MetricGroup that is used to hold
subgroups of metrics. |
class |
JobManagerJobMetricGroup
Special
MetricGroup representing everything belonging to
a specific job, running on the JobManager. |
class |
JobMetricGroup<C extends ComponentMetricGroup<C>>
Special abstract
MetricGroup representing everything belonging to
a specific job. |
class |
OperatorMetricGroup
Special
MetricGroup representing an Operator. |
class |
TaskManagerJobMetricGroup
Special
MetricGroup representing everything belonging to
a specific job, running on the TaskManager. |
class |
TaskManagerMetricGroup
Special
MetricGroup representing a TaskManager. |
class |
TaskMetricGroup
Special
MetricGroup representing a Flink runtime Task. |
Modifier and Type | Class and Description |
---|---|
class |
KerberosUtils
Provides vendor-specific Kerberos
AppConfigurationEntry instances. |
Modifier and Type | Class and Description |
---|---|
class |
JaasModule
Responsible for installing a process-wide JAAS configuration.
|
Modifier and Type | Class and Description |
---|---|
class |
DefaultOperatorStateBackend
Default implementation of OperatorStateStore that provides the ability to make snapshots.
|
interface |
StateTransformationFunction<S,T>
Interface for a binary function that is used for push-down of state transformation into state backends.
|
Modifier and Type | Class and Description |
---|---|
class |
CopyOnWriteStateTableSnapshot<K,N,S>
This class represents the snapshot of a
CopyOnWriteStateTable and has a role in operator state checkpointing. |
interface |
InternalKeyContext<K>
This interface is the current context of a keyed state.
|
Modifier and Type | Class and Description |
---|---|
class |
SimpleTimerService
Implementation of
TimerService that uses a InternalTimerService . |
Modifier and Type | Class and Description |
---|---|
static class |
CoGroupedStreams.TaggedUnion<T1,T2>
Internal class for implementing tagged union co-group.
|
Modifier and Type | Method and Description |
---|---|
int |
DataStream.getId()
Returns the ID of the
DataStream in the current StreamExecutionEnvironment . |
KeySelector<T,KEY> |
KeyedStream.getKeySelector()
Gets the key selector that can get the key by which the stream if partitioned from the elements.
|
TypeInformation<KEY> |
KeyedStream.getKeyType()
Gets the type of the key by which the stream is partitioned.
|
SinkTransformation<T> |
DataStreamSink.getTransformation()
Returns the transformation that contains the actual sink operator of this sink.
|
StreamTransformation<T> |
DataStream.getTransformation()
Returns the
StreamTransformation that represents the operation that logically creates
this DataStream . |
<R> SingleOutputStreamOperator<R> |
ConnectedStreams.process(CoProcessFunction<IN1,IN2,R> coProcessFunction,
TypeInformation<R> outputType)
Applies the given
CoProcessFunction on the connected input streams,
thereby creating a transformed output stream. |
<R> SingleOutputStreamOperator<R> |
KeyedStream.process(ProcessFunction<T,R> processFunction,
TypeInformation<R> outputType)
Applies the given
ProcessFunction on the input stream, thereby
creating a transformed output stream. |
Modifier and Type | Method and Description |
---|---|
void |
StreamExecutionEnvironment.addOperator(StreamTransformation<?> transformation)
Adds an operator to the list of operators that should be executed when calling
StreamExecutionEnvironment.execute() . |
<F> F |
StreamExecutionEnvironment.clean(F f)
Returns a "closure-cleaned" version of the given function.
|
StreamGraph |
StreamExecutionEnvironment.getStreamGraph()
Getter of the
StreamGraph of the streaming job. |
Modifier and Type | Class and Description |
---|---|
class |
AggregationFunction<T> |
class |
ComparableAggregator<T> |
class |
Comparator |
class |
SumAggregator<T> |
class |
SumFunction |
Modifier and Type | Class and Description |
---|---|
class |
QueryableAppendingStateOperator<IN>
Internal operator handling queryable AppendingState instances.
|
class |
QueryableValueStateOperator<IN>
Internal operator handling queryable ValueState instances.
|
Modifier and Type | Class and Description |
---|---|
class |
ContinuousFileMonitoringFunction<OUT>
This is the single (non-parallel) monitoring task which takes a
FileInputFormat
and, depending on the FileProcessingMode and the FilePathFilter , it is responsible for:
Monitoring a user-provided path.
Deciding which files should be further read and processed.
Creating the splits corresponding to those files.
Assigning them to downstream tasks for further processing.
The splits to be read are forwarded to the downstream ContinuousFileReaderOperator
which can have parallelism greater than one. |
class |
ContinuousFileReaderOperator<OUT>
The operator that reads the
splits received from the preceding
ContinuousFileMonitoringFunction . |
class |
InputFormatSourceFunction<OUT> |
Modifier and Type | Class and Description |
---|---|
class |
StreamingFunctionUtils
Utility class that contains helper methods to work with Flink Streaming
Functions . |
Modifier and Type | Class and Description |
---|---|
class |
FoldApplyAllWindowFunction<W extends Window,T,ACC,R> |
class |
FoldApplyWindowFunction<K,W extends Window,T,ACC,R> |
class |
PassThroughAllWindowFunction<W extends Window,T> |
class |
PassThroughWindowFunction<K,W extends Window,T> |
class |
ReduceApplyAllWindowFunction<W extends Window,T,R> |
class |
ReduceApplyWindowFunction<K,W extends Window,T,R> |
class |
ReduceIterableAllWindowFunction<W extends Window,T> |
class |
ReduceIterableWindowFunction<K,W extends Window,T> |
Modifier and Type | Class and Description |
---|---|
class |
ArrayFromTuple
Converts a Tuple to an Object-Array.
|
class |
ConcatenatedExtract<FROM,OVER,TO>
Combines two extractors which will be executed one after each other.
|
interface |
Extractor<FROM,TO>
Extractors allow to extract/convert one type to another.
|
class |
FieldFromArray<OUT>
Extracts a single field out of an array.
|
class |
FieldFromTuple<OUT>
Extracts a single field out of a tuple.
|
class |
FieldsFromArray<OUT>
Extracts multiple fields from an array and puts them into a new array of the
specified type.
|
class |
FieldsFromTuple
Extracts one or more fields of the type Double from a tuple and puts them
into a new double[]
|
Modifier and Type | Class and Description |
---|---|
class |
JSONGenerator |
class |
StreamConfig |
class |
StreamEdge
An edge in the streaming topology.
|
class |
StreamGraph
Class representing the streaming topology.
|
class |
StreamGraphGenerator
A generator that generates a
StreamGraph from a graph of
StreamTransformations . |
class |
StreamingJobGraphGenerator |
class |
StreamNode
Class representing the operators in the streaming programs, with all their properties.
|
Modifier and Type | Class and Description |
---|---|
class |
InternalTimer<K,N>
Internal class for keeping track of in-flight timers.
|
interface |
InternalTimerService<N>
Interface for working with time and timers.
|
class |
ProcessOperator<K,IN,OUT> |
class |
StreamFilter<IN> |
class |
StreamFlatMap<IN,OUT> |
class |
StreamGroupedFold<IN,OUT,KEY> |
class |
StreamGroupedReduce<IN> |
class |
StreamMap<IN,OUT> |
class |
StreamProject<IN,OUT extends Tuple> |
class |
StreamSink<IN> |
class |
StreamSource<OUT,SRC extends SourceFunction<OUT>>
StreamOperator for streaming sources. |
class |
TimestampedCollector<T>
|
interface |
Triggerable<K,N>
Interface for things that can be called by
InternalTimerService . |
Modifier and Type | Class and Description |
---|---|
class |
AsyncWaitOperator<IN,OUT>
The
AsyncWaitOperator allows to asynchronously process incoming stream records. |
class |
Emitter<OUT>
Runnable responsible for consuming elements from the given queue and outputting them to the
given output/timestampedCollector.
|
interface |
OperatorActions
Interface for
StreamOperator actions. |
Modifier and Type | Interface and Description |
---|---|
interface |
AsyncCollectionResult<T>
AsyncResult sub class for asynchronous result collections. |
interface |
AsyncResult
Asynchronous result returned by the
StreamElementQueue . |
interface |
AsyncWatermarkResult
AsyncResult subclass for asynchronous result Watermark . |
class |
OrderedStreamElementQueue
Ordered
StreamElementQueue implementation. |
interface |
StreamElementQueue
Interface for blocking stream element queues for the
AsyncWaitOperator . |
class |
StreamElementQueueEntry<T>
Entry class for the
StreamElementQueue . |
class |
StreamRecordQueueEntry<OUT>
StreamElementQueueEntry implementation for StreamRecord . |
class |
UnorderedStreamElementQueue
Unordered implementation of the
StreamElementQueue . |
class |
WatermarkQueueEntry
StreamElementQueueEntry implementation for the Watermark . |
Modifier and Type | Class and Description |
---|---|
class |
CoProcessOperator<K,IN1,IN2,OUT> |
class |
CoStreamFlatMap<IN1,IN2,OUT> |
class |
CoStreamMap<IN1,IN2,OUT> |
Modifier and Type | Class and Description |
---|---|
class |
CoFeedbackTransformation<F>
This represents a feedback point in a topology.
|
class |
FeedbackTransformation<T>
This represents a feedback point in a topology.
|
class |
OneInputTransformation<IN,OUT>
This Transformation represents the application of a
OneInputStreamOperator to one input
StreamTransformation . |
class |
PartitionTransformation<T>
This transformation represents a change of partitioning of the input elements.
|
class |
SelectTransformation<T>
This transformation represents a selection of only certain upstream elements.
|
class |
SinkTransformation<T>
This Transformation represents a Sink.
|
class |
SourceTransformation<T>
This represents a Source.
|
class |
SplitTransformation<T>
This transformation represents a split of one
DataStream into several DataStreams
using an OutputSelector . |
class |
StreamTransformation<T>
A
StreamTransformation represents the operation that creates a
DataStream . |
class |
TwoInputTransformation<IN1,IN2,OUT>
This Transformation represents the application of a
TwoInputStreamOperator to two input
StreamTransformations . |
class |
UnionTransformation<T>
This transformation represents a union of several input
StreamTransformations . |
Modifier and Type | Class and Description |
---|---|
static class |
GlobalWindows.NeverTrigger
A trigger that never fires, as default Trigger for GlobalWindows.
|
Modifier and Type | Class and Description |
---|---|
class |
BarrierBuffer
The barrier buffer is
CheckpointBarrierHandler that blocks inputs with barriers until
all inputs have received the barrier for a given checkpoint. |
class |
BarrierTracker
The BarrierTracker keeps track of what checkpoint barriers have been received from
which input channels.
|
class |
BlockingQueueBroker |
class |
BufferSpiller
The buffer spiller takes the buffers and events from a data stream and adds them to a spill file.
|
interface |
CheckpointBarrierHandler
The CheckpointBarrierHandler reacts to checkpoint barrier arriving from the input channels.
|
class |
InputGateUtil
Utility for dealing with input gates.
|
class |
RecordWriterOutput<OUT>
Implementation of
Output that sends data using a RecordWriter . |
interface |
StreamingReader |
class |
StreamInputProcessor<IN>
Input reader for
OneInputStreamTask . |
class |
StreamRecordWriter<T extends IOReadableWritable>
This record writer keeps data in buffers at most for a certain timeout.
|
class |
StreamTwoInputProcessor<IN1,IN2>
Input reader for
TwoInputStreamTask . |
Modifier and Type | Class and Description |
---|---|
class |
AbstractAlignedProcessingTimeWindowOperator<KEY,IN,OUT,STATE,F extends Function>
Deprecated.
|
class |
AbstractKeyedTimePanes<Type,Key,Aggregate,Result>
Base class for a multiple key/value maps organized in panes.
|
class |
AccumulatingKeyedTimePanes<Type,Key,Result> |
class |
AccumulatingProcessingTimeWindowOperator<KEY,IN,OUT>
Deprecated.
|
class |
AggregatingKeyedTimePanes<Type,Key> |
class |
AggregatingProcessingTimeWindowOperator<KEY,IN>
Deprecated.
|
class |
EvictingWindowOperator<K,IN,OUT,W extends Window>
A
WindowOperator that also allows an Evictor to be used. |
class |
KeyMap<K,V>
A special Hash Map implementation that can be traversed efficiently in sync with other
hash maps.
|
class |
WindowOperator<K,IN,ACC,OUT,W extends Window>
An operator that implements the logic for windowing based on a
WindowAssigner and
Trigger . |
Modifier and Type | Class and Description |
---|---|
class |
BroadcastPartitioner<T>
Partitioner that selects all the output channels.
|
class |
CustomPartitionerWrapper<K,T>
Partitioner that selects the channel with a user defined partitioner function on a key.
|
class |
ForwardPartitioner<T>
Partitioner that forwards elements only to the locally running downstream operation.
|
class |
GlobalPartitioner<T>
Partitioner that sends all elements to the downstream operator with subtask ID=0;
|
class |
KeyGroupStreamPartitioner<T,K>
Partitioner selects the target channel based on the key group index.
|
class |
RebalancePartitioner<T>
Partitioner that distributes the data equally by cycling through the output
channels.
|
class |
RescalePartitioner<T>
Partitioner that distributes the data equally by cycling through the output
channels.
|
class |
ShufflePartitioner<T>
Partitioner that distributes the data equally by selecting one output channel
randomly.
|
class |
StreamPartitioner<T> |
Modifier and Type | Class and Description |
---|---|
class |
StreamElement
An element in a data stream.
|
class |
StreamElementSerializer<T>
|
class |
StreamRecord<T>
One value in a data stream.
|
Modifier and Type | Class and Description |
---|---|
class |
AsynchronousException
An exception for wrapping exceptions that are thrown by an operator in threads other than the
main compute thread of that operator.
|
class |
ExceptionInChainedOperatorException
A special exception that signifies that the cause exception came from a chained operator.
|
class |
OneInputStreamTask<IN,OUT> |
class |
OperatorChain<OUT,OP extends StreamOperator<OUT>>
The
OperatorChain contains all operators that are executed as one chain within a single
StreamTask . |
class |
OperatorStateHandles
This class holds all state handles for one operator.
|
interface |
ProcessingTimeCallback
Interface for processing-time callbacks that can be registered at a
ProcessingTimeService . |
class |
SourceStreamTask<OUT,SRC extends SourceFunction<OUT>,OP extends StreamSource<OUT,SRC>>
Task for executing streaming sources.
|
class |
StreamIterationHead<OUT> |
class |
StreamIterationTail<IN> |
class |
StreamTask<OUT,OP extends StreamOperator<OUT>>
Base class for all streaming tasks.
|
class |
StreamTaskException
An exception that is thrown by the stream vertices when encountering an
illegal condition.
|
class |
TimerException
RuntimeException for wrapping exceptions that are thrown in the timer callback of
the timer service in StreamTask . |
class |
TwoInputStreamTask<IN1,IN2,OUT> |
Modifier and Type | Class and Description |
---|---|
class |
KeySelectorUtil
Utility class that contains helper methods to manipulating
KeySelector for streaming. |
Modifier and Type | Class and Description |
---|---|
class |
FieldAccessor<T,F>
These classes encapsulate the logic of accessing a field specified by the user as either an index
or a field expression string.
|
class |
FieldAccessorFactory
Static factories for the
FieldAccessor utilities. |
Modifier and Type | Method and Description |
---|---|
static <T,F> FieldAccessor<T,F> |
FieldAccessorFactory.getAccessor(TypeInformation<T> typeInfo,
int pos,
ExecutionConfig config)
Creates a
FieldAccessor for the given field position, which can be used to get and set
the specified field on instances of this type. |
static <T,F> FieldAccessor<T,F> |
FieldAccessorFactory.getAccessor(TypeInformation<T> typeInfo,
String field,
ExecutionConfig config)
Creates a
FieldAccessor for the field that is given by a field expression,
which can be used to get and set the specified field on instances of this type. |
Modifier and Type | Class and Description |
---|---|
class |
TestingSecurityContext |
Modifier and Type | Class and Description |
---|---|
class |
AbstractCloseableRegistry<C extends Closeable,T>
This is the abstract base class for registries that allow to register instances of
Closeable , which are all
closed if this registry is closed. |
class |
CollectionUtil |
class |
ConfigurationUtil
Utility for accessing deprecated
Configuration values. |
class |
ExceptionUtils
A collection of utility functions for dealing with exceptions and exception workflows.
|
class |
FutureUtil |
class |
InstantiationUtil
Utility class to create instances from class objects and checking failure reasons.
|
interface |
IterableIterator<E>
|
class |
MavenForkNumberPrefixLayout
The logging layout used to prefix each log event with the Maven fork number.
|
interface |
Migration
Tagging interface for migration related classes.
|
interface |
MutableObjectIterator<E>
A simple iterator interface.
|
class |
NetUtils |
class |
OperatingSystem
An enumeration indicating the operating system that the JVM runs on.
|
class |
Preconditions
A collection of static utility methods to validate input.
|
class |
ReflectionUtil |
class |
SerializedValue<T>
This class is used to transfer (via serialization) objects whose classes are not available
in the system class loader.
|
class |
SimpleStringUtils
Utility class for efficient string operations on strings.
|
interface |
Visitable<T extends Visitable<T>>
This interface marks types as visitable during a traversal.
|
interface |
Visitor<T extends Visitable<T>>
A visitor encapsulates functionality that is applied to each node in the process of a traversal of a tree or DAD.
|
interface |
WrappingProxy<T> |
class |
WrappingProxyUtil |
Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.