OUT
- The type of the elements produced by this data source.@Public public class DataSource<OUT> extends Operator<OUT,DataSource<OUT>>
minResources, name, parallelism, preferredResources
Constructor and Description |
---|
DataSource(ExecutionEnvironment context,
InputFormat<OUT,?> inputFormat,
TypeInformation<OUT> type,
String dataSourceLocationName)
Creates a new data source.
|
Modifier and Type | Method and Description |
---|---|
InputFormat<OUT,?> |
getInputFormat()
Gets the input format that is executed by this data source.
|
Configuration |
getParameters() |
SplitDataProperties<OUT> |
getSplitDataProperties()
Returns the
SplitDataProperties for the
InputSplit s of this DataSource
for configurations. |
protected GenericDataSourceBase<OUT,?> |
translateToDataFlow() |
DataSource<OUT> |
withParameters(Configuration parameters)
Pass a configuration to the InputFormat.
|
getMinResources, getName, getParallelism, getPreferredResources, getResultType, name, setParallelism
aggregate, checkSameExecutionContext, clean, coGroup, collect, combineGroup, count, cross, crossWithHuge, crossWithTiny, distinct, distinct, distinct, distinct, fillInType, filter, first, flatMap, fullOuterJoin, fullOuterJoin, getExecutionEnvironment, getType, groupBy, groupBy, groupBy, iterate, iterateDelta, join, join, joinWithHuge, joinWithTiny, leftOuterJoin, leftOuterJoin, map, mapPartition, max, maxBy, min, minBy, output, partitionByHash, partitionByHash, partitionByHash, partitionByRange, partitionByRange, partitionByRange, partitionCustom, partitionCustom, partitionCustom, print, print, printOnTaskManager, printToErr, printToErr, project, rebalance, reduce, reduceGroup, rightOuterJoin, rightOuterJoin, runOperation, sortPartition, sortPartition, sortPartition, sum, union, write, write, writeAsCsv, writeAsCsv, writeAsCsv, writeAsCsv, writeAsFormattedText, writeAsFormattedText, writeAsText, writeAsText
public DataSource(ExecutionEnvironment context, InputFormat<OUT,?> inputFormat, TypeInformation<OUT> type, String dataSourceLocationName)
context
- The environment in which the data source gets executed.inputFormat
- The input format that the data source executes.type
- The type of the elements produced by this input format.@Internal public InputFormat<OUT,?> getInputFormat()
public DataSource<OUT> withParameters(Configuration parameters)
parameters
- Configuration parameterspublic Configuration getParameters()
@PublicEvolving public SplitDataProperties<OUT> getSplitDataProperties()
SplitDataProperties
for the
InputSplit
s of this DataSource
for configurations.
SplitDataProperties can help to generate more efficient execution plans.
IMPORTANT: Incorrect configuration of SplitDataProperties can cause wrong results!
protected GenericDataSourceBase<OUT,?> translateToDataFlow()
Copyright © 2014–2019 The Apache Software Foundation. All rights reserved.