public class Generator extends Object implements InputFormat<Tuple2<String,Integer>,GenericInputSplit>
Modifier and Type | Method and Description |
---|---|
void |
close()
Method that marks the end of the life-cycle of an input split.
|
void |
configure(Configuration parameters)
Configures this input format.
|
GenericInputSplit[] |
createInputSplits(int minNumSplits)
Computes the input splits.
|
static Generator |
generate(long numKeys,
int recordsPerKey) |
static Generator |
generateInfinitely(long numKeys) |
InputSplitAssigner |
getInputSplitAssigner(GenericInputSplit[] inputSplits)
Returns the assigner for the input splits.
|
BaseStatistics |
getStatistics(BaseStatistics cachedStatistics)
Gets the basic statistics from the input described by this format.
|
Tuple2<String,Integer> |
nextRecord(Tuple2<String,Integer> reuse)
Reads the next record from the input.
|
void |
open(GenericInputSplit split)
Opens a parallel instance of the input format to work on a split.
|
boolean |
reachedEnd()
Method used to check if the end of the input is reached.
|
public static Generator generate(long numKeys, int recordsPerKey)
public static Generator generateInfinitely(long numKeys)
public void configure(Configuration parameters)
InputFormat
This method is always called first on a newly instantiated input format.
configure
in interface InputFormat<Tuple2<String,Integer>,GenericInputSplit>
parameters
- The configuration with all parameters (note: not the Flink config but the
TaskConfig).public BaseStatistics getStatistics(BaseStatistics cachedStatistics)
InputFormat
When this method is called, the input format is guaranteed to be configured.
getStatistics
in interface InputFormat<Tuple2<String,Integer>,GenericInputSplit>
cachedStatistics
- The statistics that were cached. May be null.public GenericInputSplit[] createInputSplits(int minNumSplits)
InputSplitSource
createInputSplits
in interface InputFormat<Tuple2<String,Integer>,GenericInputSplit>
createInputSplits
in interface InputSplitSource<GenericInputSplit>
minNumSplits
- Number of minimal input splits, as a hint.public InputSplitAssigner getInputSplitAssigner(GenericInputSplit[] inputSplits)
InputSplitSource
getInputSplitAssigner
in interface InputFormat<Tuple2<String,Integer>,GenericInputSplit>
getInputSplitAssigner
in interface InputSplitSource<GenericInputSplit>
public void open(GenericInputSplit split) throws IOException
InputFormat
When this method is called, the input format it guaranteed to be configured.
open
in interface InputFormat<Tuple2<String,Integer>,GenericInputSplit>
split
- The split to be opened.IOException
- Thrown, if the spit could not be opened due to an I/O problem.public boolean reachedEnd()
InputFormat
When this method is called, the input format it guaranteed to be opened.
reachedEnd
in interface InputFormat<Tuple2<String,Integer>,GenericInputSplit>
public Tuple2<String,Integer> nextRecord(Tuple2<String,Integer> reuse) throws IOException
InputFormat
When this method is called, the input format it guaranteed to be opened.
nextRecord
in interface InputFormat<Tuple2<String,Integer>,GenericInputSplit>
reuse
- Object that may be reused.IOException
- Thrown, if an I/O error occurred.public void close()
InputFormat
When this method is called, the input format it guaranteed to be opened.
close
in interface InputFormat<Tuple2<String,Integer>,GenericInputSplit>
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.