org.apache.flink.api.common.io

Class GenericInputFormat<OT>

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,GenericInputSplit>

org.apache.flink.api.common.io.GenericInputFormat<OT>

All Implemented Interfaces:

Serializable, InputFormat<OT,GenericInputSplit>, InputSplitSource<GenericInputSplit>

Direct Known Subclasses:

CollectionInputFormat, IteratorInputFormat, ParallelIteratorInputFormat

@Public
public abstract class GenericInputFormat<OT>
extends RichInputFormat<OT,GenericInputSplit>

Generic base class for all Rich inputs that are not based on files.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected int	partitionNumber The partition of this split.

Constructor Summary

Constructors

Constructor and Description
GenericInputFormat()

Method Summary

Modifier and Type	Method and Description
void	close() Method that marks the end of the life-cycle of an input split.
void	configure(Configuration parameters) Configures this input format.
GenericInputSplit[]	createInputSplits(int numSplits) Creates the different splits of the input that can be processed in parallel.
DefaultInputSplitAssigner	getInputSplitAssigner(GenericInputSplit[] splits) Gets the type of the input splits that are processed by this input format.
BaseStatistics	getStatistics(BaseStatistics cachedStatistics) Gets the basic statistics from the input described by this format.
void	open(GenericInputSplit split) Opens a parallel instance of the input format to work on a split.

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.io.InputFormat

nextRecord, reachedEnd

Field Detail

partitionNumber

protected int partitionNumber

The partition of this split.

Constructor Detail

GenericInputFormat

public GenericInputFormat()

Method Detail

configure

public void configure(Configuration parameters)

Description copied from interface: InputFormat

Configures this input format. Since input formats are instantiated generically and hence parameterless, this method is the place where the input formats set their basic fields based on configuration values.

This method is always called first on a newly instantiated input format.

Parameters:

parameters - The configuration with all parameters (note: not the Flink config but the TaskConfig).

getStatistics

public BaseStatistics getStatistics(BaseStatistics cachedStatistics)
                             throws IOException

Description copied from interface: InputFormat

Gets the basic statistics from the input described by this format. If the input format does not know how to create those statistics, it may return null. This method optionally gets a cached version of the statistics. The input format may examine them and decide whether it directly returns them without spending effort to re-gather the statistics.

When this method is called, the input format it guaranteed to be configured.

Parameters:

cachedStatistics - The statistics that were cached. May be null.

Returns:

The base statistics for the input, or null, if not available.

Throws:

IOException

createInputSplits

public GenericInputSplit[] createInputSplits(int numSplits)
                                      throws IOException

Description copied from interface: InputFormat

Creates the different splits of the input that can be processed in parallel.

When this method is called, the input format it guaranteed to be configured.

Parameters:

numSplits - The minimum desired number of splits. If fewer are created, some parallel instances may remain idle.

Returns:

The splits of this input that can be processed in parallel.

Throws:

IOException - Thrown, when the creation of the splits was erroneous.

getInputSplitAssigner

public DefaultInputSplitAssigner getInputSplitAssigner(GenericInputSplit[] splits)

Description copied from interface: InputFormat

Gets the type of the input splits that are processed by this input format.

Returns:

The type of the input splits.

open

public void open(GenericInputSplit split)
          throws IOException

Description copied from interface: InputFormat

Opens a parallel instance of the input format to work on a split.

When this method is called, the input format it guaranteed to be configured.

Parameters:

split - The split to be opened.

Throws:

IOException - Thrown, if the spit could not be opened due to an I/O problem.

close

public void close()
           throws IOException

Description copied from interface: InputFormat

Method that marks the end of the life-cycle of an input split. Should be used to close channels and streams and release resources. After this method returns without an error, the input is assumed to be correctly read.

When this method is called, the input format it guaranteed to be opened.

Throws:

IOException - Thrown, if the input could not be closed properly.