public class HiveTableInputFormat extends HadoopInputFormatCommonBase<Row,HiveTableInputSplit>
Modifier and Type | Field and Description |
---|---|
protected boolean |
fetched |
protected boolean |
hasNext |
protected org.apache.hadoop.io.Writable |
key |
protected org.apache.hadoop.io.Writable |
value |
credentials
Constructor and Description |
---|
HiveTableInputFormat(org.apache.hadoop.mapred.JobConf jobConf,
CatalogTable catalogTable,
List<HiveTablePartition> partitions) |
Modifier and Type | Method and Description |
---|---|
void |
close()
Method that marks the end of the life-cycle of an input split.
|
void |
configure(Configuration parameters)
Configures this input format.
|
HiveTableInputSplit[] |
createInputSplits(int minNumSplits)
Creates the different splits of the input that can be processed in parallel.
|
protected void |
fetchNext() |
InputSplitAssigner |
getInputSplitAssigner(HiveTableInputSplit[] inputSplits)
Gets the type of the input splits that are processed by this input format.
|
BaseStatistics |
getStatistics(BaseStatistics cachedStats)
Gets the basic statistics from the input described by this format.
|
Row |
nextRecord(Row ignore)
Reads the next record from the input.
|
void |
open(HiveTableInputSplit split)
Opens a parallel instance of the input format to work on a split.
|
boolean |
reachedEnd()
Method used to check if the end of the input is reached.
|
getCredentialsFromUGI, read, write
closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext
protected transient org.apache.hadoop.io.Writable key
protected transient org.apache.hadoop.io.Writable value
protected transient boolean fetched
protected transient boolean hasNext
public HiveTableInputFormat(org.apache.hadoop.mapred.JobConf jobConf, CatalogTable catalogTable, List<HiveTablePartition> partitions)
public void open(HiveTableInputSplit split) throws IOException
InputFormat
When this method is called, the input format it guaranteed to be configured.
split
- The split to be opened.IOException
- Thrown, if the spit could not be opened due to an I/O problem.public HiveTableInputSplit[] createInputSplits(int minNumSplits) throws IOException
InputFormat
When this method is called, the input format it guaranteed to be configured.
minNumSplits
- The minimum desired number of splits. If fewer are created, some parallel
instances may remain idle.IOException
- Thrown, when the creation of the splits was erroneous.public void configure(Configuration parameters)
InputFormat
This method is always called first on a newly instantiated input format.
parameters
- The configuration with all parameters (note: not the Flink config but the TaskConfig).public BaseStatistics getStatistics(BaseStatistics cachedStats) throws IOException
InputFormat
When this method is called, the input format it guaranteed to be configured.
cachedStats
- The statistics that were cached. May be null.IOException
public InputSplitAssigner getInputSplitAssigner(HiveTableInputSplit[] inputSplits)
InputFormat
public boolean reachedEnd() throws IOException
InputFormat
When this method is called, the input format it guaranteed to be opened.
IOException
- Thrown, if an I/O error occurred.public void close() throws IOException
InputFormat
When this method is called, the input format it guaranteed to be opened.
IOException
- Thrown, if the input could not be closed properly.protected void fetchNext() throws IOException
IOException
public Row nextRecord(Row ignore) throws IOException
InputFormat
When this method is called, the input format it guaranteed to be opened.
ignore
- Object that may be reused.IOException
- Thrown, if an I/O error occurred.Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.