org.apache.flink.formats.parquet.vector.reader

Class BaseVectorizedColumnReader

java.lang.Object

org.apache.flink.formats.parquet.vector.reader.BaseVectorizedColumnReader

All Implemented Interfaces:

ColumnReader<WritableColumnVector>

Direct Known Subclasses:

ArrayColumnReader

public abstract class BaseVectorizedColumnReader
extends Object
implements ColumnReader<WritableColumnVector>

It's column level Parquet reader which is used to read a batch of records for a column, part of the code is referred from Apache Hive and Apache Parquet.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	BaseVectorizedColumnReader.NullIntIterator Reading zero always.
protected static class	BaseVectorizedColumnReader.RLEIntIterator Reading int from RunLengthBitPackingHybridDecoder.
protected static class	BaseVectorizedColumnReader.ValuesReaderIntIterator Reading int from ValuesReader.

Field Summary

Fields

Modifier and Type	Field and Description
protected ParquetDataColumnReader	dataColumn
protected int	definitionLevel
protected org.apache.flink.formats.parquet.vector.reader.BaseVectorizedColumnReader.IntIterator	definitionLevelColumn
protected org.apache.parquet.column.ColumnDescriptor	descriptor
protected ParquetDataColumnReader	dictionary The dictionary, if this column has dictionary encoding.
protected long	endOfPageValueCount value that indicates the end of the current page.
protected boolean	isCurrentPageDictionaryEncoded If true, the current page is dictionary encoded.
protected boolean	isUtcTimestamp
protected LogicalType	logicalType
protected int	maxDefLevel Maximum definition level for this column.
protected org.apache.parquet.column.page.PageReader	pageReader
protected int	pageValueCount Total values in the current page.
protected int	repetitionLevel
protected org.apache.flink.formats.parquet.vector.reader.BaseVectorizedColumnReader.IntIterator	repetitionLevelColumn Repetition/Definition/Value readers.
protected org.apache.parquet.schema.Type	type
protected long	valuesRead Total number of values read.

Constructor Summary

Constructors

Constructor and Description
BaseVectorizedColumnReader(org.apache.parquet.column.ColumnDescriptor descriptor, org.apache.parquet.column.page.PageReader pageReader, boolean isUtcTimestamp, org.apache.parquet.schema.Type parquetType, LogicalType logicalType)

Method Summary

Modifier and Type	Method and Description
protected void	readPage()
protected void	readRepetitionAndDefinitionLevels()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.formats.parquet.vector.reader.ColumnReader

readToVector

Field Detail

isUtcTimestamp

protected boolean isUtcTimestamp

valuesRead

protected long valuesRead

Total number of values read.

endOfPageValueCount

protected long endOfPageValueCount

value that indicates the end of the current page. That is, if valuesRead == endOfPageValueCount, we are at the end of the page.

dictionary

protected final ParquetDataColumnReader dictionary

The dictionary, if this column has dictionary encoding.

isCurrentPageDictionaryEncoded

protected boolean isCurrentPageDictionaryEncoded

If true, the current page is dictionary encoded.

maxDefLevel

protected final int maxDefLevel

Maximum definition level for this column.

definitionLevel

protected int definitionLevel

repetitionLevel

protected int repetitionLevel

repetitionLevelColumn

protected org.apache.flink.formats.parquet.vector.reader.BaseVectorizedColumnReader.IntIterator repetitionLevelColumn

Repetition/Definition/Value readers.

definitionLevelColumn

protected org.apache.flink.formats.parquet.vector.reader.BaseVectorizedColumnReader.IntIterator definitionLevelColumn

dataColumn

protected ParquetDataColumnReader dataColumn

pageValueCount

protected int pageValueCount

Total values in the current page.

pageReader

protected final org.apache.parquet.column.page.PageReader pageReader

descriptor

protected final org.apache.parquet.column.ColumnDescriptor descriptor

type

protected final org.apache.parquet.schema.Type type

logicalType

protected final LogicalType logicalType

Constructor Detail

BaseVectorizedColumnReader

public BaseVectorizedColumnReader(org.apache.parquet.column.ColumnDescriptor descriptor,
                                  org.apache.parquet.column.page.PageReader pageReader,
                                  boolean isUtcTimestamp,
                                  org.apache.parquet.schema.Type parquetType,
                                  LogicalType logicalType)
                           throws IOException

Throws:

IOException

Method Detail

readRepetitionAndDefinitionLevels

protected void readRepetitionAndDefinitionLevels()

readPage

protected void readPage()