org.apache.flink.formats.parquet

Class ParquetFileFormatFactory.ParquetBulkDecodingFormat

java.lang.Object

org.apache.flink.formats.parquet.ParquetFileFormatFactory.ParquetBulkDecodingFormat

All Implemented Interfaces:

BulkDecodingFormat<RowData>, DecodingFormat<BulkFormat<RowData,FileSourceSplit>>, FileBasedStatisticsReportableInputFormat, Format, ProjectableDecodingFormat<BulkFormat<RowData,FileSourceSplit>>

Enclosing class:

ParquetFileFormatFactory

@VisibleForTesting
public static class ParquetFileFormatFactory.ParquetBulkDecodingFormat
extends Object
implements ProjectableDecodingFormat<BulkFormat<RowData,FileSourceSplit>>, BulkDecodingFormat<RowData>, FileBasedStatisticsReportableInputFormat

ParquetBulkDecodingFormat which implements FileBasedStatisticsReportableInputFormat.

Constructor Summary

Constructors

Constructor and Description
ParquetBulkDecodingFormat(ReadableConfig formatOptions)

Method Summary

Modifier and Type	Method and Description
BulkFormat<RowData,FileSourceSplit>	createRuntimeDecoder(DynamicTableSource.Context sourceContext, DataType producedDataType, int[][] projections) Creates runtime decoder implementation that is configured to produce data of type Projection.of(projections).project(physicalDataType).
ChangelogMode	getChangelogMode() Returns the set of changes that a connector (and transitively the planner) can expect during runtime.
TableStats	reportStatistics(List<Path> files, DataType producedDataType) Returns the estimated statistics of this input format.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.connector.format.ProjectableDecodingFormat

createRuntimeDecoder, supportsNestedProjection

Methods inherited from interface org.apache.flink.connector.file.table.format.BulkDecodingFormat

applyFilters

Methods inherited from interface org.apache.flink.table.connector.format.DecodingFormat

applyReadableMetadata, listReadableMetadata

Constructor Detail

ParquetBulkDecodingFormat

public ParquetBulkDecodingFormat(ReadableConfig formatOptions)

Method Detail

createRuntimeDecoder

public BulkFormat<RowData,FileSourceSplit> createRuntimeDecoder(DynamicTableSource.Context sourceContext,
                                                                DataType producedDataType,
                                                                int[][] projections)

Description copied from interface: ProjectableDecodingFormat

Creates runtime decoder implementation that is configured to produce data of type Projection.of(projections).project(physicalDataType). For more details on the usage, check DecodingFormat documentation.

Specified by:

createRuntimeDecoder in interface ProjectableDecodingFormat<BulkFormat<RowData,FileSourceSplit>>

Parameters:

sourceContext - the context provides several utilities required to instantiate the runtime decoder implementation of the format

producedDataType - For more details check DecodingFormat

projections - the projections array. The array represents the mapping of the fields of the original DataType, including nested rows. For example, [[0, 2, 1], ...] specifies to include the 2nd field of the 3rd field of the 1st field in the top-level row. It's guaranteed that this array won't contain nested projections if ProjectableDecodingFormat.supportsNestedProjection() returns false. For more details, check Projection as well.

Returns:

the runtime decoder

See Also:

DecodingFormat

getChangelogMode

public ChangelogMode getChangelogMode()

Description copied from interface: Format

Returns the set of changes that a connector (and transitively the planner) can expect during runtime.

Specified by:

getChangelogMode in interface Format

reportStatistics

public TableStats reportStatistics(List<Path> files,
                                   DataType producedDataType)

Description copied from interface: FileBasedStatisticsReportableInputFormat

Returns the estimated statistics of this input format.

Specified by:

reportStatistics in interface FileBasedStatisticsReportableInputFormat

Parameters:

files - The files to be estimated.

producedDataType - the final output type of the format.