Class PbFileFormatFactory
- java.lang.Object
-
- org.apache.flink.formats.protobuf.PbFileFormatFactory
-
- All Implemented Interfaces:
BulkReaderFormatFactory
,BulkWriterFormatFactory
,DecodingFormatFactory<BulkFormat<RowData,FileSourceSplit>>
,EncodingFormatFactory<BulkWriter.Factory<RowData>>
,Factory
,FormatFactory
@Internal public class PbFileFormatFactory extends Object implements BulkReaderFormatFactory, BulkWriterFormatFactory
Throw aValidationException
when using Protobuf format factory for file system.In practice, there is no standard for storing bulk protobuf messages. This factory is present to prevent falling back to the
DeserializationSchemaAdapter
, a line-based format which could silently succeed but write unrecoverable data to disk.If your use case requires storing bulk protobuf messages on disk, the parquet file format might be the appropriate container and has an API for mapping records to protobuf messages.
-
-
Constructor Summary
Constructors Constructor Description PbFileFormatFactory()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description BulkDecodingFormat<RowData>
createDecodingFormat(DynamicTableFactory.Context context, ReadableConfig formatOptions)
Creates aBulkDecodingFormat
from the given context and format options.EncodingFormat<BulkWriter.Factory<RowData>>
createEncodingFormat(DynamicTableFactory.Context context, ReadableConfig formatOptions)
Creates a format from the given context and format options.String
factoryIdentifier()
Returns a unique identifier among same factory interfaces.Set<ConfigOption<?>>
forwardOptions()
Returns a set ofConfigOption
that are directly forwarded to the runtime implementation but don't affect the final execution topology.Set<ConfigOption<?>>
optionalOptions()
Returns a set ofConfigOption
that an implementation of this factory consumes in addition toFactory.requiredOptions()
.Set<ConfigOption<?>>
requiredOptions()
Returns a set ofConfigOption
that an implementation of this factory requires in addition toFactory.optionalOptions()
.
-
-
-
Method Detail
-
factoryIdentifier
public String factoryIdentifier()
Description copied from interface:Factory
Returns a unique identifier among same factory interfaces.For consistency, an identifier should be declared as one lower case word (e.g.
kafka
). If multiple factories exist for different versions, a version should be appended using "-" (e.g.elasticsearch-7
).- Specified by:
factoryIdentifier
in interfaceFactory
-
requiredOptions
public Set<ConfigOption<?>> requiredOptions()
Description copied from interface:Factory
Returns a set ofConfigOption
that an implementation of this factory requires in addition toFactory.optionalOptions()
.See the documentation of
Factory
for more information.- Specified by:
requiredOptions
in interfaceFactory
-
optionalOptions
public Set<ConfigOption<?>> optionalOptions()
Description copied from interface:Factory
Returns a set ofConfigOption
that an implementation of this factory consumes in addition toFactory.requiredOptions()
.See the documentation of
Factory
for more information.- Specified by:
optionalOptions
in interfaceFactory
-
forwardOptions
public Set<ConfigOption<?>> forwardOptions()
Description copied from interface:FormatFactory
Returns a set ofConfigOption
that are directly forwarded to the runtime implementation but don't affect the final execution topology.Options declared here can override options of the persisted plan during an enrichment phase. Since a restored topology is static, an implementer has to ensure that the declared options don't affect fundamental abilities such as
ChangelogMode
.For example, given a JSON format, if an option defines how to parse timestamps, changing the parsing behavior does not affect the pipeline topology and can be allowed. However, an option that defines whether the format results in a
ProjectableDecodingFormat
or not is not allowed. The wrapping connector and planner might not react to the changed abilities anymore.- Specified by:
forwardOptions
in interfaceFormatFactory
- See Also:
DynamicTableFactory.Context.getEnrichmentOptions()
-
createDecodingFormat
public BulkDecodingFormat<RowData> createDecodingFormat(DynamicTableFactory.Context context, ReadableConfig formatOptions)
Description copied from interface:BulkReaderFormatFactory
Creates aBulkDecodingFormat
from the given context and format options.- Specified by:
createDecodingFormat
in interfaceBulkReaderFormatFactory
- Specified by:
createDecodingFormat
in interfaceDecodingFormatFactory<BulkFormat<RowData,FileSourceSplit>>
-
createEncodingFormat
public EncodingFormat<BulkWriter.Factory<RowData>> createEncodingFormat(DynamicTableFactory.Context context, ReadableConfig formatOptions)
Description copied from interface:EncodingFormatFactory
Creates a format from the given context and format options.The format options have been projected to top-level options (e.g. from
format.ignore-errors
toignore-errors
).- Specified by:
createEncodingFormat
in interfaceEncodingFormatFactory<BulkWriter.Factory<RowData>>
-
-