org.apache.flink.formats.avro

Class AbstractAvroBulkFormat<A,T,SplitT extends FileSourceSplit>

java.lang.Object

org.apache.flink.formats.avro.AbstractAvroBulkFormat<A,T,SplitT>

All Implemented Interfaces:

Serializable, ResultTypeQueryable<T>, BulkFormat<T,SplitT>

@Internal
public abstract class AbstractAvroBulkFormat<A,T,SplitT extends FileSourceSplit>
extends Object
implements BulkFormat<T,SplitT>

Provides a BulkFormat for Avro records.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.connector.file.src.reader.BulkFormat

BulkFormat.Reader<T>, BulkFormat.RecordIterator<T>

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.avro.Schema	readerSchema

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractAvroBulkFormat(org.apache.avro.Schema readerSchema)

Method Summary

Modifier and Type	Method and Description
protected abstract Function<A,T>	createConverter()
org.apache.flink.formats.avro.AbstractAvroBulkFormat.AvroReader	createReader(Configuration config, SplitT split) Creates a new reader that reads from the split's path starting at the split's offset and reads length bytes after the offset.
protected abstract A	createReusedAvroRecord()
boolean	isSplittable() Checks whether this format is splittable.
org.apache.flink.formats.avro.AbstractAvroBulkFormat.AvroReader	restoreReader(Configuration config, SplitT split) Creates a new reader that reads from split.path() starting at offset and reads until length bytes after the offset.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.connector.file.src.reader.BulkFormat

getProducedType

Field Detail

readerSchema

protected final org.apache.avro.Schema readerSchema

Constructor Detail

AbstractAvroBulkFormat

protected AbstractAvroBulkFormat(org.apache.avro.Schema readerSchema)

Method Detail

createReader

public org.apache.flink.formats.avro.AbstractAvroBulkFormat.AvroReader createReader(Configuration config,
                                                                                    SplitT split)
                                                                             throws IOException

Description copied from interface: BulkFormat

Creates a new reader that reads from the split's path starting at the split's offset and reads length bytes after the offset.

Specified by:

createReader in interface BulkFormat<T,SplitT extends FileSourceSplit>

Throws:

IOException

restoreReader

public org.apache.flink.formats.avro.AbstractAvroBulkFormat.AvroReader restoreReader(Configuration config,
                                                                                     SplitT split)
                                                                              throws IOException

Description copied from interface: BulkFormat

Creates a new reader that reads from split.path() starting at offset and reads until length bytes after the offset. A number of recordsToSkip records should be read and discarded after the offset. This is typically part of restoring a reader to a checkpointed position.

Specified by:

restoreReader in interface BulkFormat<T,SplitT extends FileSourceSplit>

Throws:

IOException

isSplittable

public boolean isSplittable()

Description copied from interface: BulkFormat

Checks whether this format is splittable. Splittable formats allow Flink to create multiple splits per file, so that Flink can read multiple regions of the file concurrently.

See top-level JavaDocs (section "Splitting") for details.

Specified by:

isSplittable in interface BulkFormat<T,SplitT extends FileSourceSplit>

createReusedAvroRecord

protected abstract A createReusedAvroRecord()

createConverter

protected abstract Function<A,T> createConverter()