org.apache.flink.streaming.connectors.kafka

Interface KafkaDeserializationSchema<T>

Type Parameters:

T - The type created by the keyed deserialization schema.

All Superinterfaces:

ResultTypeQueryable<T>, Serializable

All Known Subinterfaces:

KeyedDeserializationSchema<T>

All Known Implementing Classes:

JSONKeyValueDeserializationSchema, KafkaDeserializationSchemaWrapper, TypeInformationKeyValueSerializationSchema

@PublicEvolving
public interface KafkaDeserializationSchema<T>
extends Serializable, ResultTypeQueryable<T>

The deserialization schema describes how to turn the Kafka ConsumerRecords into data types (Java/Scala objects) that are processed by Flink.

Method Summary

Modifier and Type	Method and Description
T	deserialize(org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]> record) Deserializes the Kafka record.
default void	deserialize(org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]> message, Collector<T> out) Deserializes the Kafka record.
boolean	isEndOfStream(T nextElement) Method to decide whether the element signals the end of the stream.
default void	open(DeserializationSchema.InitializationContext context) Initialization method for the schema.

Methods inherited from interface org.apache.flink.api.java.typeutils.ResultTypeQueryable

getProducedType

Method Detail

open

default void open(DeserializationSchema.InitializationContext context)
           throws Exception

Initialization method for the schema. It is called before the actual working methods deserialize(org.apache.kafka.clients.consumer.ConsumerRecord<byte[], byte[]>) and thus suitable for one time setup work.

The provided DeserializationSchema.InitializationContext can be used to access additional features such as e.g. registering user metrics.

Parameters:

context - Contextual information that can be used during initialization.

Throws:

Exception

isEndOfStream

boolean isEndOfStream(T nextElement)

Method to decide whether the element signals the end of the stream. If true is returned the element won't be emitted.

Parameters:

nextElement - The element to test for the end-of-stream signal.

Returns:

True, if the element signals end of stream, false otherwise.

deserialize

T deserialize(org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]> record)
       throws Exception

Deserializes the Kafka record.

Parameters:

record - Kafka record to be deserialized.

Returns:

The deserialized message as an object (null if the message cannot be deserialized).

Throws:

Exception

deserialize

default void deserialize(org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]> message,
                         Collector<T> out)
                  throws Exception

Deserializes the Kafka record.

Can output multiple records through the Collector. Note that number and size of the produced records should be relatively small. Depending on the source implementation records can be buffered in memory or collecting records might delay emitting checkpoint barrier.

Parameters:

message - The message, as a byte array.

out - The collector to put the resulting messages.

Throws:

Exception