org.apache.flink.connector.pulsar.source.reader.deserializer

Interface PulsarDeserializationSchema<T>

Type Parameters:

T - The output message type for sinking to downstream flink operator.

All Superinterfaces:

ResultTypeQueryable<T>, Serializable

All Known Implementing Classes:

PulsarTypeInformationWrapper

@PublicEvolving
public interface PulsarDeserializationSchema<T>
extends Serializable, ResultTypeQueryable<T>

A schema bridge for deserializing the pulsar's Message<byte[]> into a flink managed instance. We support both the pulsar's self managed schema and flink managed schema.

Method Summary

Modifier and Type	Method and Description
void	deserialize(org.apache.pulsar.client.api.Message<byte[]> message, Collector<T> out) Deserializes the pulsar message.
static <T> PulsarDeserializationSchema<T>	flinkSchema(DeserializationSchema<T> deserializationSchema) Create a PulsarDeserializationSchema by using the flink's DeserializationSchema.
static <T> PulsarDeserializationSchema<T>	flinkTypeInfo(TypeInformation<T> information, ExecutionConfig config) Create a PulsarDeserializationSchema by using the given TypeInformation.
default void	open(DeserializationSchema.InitializationContext context) Initialization method for the schema.
static <K,V> PulsarDeserializationSchema<org.apache.pulsar.common.schema.KeyValue<K,V>>	pulsarSchema(org.apache.pulsar.client.api.Schema<org.apache.pulsar.common.schema.KeyValue<K,V>> schema, Class<K> keyClass, Class<V> valueClass) Create a PulsarDeserializationSchema by using the Pulsar Schema instance.
static <T> PulsarDeserializationSchema<T>	pulsarSchema(org.apache.pulsar.client.api.Schema<T> schema) Create a PulsarDeserializationSchema by using the Pulsar Schema instance.
static <T> PulsarDeserializationSchema<T>	pulsarSchema(org.apache.pulsar.client.api.Schema<T> schema, Class<T> typeClass) Create a PulsarDeserializationSchema by using the Pulsar Schema instance.

Methods inherited from interface org.apache.flink.api.java.typeutils.ResultTypeQueryable

getProducedType

Method Detail

open

default void open(DeserializationSchema.InitializationContext context)
           throws Exception

Initialization method for the schema. It is called before the actual working methods deserialize(org.apache.pulsar.client.api.Message<byte[]>, org.apache.flink.util.Collector<T>) and thus suitable for one time setup work.

The provided DeserializationSchema.InitializationContext can be used to access additional features such as e.g. registering user metrics.

Parameters:

context - Contextual information that can be used during initialization.

Throws:

Exception

deserialize

void deserialize(org.apache.pulsar.client.api.Message<byte[]> message,
                 Collector<T> out)
          throws Exception

Deserializes the pulsar message. This message could be a raw byte message or some parsed message which decoded by pulsar schema.

You can output multiple message by using the Collector. Note that number and size of the produced records should be relatively small. Depending on the source implementation records can be buffered in memory or collecting records might delay emitting checkpoint barrier.

Parameters:

message - The message decoded by pulsar.

out - The collector to put the resulting messages.

Throws:

Exception

flinkSchema

static <T> PulsarDeserializationSchema<T> flinkSchema(DeserializationSchema<T> deserializationSchema)

Create a PulsarDeserializationSchema by using the flink's DeserializationSchema. It would consume the pulsar message as byte array and decode the message by using flink's logic.

pulsarSchema

static <T> PulsarDeserializationSchema<T> pulsarSchema(org.apache.pulsar.client.api.Schema<T> schema)

Create a PulsarDeserializationSchema by using the Pulsar Schema instance. The message bytes must be encoded by pulsar Schema.

We only support primitive types here.

pulsarSchema

static <T> PulsarDeserializationSchema<T> pulsarSchema(org.apache.pulsar.client.api.Schema<T> schema,
                                                       Class<T> typeClass)

Create a PulsarDeserializationSchema by using the Pulsar Schema instance. The message bytes must be encoded by pulsar Schema.

We only support struct types here.

pulsarSchema

static <K,V> PulsarDeserializationSchema<org.apache.pulsar.common.schema.KeyValue<K,V>> pulsarSchema(org.apache.pulsar.client.api.Schema<org.apache.pulsar.common.schema.KeyValue<K,V>> schema,
                                                                                                     Class<K> keyClass,
                                                                                                     Class<V> valueClass)

Create a PulsarDeserializationSchema by using the Pulsar Schema instance. The message bytes must be encoded by pulsar Schema.

We only support keyvalue types here.

flinkTypeInfo

static <T> PulsarDeserializationSchema<T> flinkTypeInfo(TypeInformation<T> information,
                                                        ExecutionConfig config)

Create a PulsarDeserializationSchema by using the given TypeInformation. This method is only used for treating message that was written into pulsar by TypeInformation.