org.apache.flink.connector.base.source.utils

Class SerdeUtils

java.lang.Object

org.apache.flink.connector.base.source.utils.SerdeUtils

public class SerdeUtils
extends Object

A util class with some helper method for serde in the sources.

Method Summary

Modifier and Type	Method and Description
static <SplitT extends SourceSplit,C extends Collection<SplitT>> Map<Integer,C>	deserializeSplitAssignments(byte[] serialized, SimpleVersionedSerializer<SplitT> splitSerializer, java.util.function.Function<Integer,C> collectionSupplier) Deserialize the given bytes returned by serializeSplitAssignments(Map, SimpleVersionedSerializer).
static <SplitT extends SourceSplit,C extends Collection<SplitT>> byte[]	serializeSplitAssignments(Map<Integer,C> splitAssignments, SimpleVersionedSerializer<SplitT> splitSerializer) Serialize a mapping from subtask ids to lists of assigned splits.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

serializeSplitAssignments

public static <SplitT extends SourceSplit,C extends Collection<SplitT>> byte[] serializeSplitAssignments(Map<Integer,C> splitAssignments,
                                                                                                         SimpleVersionedSerializer<SplitT> splitSerializer)
                                                                                                  throws IOException

Serialize a mapping from subtask ids to lists of assigned splits. The serialized format is following:

 4 bytes - number of subtasks
 4 bytes - split serializer version
 N bytes - [assignment_for_subtask]
                4 bytes - subtask id
                4 bytes - number of assigned splits
                N bytes - [assigned_splits]
                        4 bytes - serialized split length
                        N bytes - serialized splits
 

Type Parameters:

SplitT - the type of the splits.

C - the type of the collection to hold the assigned splits for a subtask.

Parameters:

splitAssignments - a mapping from subtask ids to lists of assigned splits.

splitSerializer - the serializer of the split.

Returns:

the serialized bytes of the given subtask to splits assignment mapping.

Throws:

IOException - when serialization failed.

deserializeSplitAssignments

public static <SplitT extends SourceSplit,C extends Collection<SplitT>> Map<Integer,C> deserializeSplitAssignments(byte[] serialized,
                                                                                                                   SimpleVersionedSerializer<SplitT> splitSerializer,
                                                                                                                   java.util.function.Function<Integer,C> collectionSupplier)
                                                                                                            throws IOException

Deserialize the given bytes returned by serializeSplitAssignments(Map, SimpleVersionedSerializer).

Type Parameters:

SplitT - the type of the splits.

C - the type of the collection to hold the assigned splits for a subtask.

Parameters:

serialized - the serialized bytes returned by serializeSplitAssignments(Map, SimpleVersionedSerializer).

splitSerializer - the split serializer for the splits.

collectionSupplier - the supplier for the Collection instance to hold the assigned splits for a subtask.

Returns:

A mapping from subtask id to its assigned splits.

Throws:

IOException - when deserialization failed.