org.apache.flink.api.java.typeutils.runtime

Class TupleComparatorBase<T>

java.lang.Object

org.apache.flink.api.common.typeutils.TypeComparator<T>

org.apache.flink.api.common.typeutils.CompositeTypeComparator<T>

org.apache.flink.api.java.typeutils.runtime.TupleComparatorBase<T>

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

TupleComparator

@Internal
public abstract class TupleComparatorBase<T>
extends CompositeTypeComparator<T>
implements Serializable

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected TypeComparator[]	comparators comparators for the key fields, in the same order as the key fields
protected Object[]	deserializedFields1
protected Object[]	deserializedFields2
static int[]	HASH_SALT A sequence of prime numbers to be used for salting the computed hash values.
protected boolean	invertNormKey
protected int[]	keyPositions key positions describe which fields are keys in what order
protected int	normalizableKeyPrefixLen
protected int[]	normalizedKeyLengths
protected int	numLeadingNormalizableKeys
protected TypeSerializer[]	serializers serializers to deserialize the first n fields for comparison

Constructor Summary

Constructors

Modifier	Constructor and Description
	TupleComparatorBase(int[] keyPositions, TypeComparator<?>[] comparators, TypeSerializer<?>[] serializers)
protected	TupleComparatorBase(TupleComparatorBase<T> toClone)

Method Summary

Modifier and Type	Method and Description
int	compareSerialized(DataInputView firstSource, DataInputView secondSource) Compares two records in serialized form.
int	compareToReference(TypeComparator<T> referencedComparator) This method compares the element that has been set as reference in this type accessor, to the element set as reference in the given type accessor.
void	getFlatComparator(List<TypeComparator> flatComparators)
protected int[]	getKeyPositions()
int	getNormalizeKeyLen() Gets the number of bytes that the normalized key would maximally take.
protected void	instantiateDeserializationUtils()
boolean	invertNormalizedKey() Flag whether normalized key comparisons should be inverted key should be interpreted inverted, i.e.
boolean	isNormalizedKeyPrefixOnly(int keyBytes) Checks, whether the given number of bytes for a normalized is only a prefix to determine the order of elements of the data type for which this comparator provides the comparison methods.
protected void	privateDuplicate(TupleComparatorBase<T> toClone)
T	readWithKeyDenormalization(T reuse, DataInputView source) Reads the record back while de-normalizing the key fields.
boolean	supportsNormalizedKey() Checks whether the data type supports the creation of a normalized key for comparison.
boolean	supportsSerializationWithKeyNormalization() Check whether this comparator supports to serialize the record in a format that replaces its keys by a normalized key.
void	writeWithKeyNormalization(T record, DataOutputView target) Writes the record in such a fashion that all keys are normalizing and at the beginning of the serialized data.

Methods inherited from class org.apache.flink.api.common.typeutils.CompositeTypeComparator

getFlatComparators

Methods inherited from class org.apache.flink.api.common.typeutils.TypeComparator

compare, compareAgainstReference, duplicate, equalToReference, extractKeys, hash, putNormalizedKey, setReference, supportsCompareAgainstReference

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

keyPositions

protected int[] keyPositions

key positions describe which fields are keys in what order

comparators

protected TypeComparator[] comparators

comparators for the key fields, in the same order as the key fields

normalizedKeyLengths

protected int[] normalizedKeyLengths

numLeadingNormalizableKeys

protected int numLeadingNormalizableKeys

normalizableKeyPrefixLen

protected int normalizableKeyPrefixLen

invertNormKey

protected boolean invertNormKey

serializers

protected TypeSerializer[] serializers

serializers to deserialize the first n fields for comparison

deserializedFields1

protected transient Object[] deserializedFields1

deserializedFields2

protected transient Object[] deserializedFields2

HASH_SALT

public static final int[] HASH_SALT

A sequence of prime numbers to be used for salting the computed hash values. Based on some empirical evidence, we are using a 32-element subsequence of the OEIS sequence #A068652 (numbers such that every cyclic permutation is a prime).

See Also:

http://en.wikipedia.org/wiki/List_of_prime_numbers, http://oeis.org/A068652

Constructor Detail

TupleComparatorBase

public TupleComparatorBase(int[] keyPositions,
                           TypeComparator<?>[] comparators,
                           TypeSerializer<?>[] serializers)

TupleComparatorBase

protected TupleComparatorBase(TupleComparatorBase<T> toClone)

Method Detail

privateDuplicate

protected void privateDuplicate(TupleComparatorBase<T> toClone)

getKeyPositions

protected int[] getKeyPositions()

getFlatComparator

public void getFlatComparator(List<TypeComparator> flatComparators)

Specified by:

getFlatComparator in class CompositeTypeComparator<T>

compareToReference

public int compareToReference(TypeComparator<T> referencedComparator)

Description copied from class: TypeComparator

This method compares the element that has been set as reference in this type accessor, to the element set as reference in the given type accessor. Similar to comparing two elements e1 and e2 via a comparator, this method can be used the following way.

 E e1 = ...;
 E e2 = ...;
 
 TypeComparator<E> acc1 = ...;
 TypeComparator<E> acc2 = ...;
 
 acc1.setReference(e1);
 acc2.setReference(e2);
 
 int comp = acc1.compareToReference(acc2);
 

The rational behind this method is that elements are typically compared using certain features that are extracted from them, (such de-serializing as a subset of fields). When setting the reference, this extraction happens. The extraction needs happen only once per element, even though an element is typically compared to many other elements when establishing a sorted order. The actual comparison performed by this method may be very cheap, as it happens on the extracted features.

Specified by:

compareToReference in class TypeComparator<T>

Parameters:

referencedComparator - The type accessors where the element for comparison has been set as reference.

Returns:

A value smaller than zero, if the reference value of referencedAccessors is smaller than the reference value of this type accessor; a value greater than zero, if it is larger; zero, if both are equal.

See Also:

TypeComparator.setReference(Object)

compareSerialized

public int compareSerialized(DataInputView firstSource,
                             DataInputView secondSource)
                      throws IOException

Description copied from class: TypeComparator

Compares two records in serialized form. The return value indicates the order of the two in the same way as defined by Comparator.compare(Object, Object).

This method may de-serialize the records or compare them directly based on their binary representation.

Specified by:

compareSerialized in class TypeComparator<T>

Parameters:

firstSource - The input view containing the first record.

secondSource - The input view containing the second record.

Returns:

An integer defining the oder among the objects in the same way as Comparator.compare(Object, Object).

Throws:

IOException - Thrown, if any of the input views raised an exception when reading the records.

See Also:

Comparator.compare(Object, Object)

supportsNormalizedKey

public boolean supportsNormalizedKey()

Description copied from class: TypeComparator

Checks whether the data type supports the creation of a normalized key for comparison.

Specified by:

supportsNormalizedKey in class TypeComparator<T>

Returns:

True, if the data type supports the creation of a normalized key for comparison, false otherwise.

getNormalizeKeyLen

public int getNormalizeKeyLen()

Description copied from class: TypeComparator

Gets the number of bytes that the normalized key would maximally take. A value of Integer.MAX_VALUE is interpreted as infinite.

Specified by:

getNormalizeKeyLen in class TypeComparator<T>

Returns:

The number of bytes that the normalized key would maximally take.

isNormalizedKeyPrefixOnly

public boolean isNormalizedKeyPrefixOnly(int keyBytes)

Description copied from class: TypeComparator

Checks, whether the given number of bytes for a normalized is only a prefix to determine the order of elements of the data type for which this comparator provides the comparison methods. For example, if the data type is ordered with respect to an integer value it contains, then this method would return true, if the number of key bytes is smaller than four.

Specified by:

isNormalizedKeyPrefixOnly in class TypeComparator<T>

Returns:

True, if the given number of bytes is only a prefix, false otherwise.

invertNormalizedKey

public boolean invertNormalizedKey()

Description copied from class: TypeComparator

Flag whether normalized key comparisons should be inverted key should be interpreted inverted, i.e. descending.

Specified by:

invertNormalizedKey in class TypeComparator<T>

Returns:

True, if all normalized key comparisons should invert the sign of the comparison result, false if the normalized key should be used as is.

supportsSerializationWithKeyNormalization

public boolean supportsSerializationWithKeyNormalization()

Description copied from class: TypeComparator

Check whether this comparator supports to serialize the record in a format that replaces its keys by a normalized key.

Specified by:

supportsSerializationWithKeyNormalization in class TypeComparator<T>

Returns:

True, if the comparator supports that specific form of serialization, false if not.

writeWithKeyNormalization

public void writeWithKeyNormalization(T record,
                                      DataOutputView target)
                               throws IOException

Description copied from class: TypeComparator

Writes the record in such a fashion that all keys are normalizing and at the beginning of the serialized data. This must only be used when for all the key fields the full normalized key is used. The method #supportsSerializationWithKeyNormalization() allows to check that.

Specified by:

writeWithKeyNormalization in class TypeComparator<T>

Parameters:

record - The record object into which to read the record data.

target - The stream to which to write the data,

Throws:

IOException

See Also:

TypeComparator.supportsSerializationWithKeyNormalization(), TypeComparator.readWithKeyDenormalization(Object, DataInputView), NormalizableKey.copyNormalizedKey(MemorySegment, int, int)

readWithKeyDenormalization

public T readWithKeyDenormalization(T reuse,
                                    DataInputView source)
                             throws IOException

Description copied from class: TypeComparator

Reads the record back while de-normalizing the key fields. This must only be used when for all the key fields the full normalized key is used, which is hinted by the #supportsSerializationWithKeyNormalization() method.

Specified by:

readWithKeyDenormalization in class TypeComparator<T>

Parameters:

reuse - The reuse object into which to read the record data.

source - The stream from which to read the data,

Throws:

IOException

See Also:

TypeComparator.supportsSerializationWithKeyNormalization(), TypeComparator.writeWithKeyNormalization(Object, DataOutputView), NormalizableKey.copyNormalizedKey(MemorySegment, int, int)

instantiateDeserializationUtils

protected final void instantiateDeserializationUtils()