org.apache.flink.table.runtime.operators.sort

Class BufferedKVExternalSorter

java.lang.Object

org.apache.flink.table.runtime.operators.sort.BufferedKVExternalSorter

public class BufferedKVExternalSorter
extends Object

Sorter for buffered input in the form of Key-Value Style. First, sort and spill buffered inputs (without data copy, just write index and normalized key). Second, merge disk outputs and return iterator.

For Hash Aggregation：We store the data in MemorySegmentHashTable in KeyValue format. When memory is not enough, we spill all the data in memory onto disk and degenerate it into Sort Aggregation. So we need a BufferedKVExternalSorter to write the data that already in memory to disk, and then carry out SortMerge.

Constructor Summary

Constructors

Constructor and Description
BufferedKVExternalSorter(IOManager ioManager, BinaryRowDataSerializer keySerializer, BinaryRowDataSerializer valueSerializer, NormalizedKeyComputer nKeyComputer, RecordComparator comparator, int pageSize, int maxNumFileHandles, boolean compressionEnabled, int compressionBlockSize)

Method Summary

Modifier and Type	Method and Description
void	close()
MutableObjectIterator<Tuple2<BinaryRowData,BinaryRowData>>	getKVIterator()
void	sortAndSpill(ArrayList<MemorySegment> recordBufferSegments, long numElements, MemorySegmentPool pool)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BufferedKVExternalSorter

public BufferedKVExternalSorter(IOManager ioManager,
                                BinaryRowDataSerializer keySerializer,
                                BinaryRowDataSerializer valueSerializer,
                                NormalizedKeyComputer nKeyComputer,
                                RecordComparator comparator,
                                int pageSize,
                                int maxNumFileHandles,
                                boolean compressionEnabled,
                                int compressionBlockSize)

Method Detail

getKVIterator

public MutableObjectIterator<Tuple2<BinaryRowData,BinaryRowData>> getKVIterator()
                                                                         throws IOException

Throws:

IOException

sortAndSpill

public void sortAndSpill(ArrayList<MemorySegment> recordBufferSegments,
                         long numElements,
                         MemorySegmentPool pool)
                  throws IOException

Throws:

IOException

close

public void close()