public class MergeTreeWriter extends Object implements RecordWriter<KeyValue>, MemoryOwner
RecordWriter
to write records and generate CompactIncrement
.RecordWriter.CommitIncrement
Constructor and Description |
---|
MergeTreeWriter(boolean writeBufferSpillable,
int sortMaxFan,
org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
CompactManager compactManager,
long maxSequenceNumber,
Comparator<org.apache.flink.table.data.RowData> keyComparator,
MergeFunction<KeyValue> mergeFunction,
KeyValueFileWriterFactory writerFactory,
boolean commitForceCompact,
CoreOptions.ChangelogProducer changelogProducer) |
Modifier and Type | Method and Description |
---|---|
void |
addNewFiles(List<DataFileMeta> files)
Add files to the internal
CompactManager . |
void |
close()
Close this writer, the call will delete newly generated but not committed files.
|
void |
compact(boolean fullCompaction)
Compact files related to the writer.
|
void |
flushMemory()
Flush memory of owner, release memory.
|
long |
memoryOccupancy()
Memory occupancy size of this owner.
|
RecordWriter.CommitIncrement |
prepareCommit(boolean blocking)
Prepare for a commit.
|
void |
setMemoryPool(org.apache.flink.table.runtime.util.MemorySegmentPool memoryPool)
Set
MemorySegmentPool for the owner. |
void |
sync()
Sync the writer.
|
void |
write(KeyValue kv)
Add a key-value element to the writer.
|
public MergeTreeWriter(boolean writeBufferSpillable, int sortMaxFan, org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager, CompactManager compactManager, long maxSequenceNumber, Comparator<org.apache.flink.table.data.RowData> keyComparator, MergeFunction<KeyValue> mergeFunction, KeyValueFileWriterFactory writerFactory, boolean commitForceCompact, CoreOptions.ChangelogProducer changelogProducer)
public void setMemoryPool(org.apache.flink.table.runtime.util.MemorySegmentPool memoryPool)
MemoryOwner
MemorySegmentPool
for the owner.setMemoryPool
in interface MemoryOwner
public void write(KeyValue kv) throws Exception
RecordWriter
write
in interface RecordWriter<KeyValue>
Exception
public void compact(boolean fullCompaction) throws Exception
RecordWriter
compact
in interface RecordWriter<KeyValue>
fullCompaction
- whether to trigger full compaction or just normal compactionException
public void addNewFiles(List<DataFileMeta> files)
RecordWriter
CompactManager
.addNewFiles
in interface RecordWriter<KeyValue>
files
- files to addpublic long memoryOccupancy()
MemoryOwner
memoryOccupancy
in interface MemoryOwner
public void flushMemory() throws Exception
MemoryOwner
flushMemory
in interface MemoryOwner
Exception
public RecordWriter.CommitIncrement prepareCommit(boolean blocking) throws Exception
RecordWriter
prepareCommit
in interface RecordWriter<KeyValue>
blocking
- if this method need to wait for current compaction to completeException
public void sync() throws Exception
RecordWriter
sync
in interface RecordWriter<KeyValue>
Exception
public void close() throws Exception
RecordWriter
close
in interface RecordWriter<KeyValue>
Exception
Copyright © 2019–2023 The Apache Software Foundation. All rights reserved.