@Internal public class PulsarOrderedSourceReader<OUT> extends SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
Modifier and Type | Field and Description |
---|---|
protected org.apache.pulsar.client.admin.PulsarAdmin |
pulsarAdmin |
protected org.apache.pulsar.client.api.PulsarClient |
pulsarClient |
protected SourceConfiguration |
sourceConfiguration |
config, context, options, recordEmitter, splitFetcherManager
Constructor and Description |
---|
PulsarOrderedSourceReader(FutureCompletingBlockingQueue<RecordsWithSplitIds<PulsarMessage<OUT>>> elementsQueue,
java.util.function.Supplier<PulsarOrderedPartitionSplitReader<OUT>> splitReaderSupplier,
SourceReaderContext context,
SourceConfiguration sourceConfiguration,
org.apache.pulsar.client.api.PulsarClient pulsarClient,
org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin) |
Modifier and Type | Method and Description |
---|---|
void |
close() |
protected PulsarPartitionSplitState |
initializedState(PulsarPartitionSplit split)
When new splits are added to the reader.
|
void |
notifyCheckpointComplete(long checkpointId)
We have an empty default implementation here because most source readers do not have to
implement the method.
|
protected void |
onSplitFinished(Map<String,PulsarPartitionSplitState> finishedSplitIds)
Handles the finished splits to clean the state if needed.
|
InputStatus |
pollNext(ReaderOutput<OUT> output)
Poll the next available record into the
ReaderOutput . |
List<PulsarPartitionSplit> |
snapshotState(long checkpointId)
Checkpoint on the state of the source.
|
void |
start()
Start the reader.
|
protected PulsarPartitionSplit |
toSplitType(String splitId,
PulsarPartitionSplitState splitState)
Convert a mutable SplitStateT to immutable SplitT.
|
addSplits, getNumberOfCurrentlyAssignedSplits, handleSourceEvents, isAvailable, notifyNoMoreSplits
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
notifyCheckpointAborted
protected final SourceConfiguration sourceConfiguration
protected final org.apache.pulsar.client.api.PulsarClient pulsarClient
protected final org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin
public PulsarOrderedSourceReader(FutureCompletingBlockingQueue<RecordsWithSplitIds<PulsarMessage<OUT>>> elementsQueue, java.util.function.Supplier<PulsarOrderedPartitionSplitReader<OUT>> splitReaderSupplier, SourceReaderContext context, SourceConfiguration sourceConfiguration, org.apache.pulsar.client.api.PulsarClient pulsarClient, org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin)
public void start()
SourceReader
start
in interface SourceReader<OUT,PulsarPartitionSplit>
start
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
public InputStatus pollNext(ReaderOutput<OUT> output) throws Exception
SourceReader
ReaderOutput
.
The implementation must make sure this method is non-blocking.
Although the implementation can emit multiple records into the given ReaderOutput, it is
recommended not doing so. Instead, emit one record into the ReaderOutput and return a InputStatus.MORE_AVAILABLE
to let the caller thread know there are more records available.
pollNext
in interface SourceReader<OUT,PulsarPartitionSplit>
pollNext
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
Exception
protected void onSplitFinished(Map<String,PulsarPartitionSplitState> finishedSplitIds)
SourceReaderBase
onSplitFinished
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
public List<PulsarPartitionSplit> snapshotState(long checkpointId)
SourceReader
snapshotState
in interface SourceReader<OUT,PulsarPartitionSplit>
snapshotState
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
public void notifyCheckpointComplete(long checkpointId)
SourceReader
checkpointId
- The ID of the checkpoint that has been completed.CheckpointListener.notifyCheckpointComplete(long)
public void close() throws Exception
close
in interface AutoCloseable
Exception
protected PulsarPartitionSplitState initializedState(PulsarPartitionSplit split)
SourceReaderBase
initializedState
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
split
- a newly added split.protected PulsarPartitionSplit toSplitType(String splitId, PulsarPartitionSplitState splitState)
SourceReaderBase
toSplitType
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
splitState
- splitState.Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.