@Internal public class PulsarUnorderedSourceReader<OUT> extends SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
Modifier and Type | Field and Description |
---|---|
protected org.apache.pulsar.client.admin.PulsarAdmin |
pulsarAdmin |
protected org.apache.pulsar.client.api.PulsarClient |
pulsarClient |
protected SourceConfiguration |
sourceConfiguration |
config, context, options, recordEmitter, splitFetcherManager
Constructor and Description |
---|
PulsarUnorderedSourceReader(FutureCompletingBlockingQueue<RecordsWithSplitIds<PulsarMessage<OUT>>> elementsQueue,
java.util.function.Supplier<PulsarUnorderedPartitionSplitReader<OUT>> splitReaderSupplier,
SourceReaderContext context,
SourceConfiguration sourceConfiguration,
org.apache.pulsar.client.api.PulsarClient pulsarClient,
org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin,
org.apache.pulsar.client.api.transaction.TransactionCoordinatorClient coordinatorClient) |
Modifier and Type | Method and Description |
---|---|
void |
addSplits(List<PulsarPartitionSplit> splits)
Adds a list of splits for this reader to read.
|
void |
close() |
protected PulsarPartitionSplitState |
initializedState(PulsarPartitionSplit split)
When new splits are added to the reader.
|
void |
notifyCheckpointComplete(long checkpointId)
We have an empty default implementation here because most source readers do not have to
implement the method.
|
protected void |
onSplitFinished(Map<String,PulsarPartitionSplitState> finishedSplitIds)
Handles the finished splits to clean the state if needed.
|
List<PulsarPartitionSplit> |
snapshotState(long checkpointId)
Checkpoint on the state of the source.
|
void |
start()
Start the reader.
|
protected PulsarPartitionSplit |
toSplitType(String splitId,
PulsarPartitionSplitState splitState)
Convert a mutable SplitStateT to immutable SplitT.
|
getNumberOfCurrentlyAssignedSplits, handleSourceEvents, isAvailable, notifyNoMoreSplits, pollNext
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
notifyCheckpointAborted
protected final SourceConfiguration sourceConfiguration
protected final org.apache.pulsar.client.api.PulsarClient pulsarClient
protected final org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin
public PulsarUnorderedSourceReader(FutureCompletingBlockingQueue<RecordsWithSplitIds<PulsarMessage<OUT>>> elementsQueue, java.util.function.Supplier<PulsarUnorderedPartitionSplitReader<OUT>> splitReaderSupplier, SourceReaderContext context, SourceConfiguration sourceConfiguration, org.apache.pulsar.client.api.PulsarClient pulsarClient, org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin, @Nullable org.apache.pulsar.client.api.transaction.TransactionCoordinatorClient coordinatorClient)
public void start()
SourceReader
start
in interface SourceReader<OUT,PulsarPartitionSplit>
start
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
public void addSplits(List<PulsarPartitionSplit> splits)
SourceReader
SplitEnumeratorContext.assignSplit(SourceSplit, int)
or SplitEnumeratorContext.assignSplits(SplitsAssignment)
.addSplits
in interface SourceReader<OUT,PulsarPartitionSplit>
addSplits
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
splits
- The splits assigned by the split enumerator.protected void onSplitFinished(Map<String,PulsarPartitionSplitState> finishedSplitIds)
SourceReaderBase
onSplitFinished
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
public List<PulsarPartitionSplit> snapshotState(long checkpointId)
SourceReader
snapshotState
in interface SourceReader<OUT,PulsarPartitionSplit>
snapshotState
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
public void notifyCheckpointComplete(long checkpointId) throws Exception
SourceReader
checkpointId
- The ID of the checkpoint that has been completed.Exception
- This method can propagate exceptions, which leads to a failure/recovery for
the task. Note that this will NOT lead to the checkpoint being revoked.CheckpointListener.notifyCheckpointComplete(long)
protected PulsarPartitionSplitState initializedState(PulsarPartitionSplit split)
SourceReaderBase
initializedState
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
split
- a newly added split.protected PulsarPartitionSplit toSplitType(String splitId, PulsarPartitionSplitState splitState)
SourceReaderBase
toSplitType
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
splitState
- splitState.public void close() throws Exception
close
in interface AutoCloseable
close
in class SourceReaderBase<PulsarMessage<OUT>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>
Exception
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.