OUT
- the type of the pulsar source message that would be serialized to downstream.@Internal public class PulsarUnorderedPartitionSplitReader<OUT> extends Object
PulsarPartitionSplit
, it would be closed once the PulsarUnorderedSourceReader
is closed.Modifier and Type | Field and Description |
---|---|
protected PulsarDeserializationSchema<OUT> |
deserializationSchema |
protected org.apache.pulsar.client.admin.PulsarAdmin |
pulsarAdmin |
protected org.apache.pulsar.client.api.PulsarClient |
pulsarClient |
protected org.apache.pulsar.client.api.Consumer<byte[]> |
pulsarConsumer |
protected PulsarPartitionSplit |
registeredSplit |
protected SourceConfiguration |
sourceConfiguration |
Constructor and Description |
---|
PulsarUnorderedPartitionSplitReader(org.apache.pulsar.client.api.PulsarClient pulsarClient,
org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin,
SourceConfiguration sourceConfiguration,
PulsarDeserializationSchema<OUT> deserializationSchema,
org.apache.pulsar.client.api.transaction.TransactionCoordinatorClient coordinatorClient) |
Modifier and Type | Method and Description |
---|---|
protected void |
afterCreatingConsumer(PulsarPartitionSplit split,
org.apache.pulsar.client.api.Consumer<byte[]> consumer) |
protected void |
beforeCreatingConsumer(PulsarPartitionSplit split) |
void |
close()
Close the split reader.
|
protected org.apache.pulsar.client.api.Consumer<byte[]> |
createPulsarConsumer(PulsarPartitionSplit split)
Create a specified
Consumer by the given split information. |
protected org.apache.pulsar.client.api.Consumer<byte[]> |
createPulsarConsumer(TopicPartition partition)
Create a specified
Consumer by the given topic partition. |
RecordsWithSplitIds<PulsarMessage<OUT>> |
fetch()
Fetch elements into the blocking queue for the given splits.
|
protected void |
finishedPollMessage(org.apache.pulsar.client.api.Message<byte[]> message) |
void |
handleSplitsChanges(SplitsChange<PulsarPartitionSplit> splitsChanges)
Handle the split changes.
|
protected org.apache.pulsar.client.api.Message<byte[]> |
pollMessage(java.time.Duration timeout) |
Optional<PulsarPartitionSplitState> |
snapshotState() |
void |
wakeUp()
Wake up the split reader in case the fetcher thread is blocking in
SplitReader.fetch() . |
protected final org.apache.pulsar.client.api.PulsarClient pulsarClient
protected final org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin
protected final SourceConfiguration sourceConfiguration
protected final PulsarDeserializationSchema<OUT> deserializationSchema
protected org.apache.pulsar.client.api.Consumer<byte[]> pulsarConsumer
protected PulsarPartitionSplit registeredSplit
public PulsarUnorderedPartitionSplitReader(org.apache.pulsar.client.api.PulsarClient pulsarClient, org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin, SourceConfiguration sourceConfiguration, PulsarDeserializationSchema<OUT> deserializationSchema, org.apache.pulsar.client.api.transaction.TransactionCoordinatorClient coordinatorClient)
protected org.apache.pulsar.client.api.Message<byte[]> pollMessage(java.time.Duration timeout) throws ExecutionException, InterruptedException, org.apache.pulsar.client.api.PulsarClientException
ExecutionException
InterruptedException
org.apache.pulsar.client.api.PulsarClientException
protected void finishedPollMessage(org.apache.pulsar.client.api.Message<byte[]> message)
protected void afterCreatingConsumer(PulsarPartitionSplit split, org.apache.pulsar.client.api.Consumer<byte[]> consumer)
public Optional<PulsarPartitionSplitState> snapshotState()
public RecordsWithSplitIds<PulsarMessage<OUT>> fetch() throws IOException
SplitReader
SplitReader.wakeUp()
is invoked. In that case, the
implementation may either decide to return without throwing an exception, or it can just
throw an interrupted exception. In either case, this method should be reentrant, meaning that
the next fetch call should just resume from where the last fetch call was waken up or
interrupted.fetch
in interface SplitReader<PulsarMessage<OUT>,PulsarPartitionSplit>
IOException
- when encountered IO errors, such as deserialization failures.public void handleSplitsChanges(SplitsChange<PulsarPartitionSplit> splitsChanges)
SplitReader
For the consistency of internal state in SourceReaderBase, if an invalid split is added to
the reader (for example splits without any records), it should be put back into RecordsWithSplitIds
as finished splits so that SourceReaderBase could be able to clean up
resources created for it.
handleSplitsChanges
in interface SplitReader<PulsarMessage<OUT>,PulsarPartitionSplit>
splitsChanges
- the split changes that the SplitReader needs to handle.public void wakeUp()
SplitReader
SplitReader.fetch()
.wakeUp
in interface SplitReader<PulsarMessage<OUT>,PulsarPartitionSplit>
public void close()
SplitReader
close
in interface SplitReader<PulsarMessage<OUT>,PulsarPartitionSplit>
protected void beforeCreatingConsumer(PulsarPartitionSplit split)
protected org.apache.pulsar.client.api.Consumer<byte[]> createPulsarConsumer(PulsarPartitionSplit split)
Consumer
by the given split information.protected org.apache.pulsar.client.api.Consumer<byte[]> createPulsarConsumer(TopicPartition partition)
Consumer
by the given topic partition.Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.