org.apache.flink.connector.pulsar.source.config

Class SourceConfiguration

java.lang.Object

org.apache.flink.connector.pulsar.source.config.SourceConfiguration

All Implemented Interfaces:

Serializable

@PublicEvolving
public class SourceConfiguration
extends Object
implements Serializable

The configure class for pulsar source.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SourceConfiguration(Configuration configuration)

Method Summary

Modifier and Type	Method and Description
boolean	equals(Object o)
long	getAutoCommitCursorInterval() The interval in millis for acknowledge message when you enable isEnableAutoAcknowledgeMessage() and use SubscriptionType.Failover or SubscriptionType.Exclusive as your consuming subscription type.
int	getMaxFetchRecords() The fetch counts for a split reader.
java.time.Duration	getMaxFetchTime() The fetch time for flink split reader polling message.
int	getMessageQueueCapacity() The capacity of the element queue in the source reader.
long	getPartitionDiscoveryIntervalMs() The interval in millis for flink querying topic partition information.
String	getSubscriptionDesc() Convert the subscription into a readable str.
org.apache.pulsar.client.api.SubscriptionMode	getSubscriptionMode() The pulsar's subscription mode for this flink source.
String	getSubscriptionName() The pulsar's subscription name for this flink source.
org.apache.pulsar.client.api.SubscriptionType	getSubscriptionType() The pulsar's subscription type for this flink source.
long	getTransactionTimeoutMillis() Pulsar's transaction have a timeout mechanism for uncommitted transaction.
CursorVerification	getVerifyInitialOffsets() Validate the CursorPosition generated by StartCursor.
int	hashCode()
boolean	isEnableAutoAcknowledgeMessage() This is used for all subscription type.
boolean	isEnablePartitionDiscovery() We would override the interval into a negative number when we set the connector with bounded stop cursor.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SourceConfiguration

public SourceConfiguration(Configuration configuration)

Method Detail

getMessageQueueCapacity

public int getMessageQueueCapacity()

The capacity of the element queue in the source reader.

isEnablePartitionDiscovery

public boolean isEnablePartitionDiscovery()

We would override the interval into a negative number when we set the connector with bounded stop cursor.

getPartitionDiscoveryIntervalMs

public long getPartitionDiscoveryIntervalMs()

The interval in millis for flink querying topic partition information.

isEnableAutoAcknowledgeMessage

public boolean isEnableAutoAcknowledgeMessage()

This is used for all subscription type. But the behavior may not be the same among them. If you don't enable the flink checkpoint, make sure this option is set to true.

• SubscriptionType.Shared and SubscriptionType.Key_Shared would immediately acknowledge the message after consuming it.
• SubscriptionType.Failover and SubscriptionType.Exclusive would perform a incremental acknowledge in a fixed getAutoCommitCursorInterval().

getAutoCommitCursorInterval

public long getAutoCommitCursorInterval()

The interval in millis for acknowledge message when you enable isEnableAutoAcknowledgeMessage() and use SubscriptionType.Failover or SubscriptionType.Exclusive as your consuming subscription type.

getTransactionTimeoutMillis

public long getTransactionTimeoutMillis()

Pulsar's transaction have a timeout mechanism for uncommitted transaction. We use transaction for SubscriptionType.Shared and SubscriptionType.Key_Shared when user disable isEnableAutoAcknowledgeMessage() and enable flink checkpoint. Since the checkpoint interval couldn't be acquired from SourceReaderContext.getConfiguration(), we have to expose this option. Make sure this value is greater than the checkpoint interval.

getMaxFetchTime

public java.time.Duration getMaxFetchTime()

The fetch time for flink split reader polling message. We would stop polling message and return the message in RecordsWithSplitIds when timeout or exceed the getMaxFetchRecords().

getMaxFetchRecords

public int getMaxFetchRecords()

The fetch counts for a split reader. We would stop polling message and return the message in RecordsWithSplitIds when timeout getMaxFetchTime() or exceed this value.

getVerifyInitialOffsets

public CursorVerification getVerifyInitialOffsets()

Validate the CursorPosition generated by StartCursor.

getSubscriptionName

public String getSubscriptionName()

The pulsar's subscription name for this flink source. All the readers would share this subscription name.

See Also:

ConsumerBuilder.subscriptionName(java.lang.String)

getSubscriptionType

public org.apache.pulsar.client.api.SubscriptionType getSubscriptionType()

The pulsar's subscription type for this flink source. All the readers would share this subscription type.

See Also:

SubscriptionType

getSubscriptionMode

public org.apache.pulsar.client.api.SubscriptionMode getSubscriptionMode()

The pulsar's subscription mode for this flink source. All the readers would share this subscription mode.

See Also:

SubscriptionMode

getSubscriptionDesc

public String getSubscriptionDesc()

Convert the subscription into a readable str.

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object