Modifier and Type | Field and Description |
---|---|
protected static String |
AGGREGATION_MAX_COUNT
Maximum number of items to pack into an aggregated record.
|
protected static String |
COLLECTION_MAX_COUNT
Maximum number of items to pack into an PutRecords request.
|
protected static long |
DEFAULT_RATE_LIMIT
Default values for RateLimit.
|
protected static int |
DEFAULT_THREAD_POOL_SIZE
Default values for ThreadPoolSize.
|
protected static com.amazonaws.services.kinesis.producer.KinesisProducerConfiguration.ThreadingModel |
DEFAULT_THREADING_MODEL
Default value for ThreadingModel.
|
protected static String |
RATE_LIMIT
Limits the maximum allowed put rate for a shard, as a percentage of the backend limits.
|
protected static String |
THREAD_POOL_SIZE
The maximum number of threads that the native process' thread pool will be configured with.
|
protected static String |
THREADING_MODEL
The threading model that KinesisProducer will use.
|
Constructor and Description |
---|
KinesisConfigUtil() |
Modifier and Type | Method and Description |
---|---|
static Properties |
backfillConsumerKeys(Properties configProps)
A set of configuration parameters associated with the describeStreams API may be used if: 1)
an legacy client wants to consume from Kinesis 2) a current client wants to consumer from
DynamoDB streams
|
static Properties |
getV2ConsumerAsyncClientProperties(Properties configProps) |
static com.amazonaws.services.kinesis.producer.KinesisProducerConfiguration |
getValidatedProducerConfiguration(Properties config)
Validate configuration properties for
FlinkKinesisProducer , and return a constructed
KinesisProducerConfiguration. |
static Date |
parseStreamTimestampStartingPosition(Properties consumerConfig)
Parses the timestamp in which to start consuming from the stream, from the given properties.
|
static Properties |
replaceDeprecatedProducerKeys(Properties configProps)
Replace deprecated configuration properties for
FlinkKinesisProducer . |
static void |
validateAwsConfiguration(Properties config)
Validate configuration properties related to Amazon AWS service.
|
static void |
validateConsumerConfiguration(Properties config)
Validate configuration properties for
FlinkKinesisConsumer . |
static void |
validateConsumerConfiguration(Properties config,
List<String> streams)
Validate configuration properties for
FlinkKinesisConsumer . |
static void |
validateEfoConfiguration(Properties config,
List<String> streams)
Validate if the given config is a valid EFO configuration.
|
static ConsumerConfigConstants.RecordPublisherType |
validateRecordPublisherType(Properties config)
Validate the record publisher type.
|
protected static final String COLLECTION_MAX_COUNT
protected static final String AGGREGATION_MAX_COUNT
protected static final String RATE_LIMIT
protected static final String THREADING_MODEL
protected static final String THREAD_POOL_SIZE
protected static final long DEFAULT_RATE_LIMIT
protected static final com.amazonaws.services.kinesis.producer.KinesisProducerConfiguration.ThreadingModel DEFAULT_THREADING_MODEL
protected static final int DEFAULT_THREAD_POOL_SIZE
public static void validateConsumerConfiguration(Properties config)
FlinkKinesisConsumer
.public static void validateConsumerConfiguration(Properties config, List<String> streams)
FlinkKinesisConsumer
.public static ConsumerConfigConstants.RecordPublisherType validateRecordPublisherType(Properties config)
config
- config propertiesConsumerConfigConstants.RECORD_PUBLISHER_TYPE
is set, return the parsed
record publisher type. Else return polling record publisher type.public static void validateEfoConfiguration(Properties config, List<String> streams)
config
- config properties.streams
- the streams which is sent to match the EFO consumer arn if the EFO
registration mode is set to `NONE`.public static Properties replaceDeprecatedProducerKeys(Properties configProps)
FlinkKinesisProducer
. This should be
remove along with deprecated keyspublic static Properties backfillConsumerKeys(Properties configProps)
In the context of 1), the set of configurations needs to be translated to the corresponding configurations in the Kinesis listShards API. In the mean time, keep these configs since they are applicable in the context of 2), i.e., polling data from a DynamoDB stream.
configProps
- original config properties.public static com.amazonaws.services.kinesis.producer.KinesisProducerConfiguration getValidatedProducerConfiguration(Properties config)
FlinkKinesisProducer
, and return a constructed
KinesisProducerConfiguration.public static void validateAwsConfiguration(Properties config)
public static Date parseStreamTimestampStartingPosition(Properties consumerConfig)
consumerConfig
- the properties to parse timestamp frompublic static Properties getV2ConsumerAsyncClientProperties(Properties configProps)
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.