T
- type of fetched data.public class DynamoDBStreamsDataFetcher<T> extends KinesisDataFetcher<T>
KinesisDataFetcher.FlinkKinesisProxyFactory
DEFAULT_SHARD_ASSIGNER
Constructor and Description |
---|
DynamoDBStreamsDataFetcher(List<String> streams,
SourceFunction.SourceContext<T> sourceContext,
RuntimeContext runtimeContext,
Properties configProps,
KinesisDeserializationSchema<T> deserializationSchema,
KinesisShardAssigner shardAssigner)
Constructor.
|
Modifier and Type | Method and Description |
---|---|
protected ShardConsumer |
createShardConsumer(Integer subscribedShardStateIndex,
StreamShardHandle handle,
SequenceNumber lastSeqNum,
ShardMetricsReporter shardMetricsReporter)
Create a new DynamoDB streams shard consumer.
|
protected boolean |
shouldAdvanceLastDiscoveredShardId(String shardId,
String lastSeenShardIdOfStream)
Given lastSeenShardId, check if last discovered shardId should be advanced.
|
advanceLastDiscoveredShardOfStream, awaitTermination, convertToStreamShardHandle, convertToStreamShardMetadata, createInitialSubscribedStreamsToLastDiscoveredShardsState, createShardConsumersThreadPool, discoverNewShardsToSubscribe, emitRecordAndUpdateState, emitWatermark, getClonedDeserializationSchema, getConsumerConfiguration, getCurrentTimeMillis, getSubscribedShardsState, isThisSubtaskShouldSubscribeTo, registerNewSubscribedShardState, runFetcher, shutdownFetcher, snapshotState, stopWithError, updateState
public DynamoDBStreamsDataFetcher(List<String> streams, SourceFunction.SourceContext<T> sourceContext, RuntimeContext runtimeContext, Properties configProps, KinesisDeserializationSchema<T> deserializationSchema, KinesisShardAssigner shardAssigner)
streams
- list of streams to fetch datasourceContext
- source contextruntimeContext
- runtime contextconfigProps
- config propertiesdeserializationSchema
- deserialization schemashardAssigner
- shard assignerprotected boolean shouldAdvanceLastDiscoveredShardId(String shardId, String lastSeenShardIdOfStream)
KinesisDataFetcher
shouldAdvanceLastDiscoveredShardId
in class KinesisDataFetcher<T>
protected ShardConsumer createShardConsumer(Integer subscribedShardStateIndex, StreamShardHandle handle, SequenceNumber lastSeqNum, ShardMetricsReporter shardMetricsReporter)
createShardConsumer
in class KinesisDataFetcher<T>
subscribedShardStateIndex
- the state index of the shard this consumer is subscribed tohandle
- stream handlelastSeqNum
- last sequence numbershardMetricsReporter
- the reporter to report metrics toCopyright © 2014–2020 The Apache Software Foundation. All rights reserved.