Modifier and Type | Method and Description |
---|---|
default WatermarkAlignmentParams |
WatermarkStrategy.getAlignmentParameters()
Provides configuration for watermark alignment of a maximum watermark of multiple
sources/tasks/partitions in the same watermark group.
|
default WatermarkStrategy<T> |
WatermarkStrategy.withWatermarkAlignment(String watermarkGroup,
java.time.Duration maxAllowedWatermarkDrift)
Creates a new
WatermarkStrategy that configures the maximum watermark drift from
other sources/tasks/partitions in the same watermark group. |
default WatermarkStrategy<T> |
WatermarkStrategy.withWatermarkAlignment(String watermarkGroup,
java.time.Duration maxAllowedWatermarkDrift,
java.time.Duration updateInterval)
Creates a new
WatermarkStrategy that configures the maximum watermark drift from
other sources/tasks/partitions in the same watermark group. |
Modifier and Type | Class and Description |
---|---|
class |
OutputFormatBase<OUT,V>
OutputFormatBase is the common abstract class for output formats.
|
class |
SinkUtils
Utility class for sinks.
|
Modifier and Type | Method and Description |
---|---|
default <MetaT> Optional<java.util.function.Consumer<MetaT>> |
Sink.InitContext.metadataConsumer()
Returns a metadata consumer, the
SinkWriter can publish metadata events of type
MetaT to the consumer. |
Modifier and Type | Interface and Description |
---|---|
interface |
ExternallyInducedSourceReader<T,SplitT extends SourceSplit>
Sources that implement this interface delay checkpoints when receiving a trigger message from the
checkpoint coordinator to the point when their input data/events indicate that a checkpoint
should be triggered.
|
Modifier and Type | Class and Description |
---|---|
class |
FsStateChangelogOptions
|
class |
FsStateChangelogStorage
Filesystem-based implementation of
StateChangelogStorage . |
class |
FsStateChangelogStorageForRecovery
Filesystem-based implementation of
StateChangelogStorage just for recovery. |
Modifier and Type | Class and Description |
---|---|
class |
SchedulerExecutionMode
Enum for controlling whether REACTIVE mode is enabled or not.
|
Modifier and Type | Field and Description |
---|---|
static ConfigOption<Boolean> |
DeploymentOptions.PROGRAM_CONFIG_ENABLED |
static ConfigOption<List<String>> |
DeploymentOptions.PROGRAM_CONFIG_WILDCARDS |
static ConfigOption<String> |
NettyShuffleEnvironmentOptions.SHUFFLE_COMPRESSION_CODEC
The codec to be used when compressing shuffle data.
|
static ConfigOption<Boolean> |
DeploymentOptions.SUBMIT_FAILED_JOB_ON_APPLICATION_ERROR |
Modifier and Type | Class and Description |
---|---|
class |
FileSink<IN>
A unified sink that emits its input elements to
FileSystem files within buckets. |
Modifier and Type | Interface and Description |
---|---|
interface |
PartitionCommitPolicy
Policy for commit a partition.
|
interface |
PartitionTimeExtractor
Time extractor to extract time from partition values.
|
Modifier and Type | Class and Description |
---|---|
class |
JdbcInputFormat
InputFormat to read data from a database and generate Rows.
|
class |
JdbcRowOutputFormat
OutputFormat to write Rows into a JDBC database.
|
Modifier and Type | Class and Description |
---|---|
class |
JdbcGenericParameterValuesProvider
This splits generator actually does nothing but wrapping the query parameters computed by the
user before creating the
JdbcInputFormat instance. |
class |
JdbcNumericBetweenParametersProvider
This query parameters generator is an helper class to parameterize from/to queries on a numeric
column.
|
interface |
JdbcParameterValuesProvider
This interface is used by the
JdbcInputFormat to compute the list of parallel query to
run (i.e. |
Modifier and Type | Interface and Description |
---|---|
interface |
TestResource
Basic abstractions for all resources used in connector testing framework, including
TestEnvironment annotated by TestEnv and external system annotated by TestExternalSystem . |
Modifier and Type | Interface and Description |
---|---|
interface |
ClusterControllable
Interface for triggering failover in a Flink cluster.
|
class |
MiniClusterTestEnvironment
Test environment for running jobs on Flink mini-cluster.
|
interface |
TestEnvironment
Test environment for running Flink jobs.
|
Modifier and Type | Class and Description |
---|---|
class |
DefaultContainerizedExternalSystem<C extends org.testcontainers.containers.GenericContainer<C>>
Default implementation of external system based on container.
|
interface |
ExternalContext
External context for interacting with external system in testing framework.
|
interface |
ExternalContextFactory<C extends ExternalContext>
Factory for creating
ExternalContext . |
interface |
ExternalSystemDataReader<T>
A data reader for consuming records from an external system.
|
interface |
ExternalSystemSplitDataWriter<T>
A data writer for writing records into a
SourceSplit in the external system. |
Modifier and Type | Interface and Description |
---|---|
interface |
DataStreamSinkExternalContext<T>
External context for DataStream sinks.
|
interface |
DataStreamSinkV1ExternalContext<T>
External context for DataStream sinks whose version is V1.
|
interface |
DataStreamSinkV2ExternalContext<T>
External context for DataStream sinks whose version is V2.
|
interface |
TableSinkExternalContext
External context for table sinks.
|
Modifier and Type | Interface and Description |
---|---|
interface |
DataStreamSourceExternalContext<T>
External context for DataStream sources.
|
interface |
TableSourceExternalContext
External context for table sources.
|
Modifier and Type | Class and Description |
---|---|
interface |
TestContext
Marks the field in test class defining a
ExternalContextFactory for constructing ExternalContext before invocation of each test case. |
interface |
TestEnv
Marks the field in test class defining
TestEnvironment . |
interface |
TestExternalSystem
Marks the field in test class defining external system.
|
interface |
TestSemantics
Marks the field in test class defining supported semantic:
CheckpointingMode . |
Modifier and Type | Class and Description |
---|---|
class |
SinkTestSuiteBase<T extends Comparable<T>>
Base class for sink test suite.
|
class |
SourceTestSuiteBase<T>
Base class for all test suites.
|
Modifier and Type | Class and Description |
---|---|
class |
AvroParquetReaders
A convenience builder to create
AvroParquetRecordFormat instances for the different kinds
of Avro record types. |
class |
AvroParquetWriters
Convenience builder to create
ParquetWriterFactory instances for the different Avro
types. |
Modifier and Type | Field and Description |
---|---|
static ConfigOption<Integer> |
PythonOptions.MAP_STATE_ITERATE_RESPONSE_BATCH_SIZE
The maximum number of entries sent to Python UDF worker per request when iterating a Python
MapState.
|
static ConfigOption<Integer> |
PythonOptions.MAP_STATE_READ_CACHE_SIZE
The maximum number of cached items which read from Java side in a Python MapState.
|
static ConfigOption<Integer> |
PythonOptions.MAP_STATE_WRITE_CACHE_SIZE
The maximum number of write requests cached in a Python MapState.
|
static ConfigOption<String> |
PythonOptions.PYTHON_EXECUTION_MODE
Specify the python runtime execution mode.
|
static ConfigOption<Integer> |
PythonOptions.STATE_CACHE_SIZE
The maximum number of states cached in a Python UDF worker.
|
Modifier and Type | Interface and Description |
---|---|
interface |
InboundChannelHandlerFactory
Custom netty inbound handler factory in order to make custom changes on netty inbound data.
|
interface |
OutboundChannelHandlerFactory
Custom netty outbound handler factory in order to make custom changes on netty outbound data.
|
Modifier and Type | Class and Description |
---|---|
class |
StopWithSavepointStoppingException
Exception thrown when a savepoint has been created successfully when stopping with savepoint, but
the job has not finished.
|
Modifier and Type | Interface and Description |
---|---|
interface |
DelegationTokenProvider
Delegation token provider API.
|
class |
HadoopFSDelegationTokenProvider
Delegation token provider for Hadoop filesystems.
|
Modifier and Type | Interface and Description |
---|---|
interface |
CommittableMessage<CommT>
The message send from
SinkWriter to Committer . |
class |
CommittableMessageTypeInfo<CommT>
The type information for the
CommittableMessage to make it consumable from a DataStream . |
class |
CommittableSummary<CommT>
This class tracks the information about committables belonging to one checkpoint coming from one
subtask.
|
class |
CommittableWithLineage<CommT>
Provides metadata.
|
class |
StandardSinkTopologies
This utility class provides building blocks for custom topologies.
|
interface |
WithPostCommitTopology<InputT,CommT>
Allows expert users to implement a custom topology after
Committer . |
interface |
WithPreCommitTopology<InputT,CommT>
Allows expert users to implement a custom topology after
SinkWriter and before Committer . |
interface |
WithPreWriteTopology<InputT>
Allows expert users to implement a custom topology before
SinkWriter . |
Modifier and Type | Class and Description |
---|---|
static class |
DataStream.Collector<T>
This class acts as an accessor to elements collected via
DataStream.collectAsync(Collector) . |
class |
DataStreamUtils
A collection of utilities for
DataStreams . |
class |
MultipleConnectedStreams
This is a very basic and rough stub for a class connecting multiple input
DataStream s
into one, using MultipleInputStreamOperator . |
Modifier and Type | Method and Description |
---|---|
CloseableIterator<T> |
DataStream.collectAsync()
Sets up the collection of the elements in this
DataStream , and returns an iterator
over the collected elements that can be used to retrieve elements once the job execution has
started. |
void |
DataStream.collectAsync(DataStream.Collector<T> collector)
Sets up the collection of the elements in this
DataStream , which can be retrieved
later via the given DataStream.Collector . |
Modifier and Type | Method and Description |
---|---|
void |
CheckpointConfig.enableApproximateLocalRecovery(boolean enabled)
Enables the approximate local recovery mode.
|
<OUT> DataStreamSource<OUT> |
StreamExecutionEnvironment.fromSource(Source<OUT,?,?> source,
WatermarkStrategy<OUT> timestampsAndWatermarks,
String sourceName,
TypeInformation<OUT> typeInfo)
Adds a data
Source to the environment to get a DataStream . |
boolean |
CheckpointConfig.isApproximateLocalRecoveryEnabled()
Returns whether approximate local recovery is enabled.
|
Modifier and Type | Interface and Description |
---|---|
interface |
DataGenerator<T>
Stateful and re-scalable data generator.
|
class |
DataGeneratorSource<T>
A data generator source that abstract data generator.
|
class |
RandomGenerator<T>
Random generator.
|
class |
SequenceGenerator<T>
A stateful, re-scalable
DataGenerator that emits each number from a given interval
exactly once, possibly in parallel. |
Modifier and Type | Class and Description |
---|---|
class |
AbstractInput<IN,OUT>
Base abstract implementation of
Input interface intended to be used when extending AbstractStreamOperatorV2 . |
class |
AbstractStreamOperatorFactory<OUT>
Base class for all stream operator factories.
|
class |
AbstractStreamOperatorV2<OUT>
New base class for all stream operators, intended to eventually replace
AbstractStreamOperator . |
class |
StreamOperatorParameters<OUT>
Helper class to construct
AbstractStreamOperatorV2 . |
interface |
YieldingOperatorFactory<OUT>
An operator that needs access to the
MailboxExecutor to yield to downstream operators
needs to be created through a factory implementing this interface. |
Modifier and Type | Class and Description |
---|---|
class |
FlinkKafkaShuffle
FlinkKafkaShuffle uses Kafka as a message bus to shuffle and persist data at the same
time. |
Modifier and Type | Class and Description |
---|---|
class |
CollectSink<IN>
A specialized data sink to be used by DataStreamUtils.collect().
|
class |
SocketStreamIterator<T>
An iterator that returns the data from a socket stream.
|
Modifier and Type | Interface and Description |
---|---|
interface |
Compilable
Represents an artifact that can be compiled to a
CompiledPlan . |
interface |
CompiledPlan
Represents an immutable, fully optimized, and executable entity that has been compiled from a
Table & SQL API pipeline definition.
|
class |
PlanReference
Unresolved pointer to a persisted plan.
|
static class |
PlanReference.ContentPlanReference
Plan reference to a string containing the serialized persisted plan in JSON.
|
static class |
PlanReference.FilePlanReference
Plan reference to a file in the local filesystem.
|
static class |
PlanReference.ResourcePlanReference
Plan reference to a file in the provided
ClassLoader . |
Modifier and Type | Method and Description |
---|---|
CompiledPlan |
Compilable.compilePlan()
Compiles this object into a
CompiledPlan that can be executed as one job. |
CompiledPlan |
StatementSet.compilePlan()
Compiles this object into a
CompiledPlan that can be executed as one job. |
CompiledPlan |
TableEnvironment.compilePlanSql(String stmt)
Compiles a SQL DML statement into a
CompiledPlan . |
default TableResult |
TableEnvironment.executePlan(PlanReference planReference)
Shorthand for
tEnv.loadPlan(planReference).execute() . |
CompiledPlan |
TableEnvironment.loadPlan(PlanReference planReference)
Loads a plan from a
PlanReference into a CompiledPlan . |
Modifier and Type | Method and Description |
---|---|
CompiledPlan |
StatementSetImpl.compilePlan() |
CompiledPlan |
TableEnvironmentInternal.compilePlan(List<ModifyOperation> operations) |
TableResultInternal |
TableEnvironmentInternal.executePlan(InternalPlan plan) |
String |
TableEnvironmentInternal.explainPlan(InternalPlan compiledPlan,
ExplainDetail... extraDetails) |
Modifier and Type | Class and Description |
---|---|
class |
ClientWrapperClassLoader
This class loader extends
FlinkUserCodeClassLoaders.SafetyNetWrapperClassLoader , upon the addURL method, it
also exposes a removeURL method which used to remove unnecessary jar from current
classloader path. |
Modifier and Type | Method and Description |
---|---|
InternalPlan |
Planner.compilePlan(List<ModifyOperation> modifyOperations) |
String |
Planner.explainPlan(InternalPlan plan,
ExplainDetail... extraDetails) |
InternalPlan |
Planner.loadPlan(PlanReference planReference) |
List<Transformation<?>> |
Planner.translatePlan(InternalPlan plan) |
Modifier and Type | Field and Description |
---|---|
static ConfigOption<Boolean> |
InternalConfigOptions.TABLE_EXEC_NON_TEMPORAL_SORT_ENABLED |
Modifier and Type | Class and Description |
---|---|
class |
OutputFormatTableSink<T>
Deprecated.
This interface has been replaced by
DynamicTableSink . The new interface
consumes internal data structures. See FLIP-95 for more information. |
interface |
OverwritableTableSink
Deprecated.
This interface will not be supported in the new sink design around
DynamicTableSink . Use SupportsOverwrite instead. See FLIP-95 for more information. |
interface |
PartitionableTableSink
Deprecated.
This interface will not be supported in the new sink design around
DynamicTableSink . Use SupportsPartitioning instead. See FLIP-95 for more
information. |
Modifier and Type | Class and Description |
---|---|
class |
InputFormatTableSource<T>
Deprecated.
This interface has been replaced by
DynamicTableSource . The new interface
produces internal data structures. See FLIP-95 for more information. |
interface |
LimitableTableSource<T>
Deprecated.
This interface will not be supported in the new source design around
DynamicTableSource . Use SupportsLimitPushDown instead. See FLIP-95 for more
information. |
interface |
LookupableTableSource<T>
Deprecated.
This interface will not be supported in the new source design around
DynamicTableSource . Use LookupTableSource instead. See FLIP-95 for more information. |
interface |
PartitionableTableSource
Deprecated.
This interface will not be supported in the new source design around
DynamicTableSource . Use SupportsPartitionPushDown instead. See FLIP-95 for more
information. |
Modifier and Type | Class and Description |
---|---|
interface |
InjectClusterClient
Annotate a test method parameter with this annotation to inject the
ClusterClient or the
RestClusterClient instance. |
class |
MiniClusterExtension
Starts a Flink
MiniCluster and registers the respective ExecutionEnvironment and
StreamExecutionEnvironment in the correct thread local environment. |
Constructor and Description |
---|
MiniClusterExtension(java.util.function.Supplier<org.apache.flink.runtime.testutils.MiniClusterResourceConfiguration> miniClusterResourceConfigurationSupplier) |
Modifier and Type | Class and Description |
---|---|
class |
JacksonMapperFactory
Factory for Jackson mappers.
|
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.