Package org.apache.flink.connector.testutils.source.reader

Class TestingSplitEnumeratorContext<SplitT extends SourceSplit>

    • Constructor Detail

      • TestingSplitEnumeratorContext

        public TestingSplitEnumeratorContext​(int parallelism)

    • Method Detail

      • triggerAllActions

        public void triggerAllActions()

      • registerReader

        public void registerReader​(int subtask,
                           String hostname)

      • currentParallelism

        public int currentParallelism()
        Description copied from interface: SplitEnumeratorContext
        Get the current parallelism of this Source. Note that due to auto-scaling, the parallelism may change over time. Therefore the SplitEnumerator should not cache the return value of this method, but always invoke this method to get the latest parallelism.
        Specified by:
        currentParallelism in interface SplitEnumeratorContext<SplitT extends SourceSplit>
        Returns:
        the parallelism of the Source.

      • signalNoMoreSplits

        public void signalNoMoreSplits​(int subtask)
        Description copied from interface: SplitEnumeratorContext
        Signals a subtask that it will not receive any further split.
        Specified by:
        signalNoMoreSplits in interface SplitEnumeratorContext<SplitT extends SourceSplit>
        Parameters:
        subtask - The index of the operator's parallel subtask that shall be signaled it will not receive any further split.

      • callAsync

        public <T> void callAsync​(Callable<T> callable,
                          BiConsumer<T,​Throwable> handler)
        Description copied from interface: SplitEnumeratorContext
        Invoke the callable and handover the return value to the handler which will be executed by the source coordinator. When this method is invoked multiple times, The Callable s may be executed in a thread pool concurrently.

        It is important to make sure that the callable does not modify any shared state, especially the states that will be a part of the SplitEnumerator.snapshotState(long). Otherwise, there might be unexpected behavior.

        Note that an exception thrown from the handler would result in failing the job.

        Specified by:
        callAsync in interface SplitEnumeratorContext<SplitT extends SourceSplit>
        Parameters:
        callable - a callable to call.
        handler - a handler that handles the return value of or the exception thrown from the callable.

      • callAsync

        public <T> void callAsync​(Callable<T> callable,
                          BiConsumer<T,​Throwable> handler,
                          long initialDelay,
                          long period)
        Description copied from interface: SplitEnumeratorContext
        Invoke the given callable periodically and handover the return value to the handler which will be executed by the source coordinator. When this method is invoked multiple times, The Callables may be executed in a thread pool concurrently.

        It is important to make sure that the callable does not modify any shared state, especially the states that will be a part of the SplitEnumerator.snapshotState(long). Otherwise, there might be unexpected behavior.

        Note that an exception thrown from the handler would result in failing the job.

        Specified by:
        callAsync in interface SplitEnumeratorContext<SplitT extends SourceSplit>
        Parameters:
        callable - the callable to call.
        handler - a handler that handles the return value of or the exception thrown from the callable.
        initialDelay - the initial delay of calling the callable, in milliseconds.
        period - the period between two invocations of the callable, in milliseconds.

      • runInCoordinatorThread

        public void runInCoordinatorThread​(Runnable runnable)
        Description copied from interface: SplitEnumeratorContext
        Invoke the given runnable in the source coordinator thread.

        This can be useful when the enumerator needs to execute some action (like assignSplits) triggered by some external events. E.g., Watermark from another source advanced and this source now be able to assign splits to awaiting readers. The trigger can be initiated from the coordinator thread of the other source. Instead of using lock for thread safety, this API allows to run such externally triggered action in the coordinator thread. Hence, we can ensure all enumerator actions are serialized in the single coordinator thread.

        It is important that the runnable does not block.

        Specified by:
        runInCoordinatorThread in interface SplitEnumeratorContext<SplitT extends SourceSplit>
        Parameters:
        runnable - a runnable to execute

      • setIsProcessingBacklog

        public void setIsProcessingBacklog​(boolean isProcessingBacklog)
        Description copied from interface: SplitEnumeratorContext
        Reports to JM whether this source is currently processing backlog.

        When source is processing backlog, it means the records being emitted by this source is already stale and there is no processing latency requirement for these records. This allows downstream operators to optimize throughput instead of reducing latency for intermediate results.

        If no API has been explicitly invoked to specify the backlog status of a source, the source is considered to have isProcessingBacklog=false by default.

        Specified by:
        setIsProcessingBacklog in interface SplitEnumeratorContext<SplitT extends SourceSplit>