io.debezium.connector.sqlserver

Class SqlServerStreamingChangeEventSource

java.lang.Object

io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource

All Implemented Interfaces:

io.debezium.pipeline.source.spi.ChangeEventSource, io.debezium.pipeline.source.spi.StreamingChangeEventSource<io.debezium.connector.sqlserver.SqlServerPartition,io.debezium.connector.sqlserver.SqlServerOffsetContext>

Direct Known Subclasses:

SqlServerStreamFetchTask.StreamSplitReadTask

public class SqlServerStreamingChangeEventSource
extends Object
implements io.debezium.pipeline.source.spi.StreamingChangeEventSource<io.debezium.connector.sqlserver.SqlServerPartition,io.debezium.connector.sqlserver.SqlServerOffsetContext>

Copied from Debezium project(1.9.8.final) to add method afterHandleLsn(SqlServerPartition, Lsn). Also implemented SqlServerStreamingChangeEventSource#execute(ChangeEventSourceContext, SqlServerPartition, SqlServerOffsetContext). A StreamingChangeEventSource based on SQL Server change data capture functionality. A main loop polls database DDL change and change data tables and turns them into change events.

The connector uses CDC functionality of SQL Server that is implemented as as a process that monitors source table and write changes from the table into the change table.

The main loop keeps a pointer to the LSN of changes that were already processed. It queries all change tables and get result set of changes. It always finds the smallest LSN across all tables and the change is converted into the event message and sent downstream. The process repeats until all result sets are empty. The LSN is marked and the procedure repeats.

The schema changes detection follows the procedure recommended by SQL Server CDC documentation. The database operator should create one more capture process (and table) when a table schema is updated. The code detects presence of two change tables for a single source table. It decides which table is the new one depending on LSNs stored in them. The loop streams changes from the older table till there are events in new table with the LSN larger than in the old one. Then the change table is switched and streaming is executed from the new one.

Nested Class Summary

Nested classes/interfaces inherited from interface io.debezium.pipeline.source.spi.ChangeEventSource

io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext

Constructor Summary

Constructors

Constructor and Description

SqlServerStreamingChangeEventSource(io.debezium.connector.sqlserver.SqlServerConnectorConfig connectorConfig, io.debezium.connector.sqlserver.SqlServerConnection dataConnection, io.debezium.connector.sqlserver.SqlServerConnection metadataConnection, io.debezium.pipeline.EventDispatcher<io.debezium.connector.sqlserver.SqlServerPartition,io.debezium.relational.TableId> dispatcher, io.debezium.pipeline.ErrorHandler errorHandler, io.debezium.util.Clock clock, io.debezium.connector.sqlserver.SqlServerDatabaseSchema schema)

Method Summary

Modifier and Type	Method and Description
protected void	afterHandleLsn(io.debezium.connector.sqlserver.SqlServerPartition partition, io.debezium.connector.sqlserver.Lsn toLsn) expose control to the user to stop the connector.
void	execute(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext context, io.debezium.connector.sqlserver.SqlServerPartition partition, io.debezium.connector.sqlserver.SqlServerOffsetContext offsetContext)
boolean	executeIteration(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext context, io.debezium.connector.sqlserver.SqlServerPartition partition, io.debezium.connector.sqlserver.SqlServerOffsetContext offsetContext)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.debezium.pipeline.source.spi.StreamingChangeEventSource

commitOffset, init

Constructor Detail

SqlServerStreamingChangeEventSource

public SqlServerStreamingChangeEventSource(io.debezium.connector.sqlserver.SqlServerConnectorConfig connectorConfig,
                                           io.debezium.connector.sqlserver.SqlServerConnection dataConnection,
                                           io.debezium.connector.sqlserver.SqlServerConnection metadataConnection,
                                           io.debezium.pipeline.EventDispatcher<io.debezium.connector.sqlserver.SqlServerPartition,io.debezium.relational.TableId> dispatcher,
                                           io.debezium.pipeline.ErrorHandler errorHandler,
                                           io.debezium.util.Clock clock,
                                           io.debezium.connector.sqlserver.SqlServerDatabaseSchema schema)

Method Detail

execute

public void execute(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext context,
                    io.debezium.connector.sqlserver.SqlServerPartition partition,
                    io.debezium.connector.sqlserver.SqlServerOffsetContext offsetContext)
             throws InterruptedException

Specified by:

execute in interface io.debezium.pipeline.source.spi.StreamingChangeEventSource<io.debezium.connector.sqlserver.SqlServerPartition,io.debezium.connector.sqlserver.SqlServerOffsetContext>

Throws:

InterruptedException

executeIteration

public boolean executeIteration(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext context,
                                io.debezium.connector.sqlserver.SqlServerPartition partition,
                                io.debezium.connector.sqlserver.SqlServerOffsetContext offsetContext)
                         throws InterruptedException

Specified by:

executeIteration in interface io.debezium.pipeline.source.spi.StreamingChangeEventSource<io.debezium.connector.sqlserver.SqlServerPartition,io.debezium.connector.sqlserver.SqlServerOffsetContext>

Throws:

InterruptedException

afterHandleLsn

protected void afterHandleLsn(io.debezium.connector.sqlserver.SqlServerPartition partition,
                              io.debezium.connector.sqlserver.Lsn toLsn)

expose control to the user to stop the connector.