org.apache.flink.streaming.connectors.cassandra

Class CassandraTupleWriteAheadSink<IN extends Tuple>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<IN>

org.apache.flink.streaming.runtime.operators.GenericWriteAheadSink<IN>

org.apache.flink.streaming.connectors.cassandra.CassandraTupleWriteAheadSink<IN>

Type Parameters:

IN - Type of the elements emitted by this sink

All Implemented Interfaces:

Serializable, CheckpointListener, KeyContext, OneInputStreamOperator<IN,IN>, SetupableStreamOperator<IN>, StreamOperator<IN>, StreamOperatorStateHandler.CheckpointedStreamOperator, Disposable

public class CassandraTupleWriteAheadSink<IN extends Tuple>
extends GenericWriteAheadSink<IN>

Sink that emits its input elements into a Cassandra database. This sink stores incoming records within a AbstractStateBackend, and only commits them to cassandra if a checkpoint is completed.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected com.datastax.driver.core.Cluster	cluster
protected com.datastax.driver.core.Session	session

Fields inherited from class org.apache.flink.streaming.runtime.operators.GenericWriteAheadSink

LOG, serializer

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, latencyStats, metrics, output, processingTimeService

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	CassandraTupleWriteAheadSink(String insertQuery, TypeSerializer<IN> serializer, ClusterBuilder builder, CheckpointCommitter committer)

Method Summary

Modifier and Type	Method and Description
void	close() This method is called after all records have been added to the operators via the methods OneInputStreamOperator.processElement(StreamRecord), or TwoInputStreamOperator.processElement1(StreamRecord) and TwoInputStreamOperator.processElement2(StreamRecord).
void	open() This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g.
protected boolean	sendValues(Iterable<IN> values, long checkpointId, long timestamp) Write the given element into the backend.

Methods inherited from class org.apache.flink.streaming.runtime.operators.GenericWriteAheadSink

initializeState, notifyCheckpointComplete, processElement, snapshotState

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

dispose, getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, numEventTimeTimers, numProcessingTimeTimers, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

processLatencyMarker, processWatermark

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

dispose, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from interface org.apache.flink.runtime.state.CheckpointListener

notifyCheckpointAborted

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Field Detail

cluster

protected transient com.datastax.driver.core.Cluster cluster

session

protected transient com.datastax.driver.core.Session session

Constructor Detail

CassandraTupleWriteAheadSink

protected CassandraTupleWriteAheadSink(String insertQuery,
                                       TypeSerializer<IN> serializer,
                                       ClusterBuilder builder,
                                       CheckpointCommitter committer)
                                throws Exception

Throws:

Exception

Method Detail

open

public void open()
          throws Exception

Description copied from class: AbstractStreamOperator

This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.

The default implementation does nothing.

Specified by:

open in interface StreamOperator<IN extends Tuple>

Overrides:

open in class GenericWriteAheadSink<IN extends Tuple>

Throws:

Exception - An exception in this method causes the operator to fail.

close

public void close()
           throws Exception

Description copied from class: AbstractStreamOperator

This method is called after all records have been added to the operators via the methods OneInputStreamOperator.processElement(StreamRecord), or TwoInputStreamOperator.processElement1(StreamRecord) and TwoInputStreamOperator.processElement2(StreamRecord).

The method is expected to flush all remaining buffered data. Exceptions during this flushing of buffered should be propagated, in order to cause the operation to be recognized asa failed, because the last data items are not processed properly.

Specified by:

close in interface StreamOperator<IN extends Tuple>

Overrides:

close in class GenericWriteAheadSink<IN extends Tuple>

Throws:

Exception - An exception in this method causes the operator to fail.

sendValues

protected boolean sendValues(Iterable<IN> values,
                             long checkpointId,
                             long timestamp)
                      throws Exception

Description copied from class: GenericWriteAheadSink

Write the given element into the backend.

Specified by:

sendValues in class GenericWriteAheadSink<IN extends Tuple>

Parameters:

values - The values to be written

checkpointId - The checkpoint ID of the checkpoint to be written

timestamp - The wall-clock timestamp of the checkpoint

Returns:

true, if the sending was successful, false otherwise

Throws:

Exception