IN
- Type of the elements emitted by this sinkpublic abstract class CassandraSinkBase<IN,V> extends RichSinkFunction<IN> implements CheckpointedFunction
CassandraPojoSink
and CassandraTupleSink
.
In case of experiencing the following error: Error while sending value.
com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query
at consistency LOCAL_ONE (1 replica were required but only 0 acknowledged the write)
,
it is recommended to increase the Cassandra write timeout to adapt to your workload in your Cassandra cluster so that such timeout errors do not happen. For that you need to raise write_request_timeout_in_ms conf parameter in your cassandra.yml. Indeed, This exception means that Cassandra coordinator node (internal Cassandra) waited too long for an internal replication (replication to another node and did not ack the write. It is not recommended to lower the replication factor in your Cassandra cluster because it is mandatory that you do not loose data in case of a Cassandra cluster failure. Waiting for a single replica for write acknowledge is the minimum level for this guarantee in Cassandra.}
SinkFunction.Context
Modifier and Type | Field and Description |
---|---|
protected com.datastax.driver.core.Cluster |
cluster |
protected org.slf4j.Logger |
log |
protected com.datastax.driver.core.Session |
session |
Modifier and Type | Method and Description |
---|---|
void |
close()
Tear-down method for the user code.
|
protected com.datastax.driver.core.Session |
createSession() |
void |
initializeState(FunctionInitializationContext context)
This method is called when the parallel function instance is created during distributed
execution.
|
void |
invoke(IN value) |
void |
open(Configuration configuration)
Initialization method for the function.
|
abstract com.google.common.util.concurrent.ListenableFuture<V> |
send(IN value) |
void |
snapshotState(FunctionSnapshotContext ctx)
This method is called when a snapshot for a checkpoint is requested.
|
getIterationRuntimeContext, getRuntimeContext, setRuntimeContext
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
finish, invoke, writeWatermark
protected final org.slf4j.Logger log
protected transient com.datastax.driver.core.Cluster cluster
protected transient com.datastax.driver.core.Session session
public void open(Configuration configuration)
RichFunction
The configuration object passed to the function can be used for configuration and initialization. The configuration contains all parameters that were configured on the function in the program composition.
public class MyFilter extends RichFilterFunction<String> {
private String searchString;
public void open(Configuration parameters) {
this.searchString = parameters.getString("foo");
}
public boolean filter(String value) {
return value.equals(searchString);
}
}
By default, this method does nothing.
open
in interface RichFunction
open
in class AbstractRichFunction
configuration
- The configuration containing the parameters attached to the contract.Configuration
public void close() throws Exception
RichFunction
This method can be used for clean up work.
close
in interface RichFunction
close
in class AbstractRichFunction
Exception
- Implementations may forward exceptions, which are caught by the runtime.
When the runtime catches an exception, it aborts the task and lets the fail-over logic
decide whether to retry the task execution.public void initializeState(FunctionInitializationContext context) throws Exception
CheckpointedFunction
initializeState
in interface CheckpointedFunction
context
- the context for initializing the operatorException
- Thrown, if state could not be created ot restored.public void snapshotState(FunctionSnapshotContext ctx) throws Exception
CheckpointedFunction
FunctionInitializationContext
when the Function was initialized, or offered now by FunctionSnapshotContext
itself.snapshotState
in interface CheckpointedFunction
ctx
- the context for drawing a snapshot of the operatorException
- Thrown, if state could not be created ot restored.public void invoke(IN value) throws Exception
invoke
in interface SinkFunction<IN>
Exception
protected com.datastax.driver.core.Session createSession()
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.