public class BatchPartitionCommitterSink extends RichSinkFunction<CompactMessages.CompactOutput>
SinkFunction.Context
Constructor and Description |
---|
BatchPartitionCommitterSink(FileSystemFactory fsFactory,
TableMetaStoreFactory msFactory,
boolean overwrite,
boolean isToLocal,
Path tmpPath,
String[] partitionColumns,
LinkedHashMap<String,String> staticPartitions,
ObjectIdentifier identifier,
PartitionCommitPolicyFactory partitionCommitPolicyFactory) |
Modifier and Type | Method and Description |
---|---|
void |
close()
Tear-down method for the user code.
|
void |
finish()
This method is called at the end of data processing.
|
void |
invoke(CompactMessages.CompactOutput compactOutput,
SinkFunction.Context context)
Writes the given value to the sink.
|
void |
open(OpenContext openContext)
Initialization method for the function.
|
getIterationRuntimeContext, getRuntimeContext, open, setRuntimeContext
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
invoke, writeWatermark
public BatchPartitionCommitterSink(FileSystemFactory fsFactory, TableMetaStoreFactory msFactory, boolean overwrite, boolean isToLocal, Path tmpPath, String[] partitionColumns, LinkedHashMap<String,String> staticPartitions, ObjectIdentifier identifier, PartitionCommitPolicyFactory partitionCommitPolicyFactory)
public void open(OpenContext openContext) throws Exception
RichFunction
The openContext object passed to the function can be used for configuration and initialization. The openContext contains some necessary information that were configured on the function in the program composition.
public class MyFilter extends RichFilterFunction<String> {
private String searchString;
public void open(OpenContext openContext) {
// initialize the value of searchString
}
public boolean filter(String value) {
return value.equals(searchString);
}
}
By default, this method does nothing.
1. If you implement open(OpenContext openContext)
, the open(OpenContext
openContext)
will be invoked and the open(Configuration parameters)
won't be
invoked. 2. If you don't implement open(OpenContext openContext)
, the open(Configuration parameters)
will be invoked in the default implementation of the open(OpenContext openContext)
.
openContext
- The context containing information about the context in which the function
is opened.Exception
- Implementations may forward exceptions, which are caught by the runtime.
When the runtime catches an exception, it aborts the task and lets the fail-over logic
decide whether to retry the task execution.public void invoke(CompactMessages.CompactOutput compactOutput, SinkFunction.Context context) throws Exception
SinkFunction
You have to override this method when implementing a SinkFunction
, this is a
default
method for backward compatibility with the old-style method only.
compactOutput
- The input record.context
- Additional context about the input record.Exception
- This method may throw exceptions. Throwing an exception will cause the
operation to fail and may trigger recovery.public void finish() throws Exception
SinkFunction
The method is expected to flush all remaining buffered data. Exceptions will cause the pipeline to be recognized as failed, because the last data items are not processed properly. You may use this method to flush remaining buffered elements in the state into transactions which you can commit in the last checkpoint.
NOTE:This method does not need to close any resources. You should release external
resources in the AbstractRichFunction.close()
method.
Exception
- This method may throw exceptions. Throwing an exception will cause the
operation to fail and may trigger recovery.public void close() throws Exception
RichFunction
This method can be used for clean up work.
close
in interface RichFunction
close
in class AbstractRichFunction
Exception
- Implementations may forward exceptions, which are caught by the runtime.
When the runtime catches an exception, it aborts the task and lets the fail-over logic
decide whether to retry the task execution.Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.