T - The type of the output row used during reflective extraction.@PublicEvolving public abstract class AsyncTableFunction<T> extends UserDefinedFunction
This kind of function is similar to TableFunction but is executed asynchronously.
The behavior of a AsyncTableFunction can be defined by implementing a custom
evaluation method. An evaluation method must be declared publicly, not static, and named
eval. Evaluation methods can also be overloaded by implementing multiple methods named
eval.
By default, input and output data types are automatically extracted using reflection. This
includes the generic argument T of the class for determining an output data type. Input
arguments are derived from one or more eval() methods. If the reflective information is
not sufficient, it can be supported and enriched with DataTypeHint and FunctionHint annotations. See TableFunction for more examples how to annotate an
implementation class.
Note: Currently, asynchronous table functions are only supported as the runtime implementation
of LookupTableSources for performing temporal joins. By default, input and output DataTypes of AsyncTableFunction are derived similar to other UserDefinedFunctions using the logic above. However, for convenience, in a LookupTableSource the output type can simply be a Row or RowData in which case
the input and output types are derived from the table's schema with default conversion.
The first parameter of the evaluation method must be a CompletableFuture. Other
parameters specify user-defined input parameters like the "eval" method of TableFunction.
The generic type of CompletableFuture must be Collection to collect
multiple possible result values.
For each call to eval(), an async IO operation can be triggered, and once the
operation has been done, the result can be collected by calling CompletableFuture.complete(T). For each async operation, its context is stored in the operator
immediately after invoking eval(), avoiding blocking for each stream input as long
as the internal buffer is not full.
CompletableFuture can be passed into callbacks or futures to collect the result data.
An error can also be propagated to the async IO operator by calling CompletableFuture.completeExceptionally(Throwable).
For storing a user-defined function in a catalog, the class must have a default constructor and must be instantiable during runtime. Anonymous functions in Table API can only be persisted if the function is not stateful (i.e. containing only transient and static fields).
The following example shows how to perform an asynchronous request to Apache HBase:
public class HBaseAsyncTableFunction extends AsyncTableFunction<Row> {
// implement an "eval" method that takes a CompletableFuture as the first parameter
// and ends with as many parameters as you want
public void eval(CompletableFuture<Collection<Row>> result, String rowkey) {
Get get = new Get(Bytes.toBytes(rowkey));
ListenableFuture<Result> future = hbase.asyncGet(get);
Futures.addCallback(future, new FutureCallback<Result>() {
public void onSuccess(Result hbaseResult) {
List<Row> ret = process(hbaseResult);
result.complete(ret);
}
public void onFailure(Throwable thrown) {
result.completeExceptionally(thrown);
}
});
}
// you can overload the eval method here ...
}
| Constructor and Description |
|---|
AsyncTableFunction() |
| Modifier and Type | Method and Description |
|---|---|
FunctionKind |
getKind()
Returns the kind of function this definition describes.
|
TypeInference |
getTypeInference(DataTypeFactory typeFactory)
Returns the logic for performing type inference of a call to this function definition.
|
close, functionIdentifier, open, toStringclone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, waitgetRequirements, isDeterministicpublic final FunctionKind getKind()
FunctionDefinitionpublic TypeInference getTypeInference(DataTypeFactory typeFactory)
UserDefinedFunctionThe type inference process is responsible for inferring unknown types of input arguments, validating input arguments, and producing result types. The type inference process happens independent of a function body. The output of the type inference is used to search for a corresponding runtime implementation.
Instances of type inference can be created by using TypeInference.newBuilder().
See BuiltInFunctionDefinitions for concrete usage examples.
The type inference for user-defined functions is automatically extracted using reflection.
It does this by analyzing implementation methods such as eval() or accumulate() and
the generic parameters of a function class if present. If the reflective information is not
sufficient, it can be supported and enriched with DataTypeHint and FunctionHint annotations.
Note: Overriding this method is only recommended for advanced users. If a custom type inference is specified, it is the responsibility of the implementer to make sure that the output of the type inference process matches with the implementation method:
The implementation method must comply with each DataType.getConversionClass()
returned by the type inference. For example, if DataTypes.TIMESTAMP(3).bridgedTo(java.sql.Timestamp.class) is an expected argument type, the
method must accept a call eval(java.sql.Timestamp).
Regular Java calling semantics (including type widening and autoboxing) are applied when
calling an implementation method which means that the signature can be eval(java.lang.Object).
The runtime will take care of converting the data to the data format specified by the
DataType.getConversionClass() coming from the type inference logic.
getTypeInference in interface FunctionDefinitiongetTypeInference in class UserDefinedFunctionCopyright © 2014–2024 The Apache Software Foundation. All rights reserved.