Package org.apache.flink.table.functions

Class AsyncTableFunction<T>

  • Type Parameters:
    T - The type of the output row used during reflective extraction.
    All Implemented Interfaces:
    Serializable, FunctionDefinition
    Direct Known Subclasses:
    AsyncLookupFunction
    @PublicEvolving
public abstract class AsyncTableFunction<T>
extends UserDefinedFunction
    Base class for a user-defined asynchronous table function. A user-defined asynchronous table function maps zero, one, or multiple scalar values to zero, one, or multiple rows (or structured types).

    This kind of function is similar to TableFunction but is executed asynchronously.

    The behavior of a AsyncTableFunction can be defined by implementing a custom evaluation method. An evaluation method must be declared publicly, not static, and named eval. Evaluation methods can also be overloaded by implementing multiple methods named eval.

    By default, input and output data types are automatically extracted using reflection. This includes the generic argument T of the class for determining an output data type. Input arguments are derived from one or more eval() methods. If the reflective information is not sufficient, it can be supported and enriched with DataTypeHint and FunctionHint annotations. See TableFunction for more examples how to annotate an implementation class.

    Note: Currently, asynchronous table functions are only supported as the runtime implementation of LookupTableSources for performing temporal joins. By default, input and output DataTypes of AsyncTableFunction are derived similar to other UserDefinedFunctions using the logic above. However, for convenience, in a LookupTableSource the output type can simply be a Row or RowData in which case the input and output types are derived from the table's schema with default conversion.

    The first parameter of the evaluation method must be a CompletableFuture. Other parameters specify user-defined input parameters like the "eval" method of TableFunction. The generic type of CompletableFuture must be Collection to collect multiple possible result values.

    For each call to eval(), an async IO operation can be triggered, and once the operation has been done, the result can be collected by calling CompletableFuture.complete(T). For each async operation, its context is stored in the operator immediately after invoking eval(), avoiding blocking for each stream input as long as the internal buffer is not full.

    CompletableFuture can be passed into callbacks or futures to collect the result data. An error can also be propagated to the async IO operator by calling CompletableFuture.completeExceptionally(Throwable).

    For storing a user-defined function in a catalog, the class must have a default constructor and must be instantiable during runtime. Anonymous functions in Table API can only be persisted if the function is not stateful (i.e. containing only transient and static fields).

    The following example shows how to perform an asynchronous request to Apache HBase: 

    
 public class HBaseAsyncTableFunction extends AsyncTableFunction<Row> {

   // implement an "eval" method that takes a CompletableFuture as the first parameter
   // and ends with as many parameters as you want
   public void eval(CompletableFuture<Collection<Row>> result, String rowkey) {
     Get get = new Get(Bytes.toBytes(rowkey));
     ListenableFuture<Result> future = hbase.asyncGet(get);
     Futures.addCallback(future, new FutureCallback<Result>() {
       public void onSuccess(Result hbaseResult) {
         List<Row> ret = process(hbaseResult);
         result.complete(ret);
       }
       public void onFailure(Throwable thrown) {
         result.completeExceptionally(thrown);
       }
     });
   }

   // you can overload the eval method here ...
 }
    See Also:
    Serialized Form

    • Constructor Detail

      • AsyncTableFunction

        public AsyncTableFunction()

    • Method Detail

      • getKind

        public final FunctionKind getKind()
        Description copied from interface: FunctionDefinition
        Returns the kind of function this definition describes.

      • getTypeInference

        public TypeInference getTypeInference​(DataTypeFactory typeFactory)
        Description copied from class: UserDefinedFunction
        Returns the logic for performing type inference of a call to this function definition.

        The type inference process is responsible for inferring unknown types of input arguments, validating input arguments, and producing result types. The type inference process happens independent of a function body. The output of the type inference is used to search for a corresponding runtime implementation.

        Instances of type inference can be created by using TypeInference.newBuilder().

        See BuiltInFunctionDefinitions for concrete usage examples.

        The type inference for user-defined functions is automatically extracted using reflection. It does this by analyzing implementation methods such as eval() or accumulate() and the generic parameters of a function class if present. If the reflective information is not sufficient, it can be supported and enriched with DataTypeHint and FunctionHint annotations.

        Note: Overriding this method is only recommended for advanced users. If a custom type inference is specified, it is the responsibility of the implementer to make sure that the output of the type inference process matches with the implementation method:

        The implementation method must comply with each DataType.getConversionClass() returned by the type inference. For example, if DataTypes.TIMESTAMP(3).bridgedTo(java.sql.Timestamp.class) is an expected argument type, the method must accept a call eval(java.sql.Timestamp).

        Regular Java calling semantics (including type widening and autoboxing) are applied when calling an implementation method which means that the signature can be eval(java.lang.Object).

        The runtime will take care of converting the data to the data format specified by the DataType.getConversionClass() coming from the type inference logic.

        Specified by:
        getTypeInference in interface FunctionDefinition
        Specified by:
        getTypeInference in class UserDefinedFunction