This documentation is for an unreleased version of Apache Flink Stateful Functions. We recommend you use the latest stable version.

Embedded Module Configuration #

Embedded modules allow users to load code into the Stateful Functions runtime that is executed directly within the cluster. This is usually to allow plugging in custom ingress and egress implementations. Additionally, an embedded module may include embedded functions that run within the cluster.

Embedded modules should be used with care, they cannot be deployed or scaled without downtime and can effect the performance and stability of the entire cluster.

If your application is comprised mostly of embedded elements, the community encourages the use of Stateful Functions DataStream Interop.

To get started, add the embedded Java SDK as a dependency to your application.

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>statefun-sdk-embedded</artifactId>
    <version>3.4-SNAPSHOT</version>
</dependency>

Defining an Embedded Module #

This module type only supports JVM-based languages and is defined by implementing the StatefulFunctionModule interface. Embedded modules offer a single configuration method where stateful functions bind to the system based on their function type. Runtime configurations are available through the globalConfiguration, which is the union of all configurations in the applications flink-conf.yaml under the prefix statefun.module.global-config, and any command line arguments passed in the form –key value.

package org.apache.flink.statefun.docs;

import java.util.Map;
import org.apche.flink.statefun.sdk.spi.StatefulFunctionModule;

public class EmbeddedModule implements StatefulFunctionModule {
    public void configure(Map<String, String> globalConfiguration, Binder binder) {
        // Embedded functions, ingresses, routers, and egresses
        // can be bound to the Binder
    }
}

Embedded modules leverage Java’s Service Provider Interfaces (SPI) for discovery. This means that every JAR should contain a file org.apache.flink.statefun.sdk.spi.StatefulFunctionModule in the META_INF/services resource directory that lists all available modules that it provides.

org.apache.flink.statefun.docs.EmbeddedModule

Deployment #

Embedded modules should be packaged as a fat-jar, containing all required dependencies and added to the StateFun runtime image.

FROM apache/flink-statefun:3.4-SNAPSHOT

RUN mkdir -p /opt/statefun/modules/my-embedded
COPY embedded.jar /opt/statefun/modules/my-embedded/embedded.jar