Reference

Custom Resource Defintion Reference #

This serves as a full reference for FlinkDeployment and FlinkSessionJob custom resource definitions, including all possible configuration parameters.

FlinkDeployment #

Class: org.apache.flink.kubernetes.operator.crd.FlinkDeployment

Description: Custom resource that represents both Application and Session deployments.

Parameter Type Docs
spec org.apache.flink.kubernetes.operator.crd.spec.FlinkDeploymentSpec Spec that describes a Flink application or session cluster deployment.
status org.apache.flink.kubernetes.operator.crd.status.FlinkDeploymentStatus Last observed status of the Flink deployment.

FlinkSessionJob #

Class: org.apache.flink.kubernetes.operator.crd.FlinkSessionJob

Description: Custom resource that represents a session job deployment.

Parameter Type Docs
spec org.apache.flink.kubernetes.operator.crd.spec.FlinkSessionJobSpec Spec that describes a session job deployment.
status org.apache.flink.kubernetes.operator.crd.status.FlinkSessionJobStatus Last observed status of the session job deployment.

Spec #

FlinkDeploymentSpec #

Class: org.apache.flink.kubernetes.operator.api.spec.FlinkDeploymentSpec

Description: Spec that describes a Flink application or session cluster deployment.

Parameter Type Docs
job org.apache.flink.kubernetes.operator.api.spec.JobSpec Job specification for application deployments/session job. Null for session clusters.
restartNonce java.lang.Long Nonce used to manually trigger restart for the cluster/session job. In order to trigger restart, change the number to a different non-null value.
flinkConfiguration java.util.Map<java.lang.String,java.lang.String> Flink configuration overrides for the Flink deployment or Flink session job.
image java.lang.String Flink docker image used to start the Job and TaskManager pods.
imagePullPolicy java.lang.String Image pull policy of the Flink docker image.
serviceAccount java.lang.String Kubernetes service used by the Flink deployment.
flinkVersion org.apache.flink.kubernetes.operator.api.spec.FlinkVersion Flink image version.
ingress org.apache.flink.kubernetes.operator.api.spec.IngressSpec Ingress specs.
podTemplate io.fabric8.kubernetes.api.model.PodTemplateSpec Base pod template for job and task manager pods. Can be overridden by the jobManager and taskManager pod templates.
jobManager org.apache.flink.kubernetes.operator.api.spec.JobManagerSpec JobManager specs.
taskManager org.apache.flink.kubernetes.operator.api.spec.TaskManagerSpec TaskManager specs.
logConfiguration java.util.Map<java.lang.String,java.lang.String> Log configuration overrides for the Flink deployment. Format logConfigFileName -> configContent.
mode org.apache.flink.kubernetes.operator.api.spec.KubernetesDeploymentMode Deployment mode of the Flink cluster, native or standalone.

FlinkSessionJobSpec #

Class: org.apache.flink.kubernetes.operator.api.spec.FlinkSessionJobSpec

Description: Spec that describes a Flink session job.

Parameter Type Docs
job org.apache.flink.kubernetes.operator.api.spec.JobSpec Job specification for application deployments/session job. Null for session clusters.
restartNonce java.lang.Long Nonce used to manually trigger restart for the cluster/session job. In order to trigger restart, change the number to a different non-null value.
flinkConfiguration java.util.Map<java.lang.String,java.lang.String> Flink configuration overrides for the Flink deployment or Flink session job.
deploymentName java.lang.String The name of the target session cluster deployment.

FlinkVersion #

Class: org.apache.flink.kubernetes.operator.api.spec.FlinkVersion

Description: Enumeration for supported Flink versions.

Value Docs
v1_13 No longer supported since 1.7 operator release.
v1_14 No longer supported since 1.7 operator release.
v1_15
v1_16
v1_17
v1_18
v1_19

IngressSpec #

Class: org.apache.flink.kubernetes.operator.api.spec.IngressSpec

Description: Ingress spec.

Parameter Type Docs
template java.lang.String Ingress template for the JobManager service.
className java.lang.String Ingress className for the Flink deployment.
annotations java.util.Map<java.lang.String,java.lang.String> Ingress annotations.
labels java.util.Map<java.lang.String,java.lang.String> Ingress labels.
tls java.util.List<io.fabric8.kubernetes.api.model.networking.v1.IngressTLS> Ingress tls.

JobManagerSpec #

Class: org.apache.flink.kubernetes.operator.api.spec.JobManagerSpec

Description: JobManager spec.

Parameter Type Docs
resource org.apache.flink.kubernetes.operator.api.spec.Resource Resource specification for the JobManager pods.
replicas int Number of JobManager replicas. Must be 1 for non-HA deployments.
podTemplate io.fabric8.kubernetes.api.model.PodTemplateSpec JobManager pod template. It will be merged with FlinkDeploymentSpec.podTemplate.

JobSpec #

Class: org.apache.flink.kubernetes.operator.api.spec.JobSpec

Description: Flink job spec.

Parameter Type Docs
jarURI java.lang.String Optional URI of the job jar within the Flink docker container. For example: local:///opt/flink/examples/streaming/StateMachineExample.jar. If not specified the job jar should be available in the system classpath.
parallelism int Parallelism of the Flink job.
entryClass java.lang.String Fully qualified main class name of the Flink job.
args java.lang.String[] Arguments for the Flink job main class.
state org.apache.flink.kubernetes.operator.api.spec.JobState Desired state for the job.
savepointTriggerNonce java.lang.Long Nonce used to manually trigger savepoint for the running job. In order to trigger a savepoint, change the number to a different non-null value.
initialSavepointPath java.lang.String Savepoint path used by the job the first time it is deployed or during savepoint redeployments (triggered by changing the savepointRedeployNonce).
checkpointTriggerNonce java.lang.Long Nonce used to manually trigger checkpoint for the running job. In order to trigger a checkpoint, change the number to a different non-null value.
upgradeMode org.apache.flink.kubernetes.operator.api.spec.UpgradeMode Upgrade mode of the Flink job.
allowNonRestoredState java.lang.Boolean Allow checkpoint state that cannot be mapped to any job vertex in tasks.
savepointRedeployNonce java.lang.Long Nonce used to trigger a full redeployment of the job from the savepoint path specified in initialSavepointPath. In order to trigger redeployment, change the number to a different non-null value. Rollback is not possible after redeployment.

JobState #

Class: org.apache.flink.kubernetes.operator.api.spec.JobState

Description: Enum describing the desired job state.

Value Docs
running Job is expected to be processing data.
suspended Processing is suspended with the intention of continuing later.

KubernetesDeploymentMode #

Class: org.apache.flink.kubernetes.operator.api.spec.KubernetesDeploymentMode

Description: Enum to control Flink deployment mode on Kubernetes.

Value Docs
native Deploys Flink using Flinks native Kubernetes support. Only supported for newer versions of Flink
standalone Deploys Flink on-top of kubernetes in standalone mode.

Resource #

Class: org.apache.flink.kubernetes.operator.api.spec.Resource

Description: Resource spec.

Parameter Type Docs
cpu java.lang.Double Amount of CPU allocated to the pod.
memory java.lang.String Amount of memory allocated to the pod. Example: 1024m, 1g
ephemeralStorage java.lang.String Amount of ephemeral storage allocated to the pod. Example: 1024m, 2G

TaskManagerSpec #

Class: org.apache.flink.kubernetes.operator.api.spec.TaskManagerSpec

Description: TaskManager spec.

Parameter Type Docs
resource org.apache.flink.kubernetes.operator.api.spec.Resource Resource specification for the TaskManager pods.
replicas java.lang.Integer Number of TaskManager replicas. If defined, takes precedence over parallelism
podTemplate io.fabric8.kubernetes.api.model.PodTemplateSpec TaskManager pod template. It will be merged with FlinkDeploymentSpec.podTemplate.

UpgradeMode #

Class: org.apache.flink.kubernetes.operator.api.spec.UpgradeMode

Description: Enum to control Flink job upgrade behavior.

Value Docs
savepoint Job is upgraded by first taking a savepoint of the running job, shutting it down and restoring from the savepoint.
last-state Job is upgraded using any latest checkpoint or savepoint available.
stateless Job is upgraded with empty state.

Status #

Checkpoint #

Class: org.apache.flink.kubernetes.operator.api.status.Checkpoint

Description: Represents information about a finished checkpoint.

Parameter Type Docs
timeStamp long Millisecond timestamp at the start of the checkpoint operation.
triggerType org.apache.flink.kubernetes.operator.api.status.SnapshotTriggerType Checkpoint trigger mechanism.
formatType org.apache.flink.kubernetes.operator.api.status.CheckpointType Checkpoint format.
triggerNonce java.lang.Long Nonce value used when the checkpoint was triggered manually {@link SnapshotTriggerType#MANUAL}, null for other types of checkpoint.

CheckpointInfo #

Class: org.apache.flink.kubernetes.operator.api.status.CheckpointInfo

Description: Stores checkpoint-related information.

Parameter Type Docs
lastCheckpoint org.apache.flink.kubernetes.operator.api.status.Checkpoint Last completed checkpoint by the operator.
triggerId java.lang.String Trigger id of a pending checkpoint operation.
triggerTimestamp java.lang.Long Trigger timestamp of a pending checkpoint operation.
triggerType org.apache.flink.kubernetes.operator.api.status.SnapshotTriggerType Checkpoint trigger mechanism.
formatType org.apache.flink.kubernetes.operator.api.status.CheckpointType Checkpoint format.
lastPeriodicCheckpointTimestamp long Trigger timestamp of last periodic checkpoint operation.

CheckpointType #

Class: org.apache.flink.kubernetes.operator.api.status.CheckpointType

Description: Checkpoint format type.

Value Docs
FULL
INCREMENTAL
UNKNOWN Checkpoint format unknown, if the checkpoint was not triggered by the operator.
description org.apache.flink.configuration.description.InlineElement

FlinkDeploymentReconciliationStatus #

Class: org.apache.flink.kubernetes.operator.api.status.FlinkDeploymentReconciliationStatus

Description: Status of the last reconcile step for the flink deployment.

Parameter Type Docs
reconciliationTimestamp long Epoch timestamp of the last successful reconcile operation.
lastReconciledSpec java.lang.String Last reconciled deployment spec. Used to decide whether further reconciliation steps are necessary.
lastStableSpec java.lang.String Last stable deployment spec according to the specified stability condition. If a rollback strategy is defined this will be the target to roll back to.
state org.apache.flink.kubernetes.operator.api.status.ReconciliationState Deployment state of the last reconciled spec.

FlinkDeploymentStatus #

Class: org.apache.flink.kubernetes.operator.api.status.FlinkDeploymentStatus

Description: Last observed status of the Flink deployment.

Parameter Type Docs
jobStatus org.apache.flink.kubernetes.operator.api.status.JobStatus Last observed status of the Flink job on Application/Session cluster.
error java.lang.String Error information about the FlinkDeployment/FlinkSessionJob.
observedGeneration java.lang.Long Last observed generation of the FlinkDeployment/FlinkSessionJob.
lifecycleState org.apache.flink.kubernetes.operator.api.lifecycle.ResourceLifecycleState Lifecycle state of the Flink resource (including being rolled back, failed etc.).
clusterInfo java.util.Map<java.lang.String,java.lang.String> Information from running clusters.
jobManagerDeploymentStatus org.apache.flink.kubernetes.operator.api.status.JobManagerDeploymentStatus Last observed status of the JobManager deployment.
reconciliationStatus org.apache.flink.kubernetes.operator.api.status.FlinkDeploymentReconciliationStatus Status of the last reconcile operation.
taskManager org.apache.flink.kubernetes.operator.api.status.TaskManagerInfo Information about the TaskManagers for the scale subresource.

FlinkSessionJobReconciliationStatus #

Class: org.apache.flink.kubernetes.operator.api.status.FlinkSessionJobReconciliationStatus

Description: Status of the last reconcile step for the flink sessionjob.

Parameter Type Docs
reconciliationTimestamp long Epoch timestamp of the last successful reconcile operation.
lastReconciledSpec java.lang.String Last reconciled deployment spec. Used to decide whether further reconciliation steps are necessary.
lastStableSpec java.lang.String Last stable deployment spec according to the specified stability condition. If a rollback strategy is defined this will be the target to roll back to.
state org.apache.flink.kubernetes.operator.api.status.ReconciliationState Deployment state of the last reconciled spec.

FlinkSessionJobStatus #

Class: org.apache.flink.kubernetes.operator.api.status.FlinkSessionJobStatus

Description: Last observed status of the Flink Session job.

Parameter Type Docs
jobStatus org.apache.flink.kubernetes.operator.api.status.JobStatus Last observed status of the Flink job on Application/Session cluster.
error java.lang.String Error information about the FlinkDeployment/FlinkSessionJob.
observedGeneration java.lang.Long Last observed generation of the FlinkDeployment/FlinkSessionJob.
lifecycleState org.apache.flink.kubernetes.operator.api.lifecycle.ResourceLifecycleState Lifecycle state of the Flink resource (including being rolled back, failed etc.).
reconciliationStatus org.apache.flink.kubernetes.operator.api.status.FlinkSessionJobReconciliationStatus Status of the last reconcile operation.

JobManagerDeploymentStatus #

Class: org.apache.flink.kubernetes.operator.api.status.JobManagerDeploymentStatus

Description: Status of the Flink JobManager Kubernetes deployment.

Value Docs
READY JobManager is running and ready to receive REST API calls.
DEPLOYED_NOT_READY JobManager is running but not ready yet to receive REST API calls.
DEPLOYING JobManager process is starting up.
MISSING JobManager deployment not found, probably not started or killed by user.
ERROR Deployment in terminal error, requires spec change for reconciliation to continue.

JobStatus #

Class: org.apache.flink.kubernetes.operator.api.status.JobStatus

Description: Last observed status of the Flink job within an application deployment.

Parameter Type Docs
jobName java.lang.String Name of the job.
jobId java.lang.String Flink JobId of the Job.
state java.lang.String Last observed state of the job.
startTime java.lang.String Start time of the job.
updateTime java.lang.String Update time of the job.
savepointInfo org.apache.flink.kubernetes.operator.api.status.SavepointInfo Information about pending and last savepoint for the job.
checkpointInfo org.apache.flink.kubernetes.operator.api.status.CheckpointInfo Information about pending and last checkpoint for the job.

ReconciliationState #

Class: org.apache.flink.kubernetes.operator.api.status.ReconciliationState

Description: Current state of the reconciliation.

Value Docs
DEPLOYED The lastReconciledSpec is currently deployed.
UPGRADING The spec is being upgraded.
ROLLING_BACK In the process of rolling back to the lastStableSpec.
ROLLED_BACK Rolled back to the lastStableSpec.

Savepoint #

Class: org.apache.flink.kubernetes.operator.api.status.Savepoint

Description: Represents information about a finished savepoint.

Parameter Type Docs
timeStamp long Millisecond timestamp at the start of the savepoint operation.
location java.lang.String External pointer of the savepoint can be used to recover jobs.
triggerType org.apache.flink.kubernetes.operator.api.status.SnapshotTriggerType Savepoint trigger mechanism.
formatType org.apache.flink.kubernetes.operator.api.status.SavepointFormatType Savepoint format.
triggerNonce java.lang.Long Nonce value used when the savepoint was triggered manually {@link SnapshotTriggerType#MANUAL}, null for other types of savepoints.

SavepointFormatType #

Class: org.apache.flink.kubernetes.operator.api.status.SavepointFormatType

Description: Savepoint format type.

Value Docs
CANONICAL A canonical, common for all state backends format.
NATIVE A format specific for the chosen state backend.
UNKNOWN Savepoint format unknown, if the savepoint was not triggered by the operator.

SavepointInfo #

Class: org.apache.flink.kubernetes.operator.api.status.SavepointInfo

Description: Stores savepoint related information.

Parameter Type Docs
lastSavepoint org.apache.flink.kubernetes.operator.api.status.Savepoint Last completed savepoint by the operator.
triggerId java.lang.String Trigger id of a pending savepoint operation.
triggerTimestamp java.lang.Long Trigger timestamp of a pending savepoint operation.
triggerType org.apache.flink.kubernetes.operator.api.status.SnapshotTriggerType Savepoint trigger mechanism.
formatType org.apache.flink.kubernetes.operator.api.status.SavepointFormatType Savepoint format.
savepointHistory java.util.List<org.apache.flink.kubernetes.operator.api.status.Savepoint> List of recent savepoints.
lastPeriodicSavepointTimestamp long Trigger timestamp of last periodic savepoint operation.

SnapshotTriggerType #

Class: org.apache.flink.kubernetes.operator.api.status.SnapshotTriggerType

Description: Snapshot trigger mechanism.

Value Docs
MANUAL Snapshot manually triggered by changing a triggerNonce.
PERIODIC Snapshot periodically triggered by the operator.
UPGRADE Snapshot triggered during stateful upgrade.
UNKNOWN Snapshot trigger mechanism unknown, such as savepoint retrieved directly from Flink job.

TaskManagerInfo #

Class: org.apache.flink.kubernetes.operator.api.status.TaskManagerInfo

Description: Last observed status of the Flink job within an application deployment.

Parameter Type Docs
labelSelector java.lang.String TaskManager label selector.
replicas int Number of TaskManager replicas if defined in the spec.