org.apache.flink.ml.api.core

Class Pipeline

java.lang.Object

org.apache.flink.ml.api.core.Pipeline

All Implemented Interfaces:

Estimator<Pipeline,Pipeline>, Model<Pipeline>, Transformer<Pipeline>

@PublicEvolving
public final class Pipeline
extends Object
implements Estimator<Pipeline,Pipeline>, Transformer<Pipeline>, Model<Pipeline>

A pipeline is a linear workflow which chains Estimators and Transformers to execute an algorithm.

A pipeline itself can either act as an Estimator or a Transformer, depending on the stages it includes. More specifically:

• If a Pipeline has an Estimator, one needs to call fit(TableEnvironment, Table) before use the pipeline as a Transformer . In this case the Pipeline is an Estimator and can produce a Pipeline as a Model.
• If a Pipeline has no Estimator, it is a Transformer and can be applied to a Table directly. In this case, fit(TableEnvironment, Table) will simply return the pipeline itself.

In addition, a pipeline can also be used as a PipelineStage in another pipeline, just like an ordinary Estimator or Transformer as describe above.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
Pipeline()
Pipeline(List<org.apache.flink.ml.api.core.PipelineStage> stages)
Pipeline(String pipelineJson)

Method Summary

Modifier and Type	Method and Description
Pipeline	appendStage(org.apache.flink.ml.api.core.PipelineStage stage) Appends a PipelineStage to the tail of this pipeline.
Pipeline	fit(TableEnvironment tEnv, Table input) Train the pipeline to fit on the records in the given Table.
Params	getParams() Returns the all the parameters.
List<org.apache.flink.ml.api.core.PipelineStage>	getStages() Returns a list of all stages in this pipeline in order, the list is immutable.
void	loadJson(String json)
boolean	needFit() Check whether the pipeline acts as an Estimator or not.
String	toJson()
Table	transform(TableEnvironment tEnv, Table input) Generate a result table by applying all the stages in this pipeline to the input table in order.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.ml.api.misc.param.WithParams

get, set

Constructor Detail

Pipeline

public Pipeline()

Pipeline

public Pipeline(String pipelineJson)

Pipeline

public Pipeline(List<org.apache.flink.ml.api.core.PipelineStage> stages)

Method Detail

appendStage

public Pipeline appendStage(org.apache.flink.ml.api.core.PipelineStage stage)

Appends a PipelineStage to the tail of this pipeline. Pipeline is editable only via this method. The PipelineStage must be Estimator, Transformer, Model or Pipeline.

Parameters:

stage - the stage to be appended

getStages

public List<org.apache.flink.ml.api.core.PipelineStage> getStages()

Returns a list of all stages in this pipeline in order, the list is immutable.

Returns:

an immutable list of all stages in this pipeline in order.

needFit

public boolean needFit()

Check whether the pipeline acts as an Estimator or not. When the return value is true, that means this pipeline contains an Estimator and thus users must invoke fit(TableEnvironment, Table) before they can use this pipeline as a Transformer. Otherwise, the pipeline can be used as a Transformer directly.

Returns:

true if this pipeline has an Estimator, false otherwise

getParams

public Params getParams()

Description copied from interface: WithParams

Returns the all the parameters.

Returns:

all the parameters.

fit

public Pipeline fit(TableEnvironment tEnv,
                    Table input)

Train the pipeline to fit on the records in the given Table.

This method go through all the PipelineStages in order and does the following on each stage until the last Estimator(inclusive).

• If a stage is an Estimator, invoke Estimator.fit(TableEnvironment, Table) with the input table to generate a Model, transform the the input table with the generated Model to get a result table, then pass the result table to the next stage as input.
• If a stage is a Transformer, invoke Transformer.transform(TableEnvironment, Table) on the input table to get a result table, and pass the result table to the next stage as input.

After all the Estimators are trained to fit their input tables, a new pipeline will be created with the same stages in this pipeline, except that all the Estimators in the new pipeline are replaced with their corresponding Models generated in the above process.

If there is no Estimator in the pipeline, the method returns a copy of this pipeline.

Specified by:

fit in interface Estimator<Pipeline,Pipeline>

Parameters:

tEnv - the table environment to which the input table is bound.

input - the table with records to train the Pipeline.

Returns:

a pipeline with same stages as this Pipeline except all Estimators replaced with their corresponding Models.

transform

public Table transform(TableEnvironment tEnv,
                       Table input)

Generate a result table by applying all the stages in this pipeline to the input table in order.

Specified by:

transform in interface Transformer<Pipeline>

Parameters:

tEnv - the table environment to which the input table is bound.

input - the table to be transformed

Returns:

a result table with all the stages applied to the input tables in order.

toJson

public String toJson()

loadJson

public void loadJson(String json)