org.apache.flink.streaming.api.datastream

Class CachedDataStream<T>

java.lang.Object

org.apache.flink.streaming.api.datastream.DataStream<T>

org.apache.flink.streaming.api.datastream.CachedDataStream<T>

Type Parameters:

T - The type of the elements in this stream.

@PublicEvolving
public class CachedDataStream<T>
extends DataStream<T>

CachedDataStream represents a DataStream whose intermediate result will be cached at the first time when it is computed. And the cached intermediate result can be used in later job that using the same CachedDataStream to avoid re-computing the intermediate result.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.streaming.api.datastream.DataStream

DataStream.Collector<T>

Field Summary

Fields inherited from class org.apache.flink.streaming.api.datastream.DataStream

environment, transformation

Constructor Summary

Constructors

Constructor and Description
CachedDataStream(StreamExecutionEnvironment environment, Transformation<T> transformation) Create a new CachedDataStream in the given execution environment that wrap the given physical transformation to indicates that the transformation should be cached.

Method Summary

Modifier and Type	Method and Description
void	invalidate() Invalidate the cache intermediate result of this DataStream to release the physical resources.

Methods inherited from class org.apache.flink.streaming.api.datastream.DataStream

addSink, assignTimestampsAndWatermarks, assignTimestampsAndWatermarks, assignTimestampsAndWatermarks, broadcast, broadcast, clean, coGroup, collectAsync, collectAsync, connect, connect, countWindowAll, countWindowAll, doTransform, executeAndCollect, executeAndCollect, executeAndCollect, executeAndCollect, filter, flatMap, flatMap, forward, fullWindowPartition, getExecutionConfig, getExecutionEnvironment, getId, getMinResources, getParallelism, getPreferredResources, getTransformation, getType, global, iterate, iterate, join, keyBy, keyBy, keyBy, keyBy, map, map, partitionCustom, partitionCustom, partitionCustom, print, print, printToErr, printToErr, process, process, project, rebalance, rescale, setConnectionType, shuffle, sinkTo, sinkTo, sinkTo, sinkTo, timeWindowAll, timeWindowAll, transform, transform, union, windowAll, writeAsCsv, writeAsCsv, writeAsCsv, writeAsText, writeAsText, writeToSocket, writeUsingOutputFormat

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CachedDataStream

public CachedDataStream(StreamExecutionEnvironment environment,
                        Transformation<T> transformation)

Create a new CachedDataStream in the given execution environment that wrap the given physical transformation to indicates that the transformation should be cached.

Parameters:

environment - The StreamExecutionEnvironment

transformation - The physical transformation whose intermediate result should be cached.

Method Detail

invalidate

public void invalidate()
                throws Exception

Invalidate the cache intermediate result of this DataStream to release the physical resources. Users are not required to invoke this method to release physical resources unless they want to. Cache will be recreated if it is used after invalidated.

Throws:

Exception