org.apache.flink.graph

Class Graph<K,VV,EV>

java.lang.Object

org.apache.flink.graph.Graph<K,VV,EV>

Type Parameters:

K - the key type for edge and vertex identifiers

VV - the value type for vertices

EV - the value type for edges

public class Graph<K,VV,EV>
extends Object

Represents a Graph consisting of edges and vertices.

See Also:

Edge, Vertex

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Graph(DataSet<Vertex<K,VV>> vertices, DataSet<Edge<K,EV>> edges, ExecutionEnvironment context) Creates a graph from two DataSets: vertices and edges.

Method Summary

Modifier and Type	Method and Description
Graph<K,VV,EV>	addEdge(Vertex<K,VV> source, Vertex<K,VV> target, EV edgeValue) Adds the given edge to the graph.
Graph<K,VV,EV>	addEdges(List<Edge<K,EV>> newEdges) Adds the given list edges to the graph.
Graph<K,VV,EV>	addVertex(Vertex<K,VV> vertex) Adds the input vertex to the graph.
Graph<K,VV,EV>	addVertices(List<Vertex<K,VV>> verticesToAdd) Adds the list of vertices, passed as input, to the graph.
Graph<K,VV,EV>	difference(Graph<K,VV,EV> graph) Performs Difference on the vertex and edge sets of the input graphs removes common vertices and edges.
Graph<K,VV,EV>	filterOnEdges(FilterFunction<Edge<K,EV>> edgeFilter) Apply a filtering function to the graph and return a sub-graph that satisfies the predicates only for the edges.
Graph<K,VV,EV>	filterOnVertices(FilterFunction<Vertex<K,VV>> vertexFilter) Apply a filtering function to the graph and return a sub-graph that satisfies the predicates only for the vertices.
static <K,EV> Graph<K,NullValue,EV>	fromCollection(Collection<Edge<K,EV>> edges, ExecutionEnvironment context) Creates a graph from a Collection of edges.
static <K,VV,EV> Graph<K,VV,EV>	fromCollection(Collection<Edge<K,EV>> edges, MapFunction<K,VV> vertexValueInitializer, ExecutionEnvironment context) Creates a graph from a Collection of edges.
static <K,VV,EV> Graph<K,VV,EV>	fromCollection(Collection<Vertex<K,VV>> vertices, Collection<Edge<K,EV>> edges, ExecutionEnvironment context) Creates a graph from a Collection of vertices and a Collection of edges.
static GraphCsvReader	fromCsvReader(String edgesPath, ExecutionEnvironment context) Creates a graph from a CSV file of edges.
static <K,VV> GraphCsvReader	fromCsvReader(String edgesPath, MapFunction<K,VV> vertexValueInitializer, ExecutionEnvironment context) Creates a graph from a CSV file of edges.
static GraphCsvReader	fromCsvReader(String verticesPath, String edgesPath, ExecutionEnvironment context) Creates a Graph from a CSV file of vertices and a CSV file of edges.
static <K,EV> Graph<K,NullValue,EV>	fromDataSet(DataSet<Edge<K,EV>> edges, ExecutionEnvironment context) Creates a graph from a DataSet of edges.
static <K,VV,EV> Graph<K,VV,EV>	fromDataSet(DataSet<Edge<K,EV>> edges, MapFunction<K,VV> vertexValueInitializer, ExecutionEnvironment context) Creates a graph from a DataSet of edges.
static <K,VV,EV> Graph<K,VV,EV>	fromDataSet(DataSet<Vertex<K,VV>> vertices, DataSet<Edge<K,EV>> edges, ExecutionEnvironment context) Creates a graph from a DataSet of vertices and a DataSet of edges.
static <K> Graph<K,NullValue,NullValue>	fromTuple2DataSet(DataSet<Tuple2<K,K>> edges, ExecutionEnvironment context) Creates a graph from a DataSet of Tuple2 objects for edges.
static <K,VV> Graph<K,VV,NullValue>	fromTuple2DataSet(DataSet<Tuple2<K,K>> edges, MapFunction<K,VV> vertexValueInitializer, ExecutionEnvironment context) Creates a graph from a DataSet of Tuple2 objects for edges.
static <K,VV,EV> Graph<K,VV,EV>	fromTupleDataSet(DataSet<Tuple2<K,VV>> vertices, DataSet<Tuple3<K,K,EV>> edges, ExecutionEnvironment context) Creates a graph from a DataSet of Tuple2 objects for vertices and Tuple3 objects for edges.
static <K,EV> Graph<K,NullValue,EV>	fromTupleDataSet(DataSet<Tuple3<K,K,EV>> edges, ExecutionEnvironment context) Creates a graph from a DataSet of Tuple3 objects for edges.
static <K,VV,EV> Graph<K,VV,EV>	fromTupleDataSet(DataSet<Tuple3<K,K,EV>> edges, MapFunction<K,VV> vertexValueInitializer, ExecutionEnvironment context) Creates a graph from a DataSet of Tuple3 objects for edges.
ExecutionEnvironment	getContext()
DataSet<Tuple2<K,LongValue>>	getDegrees() Return the degree of all vertices in the graph.
DataSet<Tuple2<K,K>>	getEdgeIds()
DataSet<Edge<K,EV>>	getEdges()
DataSet<Tuple3<K,K,EV>>	getEdgesAsTuple3()
DataSet<Triplet<K,VV,EV>>	getTriplets() This method allows access to the graph's edge values along with its source and target vertex values.
Graph<K,VV,EV>	getUndirected() This operation adds all inverse-direction edges to the graph.
DataSet<K>	getVertexIds()
DataSet<Vertex<K,VV>>	getVertices()
DataSet<Tuple2<K,VV>>	getVerticesAsTuple2()
<T> DataSet<T>	groupReduceOnEdges(EdgesFunction<K,EV,T> edgesFunction, EdgeDirection direction) Groups by vertex and computes a GroupReduce transformation over the edge values of each vertex.
<T> DataSet<T>	groupReduceOnEdges(EdgesFunction<K,EV,T> edgesFunction, EdgeDirection direction, TypeInformation<T> typeInfo) Groups by vertex and computes a GroupReduce transformation over the edge values of each vertex.
<T> DataSet<T>	groupReduceOnEdges(EdgesFunctionWithVertexValue<K,VV,EV,T> edgesFunction, EdgeDirection direction) Groups by vertex and computes a GroupReduce transformation over the edge values of each vertex.
<T> DataSet<T>	groupReduceOnEdges(EdgesFunctionWithVertexValue<K,VV,EV,T> edgesFunction, EdgeDirection direction, TypeInformation<T> typeInfo) Groups by vertex and computes a GroupReduce transformation over the edge values of each vertex.
<T> DataSet<T>	groupReduceOnNeighbors(NeighborsFunction<K,VV,EV,T> neighborsFunction, EdgeDirection direction) Groups by vertex and computes a GroupReduce transformation over the neighbors (both edges and vertices) of each vertex.
<T> DataSet<T>	groupReduceOnNeighbors(NeighborsFunction<K,VV,EV,T> neighborsFunction, EdgeDirection direction, TypeInformation<T> typeInfo) Groups by vertex and computes a GroupReduce transformation over the neighbors (both edges and vertices) of each vertex.
<T> DataSet<T>	groupReduceOnNeighbors(NeighborsFunctionWithVertexValue<K,VV,EV,T> neighborsFunction, EdgeDirection direction) Groups by vertex and computes a GroupReduce transformation over the neighbors (both edges and vertices) of each vertex.
<T> DataSet<T>	groupReduceOnNeighbors(NeighborsFunctionWithVertexValue<K,VV,EV,T> neighborsFunction, EdgeDirection direction, TypeInformation<T> typeInfo) Groups by vertex and computes a GroupReduce transformation over the neighbors (both edges and vertices) of each vertex.
DataSet<Tuple2<K,LongValue>>	inDegrees() Return the in-degree of all vertices in the graph.
Graph<K,NullValue,EV>	intersect(Graph<K,VV,EV> graph, boolean distinctEdges) Performs intersect on the edge sets of the input graphs.
<T> Graph<K,VV,EV>	joinWithEdges(DataSet<Tuple3<K,K,T>> inputDataSet, EdgeJoinFunction<EV,T> edgeJoinFunction) Joins the edge DataSet with an input DataSet on the composite key of both source and target IDs and applies a user-defined transformation on the values of the matched records.
<T> Graph<K,VV,EV>	joinWithEdgesOnSource(DataSet<Tuple2<K,T>> inputDataSet, EdgeJoinFunction<EV,T> edgeJoinFunction) Joins the edge DataSet with an input Tuple2 DataSet and applies a user-defined transformation on the values of the matched records.
<T> Graph<K,VV,EV>	joinWithEdgesOnTarget(DataSet<Tuple2<K,T>> inputDataSet, EdgeJoinFunction<EV,T> edgeJoinFunction) Joins the edge DataSet with an input Tuple2 DataSet and applies a user-defined transformation on the values of the matched records.
<T> Graph<K,VV,EV>	joinWithVertices(DataSet<Tuple2<K,T>> inputDataSet, VertexJoinFunction<VV,T> vertexJoinFunction) Joins the vertex DataSet of this graph with an input Tuple2 DataSet and applies a user-defined transformation on the values of the matched records.
<NV> Graph<K,VV,NV>	mapEdges(MapFunction<Edge<K,EV>,NV> mapper) Apply a function to the attribute of each edge in the graph.
<NV> Graph<K,VV,NV>	mapEdges(MapFunction<Edge<K,EV>,NV> mapper, TypeInformation<Edge<K,NV>> returnType) Apply a function to the attribute of each edge in the graph.
<NV> Graph<K,NV,EV>	mapVertices(MapFunction<Vertex<K,VV>,NV> mapper) Apply a function to the attribute of each vertex in the graph.
<NV> Graph<K,NV,EV>	mapVertices(MapFunction<Vertex<K,VV>,NV> mapper, TypeInformation<Vertex<K,NV>> returnType) Apply a function to the attribute of each vertex in the graph.
long	numberOfEdges()
long	numberOfVertices()
DataSet<Tuple2<K,LongValue>>	outDegrees() Return the out-degree of all vertices in the graph.
DataSet<Tuple2<K,EV>>	reduceOnEdges(ReduceEdgesFunction<EV> reduceEdgesFunction, EdgeDirection direction) Compute a reduce transformation over the edge values of each vertex.
DataSet<Tuple2<K,VV>>	reduceOnNeighbors(ReduceNeighborsFunction<VV> reduceNeighborsFunction, EdgeDirection direction) Compute a reduce transformation over the neighbors' vertex values of each vertex.
Graph<K,VV,EV>	removeEdge(Edge<K,EV> edge) Removes all edges that match the given edge from the graph.
Graph<K,VV,EV>	removeEdges(List<Edge<K,EV>> edgesToBeRemoved) Removes all the edges that match the edges in the given data set from the graph.
Graph<K,VV,EV>	removeVertex(Vertex<K,VV> vertex) Removes the given vertex and its edges from the graph.
Graph<K,VV,EV>	removeVertices(List<Vertex<K,VV>> verticesToBeRemoved) Removes the given list of vertices and its edges from the graph.
Graph<K,VV,EV>	reverse() Reverse the direction of the edges in the graph.
<T> T	run(GraphAlgorithm<K,VV,EV,T> algorithm)
<T> GraphAnalytic<K,VV,EV,T>	run(GraphAnalytic<K,VV,EV,T> analytic) A GraphAnalytic is similar to a GraphAlgorithm but is terminal and results are retrieved via accumulators.
<M> Graph<K,VV,EV>	runGatherSumApplyIteration(GatherFunction<VV,EV,M> gatherFunction, SumFunction<VV,EV,M> sumFunction, ApplyFunction<K,VV,M> applyFunction, int maximumNumberOfIterations) Runs a Gather-Sum-Apply iteration on the graph.
<M> Graph<K,VV,EV>	runGatherSumApplyIteration(GatherFunction<VV,EV,M> gatherFunction, SumFunction<VV,EV,M> sumFunction, ApplyFunction<K,VV,M> applyFunction, int maximumNumberOfIterations, GSAConfiguration parameters) Runs a Gather-Sum-Apply iteration on the graph with configuration options.
<M> Graph<K,VV,EV>	runScatterGatherIteration(ScatterFunction<K,VV,M,EV> scatterFunction, GatherFunction<K,VV,M> gatherFunction, int maximumNumberOfIterations) Runs a ScatterGather iteration on the graph.
<M> Graph<K,VV,EV>	runScatterGatherIteration(ScatterFunction<K,VV,M,EV> scatterFunction, GatherFunction<K,VV,M> gatherFunction, int maximumNumberOfIterations, ScatterGatherConfiguration parameters) Runs a ScatterGather iteration on the graph with configuration options.
<M> Graph<K,VV,EV>	runVertexCentricIteration(ComputeFunction<K,VV,EV,M> computeFunction, MessageCombiner<K,M> combiner, int maximumNumberOfIterations) Runs a VertexCentricIteration on the graph.
<M> Graph<K,VV,EV>	runVertexCentricIteration(ComputeFunction<K,VV,EV,M> computeFunction, MessageCombiner<K,M> combiner, int maximumNumberOfIterations, VertexCentricConfiguration parameters) Runs a VertexCentricIteration on the graph with configuration options.
Graph<K,VV,EV>	subgraph(FilterFunction<Vertex<K,VV>> vertexFilter, FilterFunction<Edge<K,EV>> edgeFilter) Apply filtering functions to the graph and return a sub-graph that satisfies the predicates for both vertices and edges.
<NEW> Graph<K,VV,NEW>	translateEdgeValues(TranslateFunction<EV,NEW> translator) Translate Edge values using the given MapFunction.
<NEW> Graph<NEW,VV,EV>	translateGraphIds(TranslateFunction<K,NEW> translator) Translate Vertex and Edge IDs using the given MapFunction.
<NEW> Graph<K,NEW,EV>	translateVertexValues(TranslateFunction<VV,NEW> translator) Translate Vertex values using the given MapFunction.
Graph<K,VV,EV>	union(Graph<K,VV,EV> graph) Performs union on the vertices and edges sets of the input graphs removing duplicate vertices but maintaining duplicate edges.
Boolean	validate(GraphValidator<K,VV,EV> validator) Function that checks whether a Graph is a valid Graph, as defined by the given GraphValidator.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Graph

protected Graph(DataSet<Vertex<K,VV>> vertices,
                DataSet<Edge<K,EV>> edges,
                ExecutionEnvironment context)

Creates a graph from two DataSets: vertices and edges.

Parameters:

vertices - a DataSet of vertices.

edges - a DataSet of edges.

context - the flink execution environment.

Method Detail

fromCollection

public static <K,VV,EV> Graph<K,VV,EV> fromCollection(Collection<Vertex<K,VV>> vertices,
                                                      Collection<Edge<K,EV>> edges,
                                                      ExecutionEnvironment context)

Creates a graph from a Collection of vertices and a Collection of edges.

Parameters:

vertices - a Collection of vertices.

edges - a Collection of edges.

context - the flink execution environment.

Returns:

the newly created graph.

fromCollection

public static <K,EV> Graph<K,NullValue,EV> fromCollection(Collection<Edge<K,EV>> edges,
                                                          ExecutionEnvironment context)

Creates a graph from a Collection of edges. Vertices are created automatically and their values are set to NullValue.

Parameters:

edges - a Collection of edges.

context - the flink execution environment.

Returns:

the newly created graph.

fromCollection

public static <K,VV,EV> Graph<K,VV,EV> fromCollection(Collection<Edge<K,EV>> edges,
                                                      MapFunction<K,VV> vertexValueInitializer,
                                                      ExecutionEnvironment context)

Creates a graph from a Collection of edges. Vertices are created automatically and their values are set by applying the provided map function to the vertex IDs.

Parameters:

edges - a Collection of edges.

vertexValueInitializer - a map function that initializes the vertex values. It allows to apply a map transformation on the vertex ID to produce an initial vertex value.

context - the flink execution environment.

Returns:

the newly created graph.

fromDataSet

public static <K,VV,EV> Graph<K,VV,EV> fromDataSet(DataSet<Vertex<K,VV>> vertices,
                                                   DataSet<Edge<K,EV>> edges,
                                                   ExecutionEnvironment context)

Creates a graph from a DataSet of vertices and a DataSet of edges.

Parameters:

vertices - a DataSet of vertices.

edges - a DataSet of edges.

context - the flink execution environment.

Returns:

the newly created graph.

fromDataSet

public static <K,EV> Graph<K,NullValue,EV> fromDataSet(DataSet<Edge<K,EV>> edges,
                                                       ExecutionEnvironment context)

Creates a graph from a DataSet of edges. Vertices are created automatically and their values are set to NullValue.

Parameters:

edges - a DataSet of edges.

context - the flink execution environment.

Returns:

the newly created graph.

fromDataSet

public static <K,VV,EV> Graph<K,VV,EV> fromDataSet(DataSet<Edge<K,EV>> edges,
                                                   MapFunction<K,VV> vertexValueInitializer,
                                                   ExecutionEnvironment context)

Creates a graph from a DataSet of edges. Vertices are created automatically and their values are set by applying the provided map function to the vertex IDs.

Parameters:

edges - a DataSet of edges.

vertexValueInitializer - the mapper function that initializes the vertex values. It allows to apply a map transformation on the vertex ID to produce an initial vertex value.

context - the flink execution environment.

Returns:

the newly created graph.

fromTupleDataSet

public static <K,VV,EV> Graph<K,VV,EV> fromTupleDataSet(DataSet<Tuple2<K,VV>> vertices,
                                                        DataSet<Tuple3<K,K,EV>> edges,
                                                        ExecutionEnvironment context)

Creates a graph from a DataSet of Tuple2 objects for vertices and Tuple3 objects for edges.

The first field of the Tuple2 vertex object will become the vertex ID and the second field will become the vertex value. The first field of the Tuple3 object for edges will become the source ID, the second field will become the target ID, and the third field will become the edge value.

Parameters:

vertices - a DataSet of Tuple2 representing the vertices.

edges - a DataSet of Tuple3 representing the edges.

context - the flink execution environment.

Returns:

the newly created graph.

fromTupleDataSet

public static <K,EV> Graph<K,NullValue,EV> fromTupleDataSet(DataSet<Tuple3<K,K,EV>> edges,
                                                            ExecutionEnvironment context)

Creates a graph from a DataSet of Tuple3 objects for edges.

The first field of the Tuple3 object will become the source ID, the second field will become the target ID, and the third field will become the edge value.

Vertices are created automatically and their values are set to NullValue.

Parameters:

edges - a DataSet of Tuple3 representing the edges.

context - the flink execution environment.

Returns:

the newly created graph.

fromTupleDataSet

public static <K,VV,EV> Graph<K,VV,EV> fromTupleDataSet(DataSet<Tuple3<K,K,EV>> edges,
                                                        MapFunction<K,VV> vertexValueInitializer,
                                                        ExecutionEnvironment context)

Creates a graph from a DataSet of Tuple3 objects for edges.

Each Tuple3 will become one Edge, where the source ID will be the first field of the Tuple2, the target ID will be the second field of the Tuple2 and the Edge value will be the third field of the Tuple3.

Vertices are created automatically and their values are initialized by applying the provided vertexValueInitializer map function to the vertex IDs.

Parameters:

edges - a DataSet of Tuple3.

vertexValueInitializer - the mapper function that initializes the vertex values. It allows to apply a map transformation on the vertex ID to produce an initial vertex value.

context - the flink execution environment.

Returns:

the newly created graph.

fromTuple2DataSet

public static <K> Graph<K,NullValue,NullValue> fromTuple2DataSet(DataSet<Tuple2<K,K>> edges,
                                                                 ExecutionEnvironment context)

Creates a graph from a DataSet of Tuple2 objects for edges. Each Tuple2 will become one Edge, where the source ID will be the first field of the Tuple2 and the target ID will be the second field of the Tuple2.

Edge value types and Vertex values types will be set to NullValue.

Parameters:

edges - a DataSet of Tuple2.

context - the flink execution environment.

Returns:

the newly created graph.

fromTuple2DataSet

public static <K,VV> Graph<K,VV,NullValue> fromTuple2DataSet(DataSet<Tuple2<K,K>> edges,
                                                             MapFunction<K,VV> vertexValueInitializer,
                                                             ExecutionEnvironment context)

Creates a graph from a DataSet of Tuple2 objects for edges. Each Tuple2 will become one Edge, where the source ID will be the first field of the Tuple2 and the target ID will be the second field of the Tuple2.

Edge value types will be set to NullValue. Vertex values can be initialized by applying a user-defined map function on the vertex IDs.

Parameters:

edges - a DataSet of Tuple2, where the first field corresponds to the source ID and the second field corresponds to the target ID.

vertexValueInitializer - the mapper function that initializes the vertex values. It allows to apply a map transformation on the vertex ID to produce an initial vertex value.

context - the flink execution environment.

Returns:

the newly created graph.

fromCsvReader

public static GraphCsvReader fromCsvReader(String verticesPath,
                                           String edgesPath,
                                           ExecutionEnvironment context)

Creates a Graph from a CSV file of vertices and a CSV file of edges.

Parameters:

verticesPath - path to a CSV file with the Vertex data.

edgesPath - path to a CSV file with the Edge data

context - the Flink execution environment.

Returns:

An instance of GraphCsvReader, on which calling methods to specify types of the Vertex ID, Vertex value and Edge value returns a Graph.

See Also:

GraphCsvReader.types(Class, Class, Class), GraphCsvReader.vertexTypes(Class, Class), GraphCsvReader.edgeTypes(Class, Class), GraphCsvReader.keyType(Class)

fromCsvReader

public static GraphCsvReader fromCsvReader(String edgesPath,
                                           ExecutionEnvironment context)

Creates a graph from a CSV file of edges. Vertices will be created automatically.

Parameters:

edgesPath - a path to a CSV file with the Edges data

context - the execution environment.

Returns:

An instance of GraphCsvReader, on which calling methods to specify types of the Vertex ID, Vertex value and Edge value returns a Graph.

See Also:

GraphCsvReader.types(Class, Class, Class), GraphCsvReader.vertexTypes(Class, Class), GraphCsvReader.edgeTypes(Class, Class), GraphCsvReader.keyType(Class)

fromCsvReader

public static <K,VV> GraphCsvReader fromCsvReader(String edgesPath,
                                                  MapFunction<K,VV> vertexValueInitializer,
                                                  ExecutionEnvironment context)

Creates a graph from a CSV file of edges. Vertices will be created automatically and Vertex values can be initialized using a user-defined mapper.

Parameters:

edgesPath - a path to a CSV file with the Edge data

vertexValueInitializer - the mapper function that initializes the vertex values. It allows to apply a map transformation on the vertex ID to produce an initial vertex value.

context - the execution environment.

Returns:

An instance of GraphCsvReader, on which calling methods to specify types of the Vertex ID, Vertex Value and Edge value returns a Graph.

See Also:

GraphCsvReader.types(Class, Class, Class), GraphCsvReader.vertexTypes(Class, Class), GraphCsvReader.edgeTypes(Class, Class), GraphCsvReader.keyType(Class)

getContext

public ExecutionEnvironment getContext()

Returns:

the flink execution environment.

validate

public Boolean validate(GraphValidator<K,VV,EV> validator)
                 throws Exception

Function that checks whether a Graph is a valid Graph, as defined by the given GraphValidator.

Returns:

true if the Graph is valid.

Throws:

Exception

getVertices

public DataSet<Vertex<K,VV>> getVertices()

Returns:

the vertex DataSet.

getEdges

public DataSet<Edge<K,EV>> getEdges()

Returns:

the edge DataSet.

getVerticesAsTuple2

public DataSet<Tuple2<K,VV>> getVerticesAsTuple2()

Returns:

the vertex DataSet as Tuple2.

getEdgesAsTuple3

public DataSet<Tuple3<K,K,EV>> getEdgesAsTuple3()

Returns:

the edge DataSet as Tuple3.

getTriplets

public DataSet<Triplet<K,VV,EV>> getTriplets()

This method allows access to the graph's edge values along with its source and target vertex values.

Returns:

a triplet DataSet consisting of (srcVertexId, trgVertexId, srcVertexValue, trgVertexValue, edgeValue)

mapVertices

public <NV> Graph<K,NV,EV> mapVertices(MapFunction<Vertex<K,VV>,NV> mapper)

Apply a function to the attribute of each vertex in the graph.

Parameters:

mapper - the map function to apply.

Returns:

a new graph

mapVertices

public <NV> Graph<K,NV,EV> mapVertices(MapFunction<Vertex<K,VV>,NV> mapper,
                                       TypeInformation<Vertex<K,NV>> returnType)

Apply a function to the attribute of each vertex in the graph.

Parameters:

mapper - the map function to apply.

returnType - the explicit return type.

Returns:

a new graph

mapEdges

public <NV> Graph<K,VV,NV> mapEdges(MapFunction<Edge<K,EV>,NV> mapper)

Apply a function to the attribute of each edge in the graph.

Parameters:

mapper - the map function to apply.

Returns:

a new graph

mapEdges

public <NV> Graph<K,VV,NV> mapEdges(MapFunction<Edge<K,EV>,NV> mapper,
                                    TypeInformation<Edge<K,NV>> returnType)

Apply a function to the attribute of each edge in the graph.

Parameters:

mapper - the map function to apply.

returnType - the explicit return type.

Returns:

a new graph

translateGraphIds

public <NEW> Graph<NEW,VV,EV> translateGraphIds(TranslateFunction<K,NEW> translator)
                                         throws Exception

Translate Vertex and Edge IDs using the given MapFunction.

Type Parameters:

NEW - new ID type

Parameters:

translator - implements conversion from K to NEW

Returns:

graph with translated vertex and edge IDs

Throws:

Exception

translateVertexValues

public <NEW> Graph<K,NEW,EV> translateVertexValues(TranslateFunction<VV,NEW> translator)
                                            throws Exception

Translate Vertex values using the given MapFunction.

Type Parameters:

NEW - new vertex value type

Parameters:

translator - implements conversion from VV to NEW

Returns:

graph with translated vertex values

Throws:

Exception

translateEdgeValues

public <NEW> Graph<K,VV,NEW> translateEdgeValues(TranslateFunction<EV,NEW> translator)
                                          throws Exception

Translate Edge values using the given MapFunction.

Type Parameters:

NEW - new edge value type

Parameters:

translator - implements conversion from EV to NEW

Returns:

graph with translated edge values

Throws:

Exception

joinWithVertices

public <T> Graph<K,VV,EV> joinWithVertices(DataSet<Tuple2<K,T>> inputDataSet,
                                           VertexJoinFunction<VV,T> vertexJoinFunction)

Joins the vertex DataSet of this graph with an input Tuple2 DataSet and applies a user-defined transformation on the values of the matched records. The vertex ID and the first field of the Tuple2 DataSet are used as the join keys.

Type Parameters:

T - the type of the second field of the input Tuple2 DataSet.

Parameters:

inputDataSet - the Tuple2 DataSet to join with. The first field of the Tuple2 is used as the join key and the second field is passed as a parameter to the transformation function.

vertexJoinFunction - the transformation function to apply. The first parameter is the current vertex value and the second parameter is the value of the matched Tuple2 from the input DataSet.

Returns:

a new Graph, where the vertex values have been updated according to the result of the vertexJoinFunction.

joinWithEdges

public <T> Graph<K,VV,EV> joinWithEdges(DataSet<Tuple3<K,K,T>> inputDataSet,
                                        EdgeJoinFunction<EV,T> edgeJoinFunction)

Joins the edge DataSet with an input DataSet on the composite key of both source and target IDs and applies a user-defined transformation on the values of the matched records. The first two fields of the input DataSet are used as join keys.

Type Parameters:

T - the type of the third field of the input Tuple3 DataSet.

Parameters:

inputDataSet - the DataSet to join with. The first two fields of the Tuple3 are used as the composite join key and the third field is passed as a parameter to the transformation function.

edgeJoinFunction - the transformation function to apply. The first parameter is the current edge value and the second parameter is the value of the matched Tuple3 from the input DataSet.

Returns:

a new Graph, where the edge values have been updated according to the result of the edgeJoinFunction.

joinWithEdgesOnSource

public <T> Graph<K,VV,EV> joinWithEdgesOnSource(DataSet<Tuple2<K,T>> inputDataSet,
                                                EdgeJoinFunction<EV,T> edgeJoinFunction)

Joins the edge DataSet with an input Tuple2 DataSet and applies a user-defined transformation on the values of the matched records. The source ID of the edges input and the first field of the input DataSet are used as join keys.

Type Parameters:

T - the type of the second field of the input Tuple2 DataSet.

Parameters:

inputDataSet - the DataSet to join with. The first field of the Tuple2 is used as the join key and the second field is passed as a parameter to the transformation function.

edgeJoinFunction - the transformation function to apply. The first parameter is the current edge value and the second parameter is the value of the matched Tuple2 from the input DataSet.

Returns:

a new Graph, where the edge values have been updated according to the result of the edgeJoinFunction.

joinWithEdgesOnTarget

public <T> Graph<K,VV,EV> joinWithEdgesOnTarget(DataSet<Tuple2<K,T>> inputDataSet,
                                                EdgeJoinFunction<EV,T> edgeJoinFunction)

Joins the edge DataSet with an input Tuple2 DataSet and applies a user-defined transformation on the values of the matched records. The target ID of the edges input and the first field of the input DataSet are used as join keys.

Type Parameters:

T - the type of the second field of the input Tuple2 DataSet.

Parameters:

inputDataSet - the DataSet to join with. The first field of the Tuple2 is used as the join key and the second field is passed as a parameter to the transformation function.

edgeJoinFunction - the transformation function to apply. The first parameter is the current edge value and the second parameter is the value of the matched Tuple2 from the input DataSet.

Returns:

a new Graph, where the edge values have been updated according to the result of the edgeJoinFunction.

subgraph

public Graph<K,VV,EV> subgraph(FilterFunction<Vertex<K,VV>> vertexFilter,
                               FilterFunction<Edge<K,EV>> edgeFilter)

Apply filtering functions to the graph and return a sub-graph that satisfies the predicates for both vertices and edges.

Parameters:

vertexFilter - the filter function for vertices.

edgeFilter - the filter function for edges.

Returns:

the resulting sub-graph.

filterOnVertices

public Graph<K,VV,EV> filterOnVertices(FilterFunction<Vertex<K,VV>> vertexFilter)

Apply a filtering function to the graph and return a sub-graph that satisfies the predicates only for the vertices.

Parameters:

vertexFilter - the filter function for vertices.

Returns:

the resulting sub-graph.

filterOnEdges

public Graph<K,VV,EV> filterOnEdges(FilterFunction<Edge<K,EV>> edgeFilter)

Apply a filtering function to the graph and return a sub-graph that satisfies the predicates only for the edges.

Parameters:

edgeFilter - the filter function for edges.

Returns:

the resulting sub-graph.

outDegrees

public DataSet<Tuple2<K,LongValue>> outDegrees()

Return the out-degree of all vertices in the graph.

Returns:

A DataSet of Tuple2<vertexId, outDegree>

inDegrees

public DataSet<Tuple2<K,LongValue>> inDegrees()

Return the in-degree of all vertices in the graph.

Returns:

A DataSet of Tuple2<vertexId, inDegree>

getDegrees

public DataSet<Tuple2<K,LongValue>> getDegrees()

Return the degree of all vertices in the graph.

Returns:

A DataSet of Tuple2<vertexId, degree>

getUndirected

public Graph<K,VV,EV> getUndirected()

This operation adds all inverse-direction edges to the graph.

Returns:

the undirected graph.

groupReduceOnEdges

public <T> DataSet<T> groupReduceOnEdges(EdgesFunctionWithVertexValue<K,VV,EV,T> edgesFunction,
                                         EdgeDirection direction)
                                  throws IllegalArgumentException

Groups by vertex and computes a GroupReduce transformation over the edge values of each vertex. The edgesFunction applied on the edges has access to both the id and the value of the grouping vertex.

For each vertex, the edgesFunction can iterate over all edges of this vertex with the specified direction, and emit any number of output elements, including none.

Type Parameters:

T - the output type

Parameters:

edgesFunction - the group reduce function to apply to the neighboring edges of each vertex.

direction - the edge direction (in-, out-, all-).

Returns:

a DataSet containing elements of type T

Throws:

IllegalArgumentException

groupReduceOnEdges

public <T> DataSet<T> groupReduceOnEdges(EdgesFunctionWithVertexValue<K,VV,EV,T> edgesFunction,
                                         EdgeDirection direction,
                                         TypeInformation<T> typeInfo)
                                  throws IllegalArgumentException

Groups by vertex and computes a GroupReduce transformation over the edge values of each vertex. The edgesFunction applied on the edges has access to both the id and the value of the grouping vertex.

For each vertex, the edgesFunction can iterate over all edges of this vertex with the specified direction, and emit any number of output elements, including none.

Type Parameters:

T - the output type

Parameters:

edgesFunction - the group reduce function to apply to the neighboring edges of each vertex.

direction - the edge direction (in-, out-, all-).

typeInfo - the explicit return type.

Returns:

a DataSet containing elements of type T

Throws:

IllegalArgumentException

groupReduceOnEdges

public <T> DataSet<T> groupReduceOnEdges(EdgesFunction<K,EV,T> edgesFunction,
                                         EdgeDirection direction)
                                  throws IllegalArgumentException

Groups by vertex and computes a GroupReduce transformation over the edge values of each vertex. The edgesFunction applied on the edges only has access to the vertex id (not the vertex value) of the grouping vertex.

For each vertex, the edgesFunction can iterate over all edges of this vertex with the specified direction, and emit any number of output elements, including none.

Type Parameters:

T - the output type

Parameters:

edgesFunction - the group reduce function to apply to the neighboring edges of each vertex.

direction - the edge direction (in-, out-, all-).

Returns:

a DataSet containing elements of type T

Throws:

IllegalArgumentException

groupReduceOnEdges

public <T> DataSet<T> groupReduceOnEdges(EdgesFunction<K,EV,T> edgesFunction,
                                         EdgeDirection direction,
                                         TypeInformation<T> typeInfo)
                                  throws IllegalArgumentException

Groups by vertex and computes a GroupReduce transformation over the edge values of each vertex. The edgesFunction applied on the edges only has access to the vertex id (not the vertex value) of the grouping vertex.

For each vertex, the edgesFunction can iterate over all edges of this vertex with the specified direction, and emit any number of output elements, including none.

Type Parameters:

T - the output type

Parameters:

edgesFunction - the group reduce function to apply to the neighboring edges of each vertex.

direction - the edge direction (in-, out-, all-).

typeInfo - the explicit return type.

Returns:

a DataSet containing elements of type T

Throws:

IllegalArgumentException

reverse

public Graph<K,VV,EV> reverse()
                       throws UnsupportedOperationException

Reverse the direction of the edges in the graph.

Returns:

a new graph with all edges reversed

Throws:

UnsupportedOperationException

numberOfVertices

public long numberOfVertices()
                      throws Exception

Returns:

a long integer representing the number of vertices

Throws:

Exception

numberOfEdges

public long numberOfEdges()
                   throws Exception

Returns:

a long integer representing the number of edges

Throws:

Exception

getVertexIds

public DataSet<K> getVertexIds()

Returns:

The IDs of the vertices as DataSet

getEdgeIds

public DataSet<Tuple2<K,K>> getEdgeIds()

Returns:

The IDs of the edges as DataSet

addVertex

public Graph<K,VV,EV> addVertex(Vertex<K,VV> vertex)

Adds the input vertex to the graph. If the vertex already exists in the graph, it will not be added again.

Parameters:

vertex - the vertex to be added

Returns:

the new graph containing the existing vertices as well as the one just added

addVertices

public Graph<K,VV,EV> addVertices(List<Vertex<K,VV>> verticesToAdd)

Adds the list of vertices, passed as input, to the graph. If the vertices already exist in the graph, they will not be added once more.

Parameters:

verticesToAdd - the list of vertices to add

Returns:

the new graph containing the existing and newly added vertices

addEdge

public Graph<K,VV,EV> addEdge(Vertex<K,VV> source,
                              Vertex<K,VV> target,
                              EV edgeValue)

Adds the given edge to the graph. If the source and target vertices do not exist in the graph, they will also be added.

Parameters:

source - the source vertex of the edge

target - the target vertex of the edge

edgeValue - the edge value

Returns:

the new graph containing the existing vertices and edges plus the newly added edge

addEdges

public Graph<K,VV,EV> addEdges(List<Edge<K,EV>> newEdges)

Adds the given list edges to the graph.

When adding an edge for a non-existing set of vertices, the edge is considered invalid and ignored.

Parameters:

newEdges - the data set of edges to be added

Returns:

a new graph containing the existing edges plus the newly added edges.

removeVertex

public Graph<K,VV,EV> removeVertex(Vertex<K,VV> vertex)

Removes the given vertex and its edges from the graph.

Parameters:

vertex - the vertex to remove

Returns:

the new graph containing the existing vertices and edges without the removed vertex and its edges

removeVertices

public Graph<K,VV,EV> removeVertices(List<Vertex<K,VV>> verticesToBeRemoved)

Removes the given list of vertices and its edges from the graph.

Parameters:

verticesToBeRemoved - the list of vertices to be removed

Returns:

the resulted graph containing the initial vertices and edges minus the vertices and edges removed.

removeEdge

public Graph<K,VV,EV> removeEdge(Edge<K,EV> edge)

Removes all edges that match the given edge from the graph.

Parameters:

edge - the edge to remove

Returns:

the new graph containing the existing vertices and edges without the removed edges

removeEdges

public Graph<K,VV,EV> removeEdges(List<Edge<K,EV>> edgesToBeRemoved)

Removes all the edges that match the edges in the given data set from the graph.

Parameters:

edgesToBeRemoved - the list of edges to be removed

Returns:

a new graph where the edges have been removed and in which the vertices remained intact

union

public Graph<K,VV,EV> union(Graph<K,VV,EV> graph)

Performs union on the vertices and edges sets of the input graphs removing duplicate vertices but maintaining duplicate edges.

Parameters:

graph - the graph to perform union with

Returns:

a new graph

difference

public Graph<K,VV,EV> difference(Graph<K,VV,EV> graph)

Performs Difference on the vertex and edge sets of the input graphs removes common vertices and edges. If a source/target vertex is removed, its corresponding edge will also be removed

Parameters:

graph - the graph to perform difference with

Returns:

a new graph where the common vertices and edges have been removed

intersect

public Graph<K,NullValue,EV> intersect(Graph<K,VV,EV> graph,
                                       boolean distinctEdges)

Performs intersect on the edge sets of the input graphs. Edges are considered equal, if they have the same source identifier, target identifier and edge value.

The method computes pairs of equal edges from the input graphs. If the same edge occurs multiple times in the input graphs, there will be multiple edge pairs to be considered. Each edge instance can only be part of one pair. If the given parameter distinctEdges is set to true, there will be exactly one edge in the output graph representing all pairs of equal edges. If the parameter is set to false, both edges of each pair will be in the output.

Vertices in the output graph will have no vertex values.

Parameters:

graph - the graph to perform intersect with

distinctEdges - if set to true, there will be exactly one edge in the output graph representing all pairs of equal edges, otherwise, for each pair, both edges will be in the output graph

Returns:

a new graph which contains only common vertices and edges from the input graphs

runScatterGatherIteration

public <M> Graph<K,VV,EV> runScatterGatherIteration(ScatterFunction<K,VV,M,EV> scatterFunction,
                                                    GatherFunction<K,VV,M> gatherFunction,
                                                    int maximumNumberOfIterations)

Runs a ScatterGather iteration on the graph. No configuration options are provided.

Parameters:

scatterFunction - the scatter function

gatherFunction - the gather function

maximumNumberOfIterations - maximum number of iterations to perform

Returns:

the updated Graph after the scatter-gather iteration has converged or after maximumNumberOfIterations.

runScatterGatherIteration

public <M> Graph<K,VV,EV> runScatterGatherIteration(ScatterFunction<K,VV,M,EV> scatterFunction,
                                                    GatherFunction<K,VV,M> gatherFunction,
                                                    int maximumNumberOfIterations,
                                                    ScatterGatherConfiguration parameters)

Runs a ScatterGather iteration on the graph with configuration options.

Parameters:

scatterFunction - the scatter function

gatherFunction - the gather function

maximumNumberOfIterations - maximum number of iterations to perform

parameters - the iteration configuration parameters

Returns:

the updated Graph after the scatter-gather iteration has converged or after maximumNumberOfIterations.

runGatherSumApplyIteration

public <M> Graph<K,VV,EV> runGatherSumApplyIteration(GatherFunction<VV,EV,M> gatherFunction,
                                                     SumFunction<VV,EV,M> sumFunction,
                                                     ApplyFunction<K,VV,M> applyFunction,
                                                     int maximumNumberOfIterations)

Runs a Gather-Sum-Apply iteration on the graph. No configuration options are provided.

Type Parameters:

M - the intermediate type used between gather, sum and apply

Parameters:

gatherFunction - the gather function collects information about adjacent vertices and edges

sumFunction - the sum function aggregates the gathered information

applyFunction - the apply function updates the vertex values with the aggregates

maximumNumberOfIterations - maximum number of iterations to perform

Returns:

the updated Graph after the gather-sum-apply iteration has converged or after maximumNumberOfIterations.

runGatherSumApplyIteration

public <M> Graph<K,VV,EV> runGatherSumApplyIteration(GatherFunction<VV,EV,M> gatherFunction,
                                                     SumFunction<VV,EV,M> sumFunction,
                                                     ApplyFunction<K,VV,M> applyFunction,
                                                     int maximumNumberOfIterations,
                                                     GSAConfiguration parameters)

Runs a Gather-Sum-Apply iteration on the graph with configuration options.

Type Parameters:

M - the intermediate type used between gather, sum and apply

Parameters:

gatherFunction - the gather function collects information about adjacent vertices and edges

sumFunction - the sum function aggregates the gathered information

applyFunction - the apply function updates the vertex values with the aggregates

maximumNumberOfIterations - maximum number of iterations to perform

parameters - the iteration configuration parameters

Returns:

the updated Graph after the gather-sum-apply iteration has converged or after maximumNumberOfIterations.

runVertexCentricIteration

public <M> Graph<K,VV,EV> runVertexCentricIteration(ComputeFunction<K,VV,EV,M> computeFunction,
                                                    MessageCombiner<K,M> combiner,
                                                    int maximumNumberOfIterations)

Runs a VertexCentricIteration on the graph. No configuration options are provided.

Parameters:

computeFunction - the vertex compute function

combiner - an optional message combiner

maximumNumberOfIterations - maximum number of iterations to perform

Returns:

the updated Graph after the vertex-centric iteration has converged or after maximumNumberOfIterations.

runVertexCentricIteration

public <M> Graph<K,VV,EV> runVertexCentricIteration(ComputeFunction<K,VV,EV,M> computeFunction,
                                                    MessageCombiner<K,M> combiner,
                                                    int maximumNumberOfIterations,
                                                    VertexCentricConfiguration parameters)

Runs a VertexCentricIteration on the graph with configuration options.

Parameters:

computeFunction - the vertex compute function

combiner - an optional message combiner

maximumNumberOfIterations - maximum number of iterations to perform

parameters - the VertexCentricConfiguration parameters

Returns:

the updated Graph after the vertex-centric iteration has converged or after maximumNumberOfIterations.

run

public <T> T run(GraphAlgorithm<K,VV,EV,T> algorithm)
          throws Exception

Type Parameters:

T - the return type

Parameters:

algorithm - the algorithm to run on the Graph

Returns:

the result of the graph algorithm

Throws:

Exception

run

public <T> GraphAnalytic<K,VV,EV,T> run(GraphAnalytic<K,VV,EV,T> analytic)
                                 throws Exception

A GraphAnalytic is similar to a GraphAlgorithm but is terminal and results are retrieved via accumulators. A Flink program has a single point of execution. A GraphAnalytic defers execution to the user to allow composing multiple analytics and algorithms into a single program.

Type Parameters:

T - the result type

Parameters:

analytic - the analytic to run on the Graph

Throws:

Exception

groupReduceOnNeighbors

public <T> DataSet<T> groupReduceOnNeighbors(NeighborsFunctionWithVertexValue<K,VV,EV,T> neighborsFunction,
                                             EdgeDirection direction)
                                      throws IllegalArgumentException

Groups by vertex and computes a GroupReduce transformation over the neighbors (both edges and vertices) of each vertex. The neighborsFunction applied on the neighbors only has access to both the vertex id and the vertex value of the grouping vertex.

For each vertex, the neighborsFunction can iterate over all neighbors of this vertex with the specified direction, and emit any number of output elements, including none.

Type Parameters:

T - the output type

Parameters:

neighborsFunction - the group reduce function to apply to the neighboring edges and vertices of each vertex.

direction - the edge direction (in-, out-, all-).

Returns:

a DataSet containing elements of type T

Throws:

IllegalArgumentException

groupReduceOnNeighbors

public <T> DataSet<T> groupReduceOnNeighbors(NeighborsFunctionWithVertexValue<K,VV,EV,T> neighborsFunction,
                                             EdgeDirection direction,
                                             TypeInformation<T> typeInfo)
                                      throws IllegalArgumentException

Groups by vertex and computes a GroupReduce transformation over the neighbors (both edges and vertices) of each vertex. The neighborsFunction applied on the neighbors only has access to both the vertex id and the vertex value of the grouping vertex.

For each vertex, the neighborsFunction can iterate over all neighbors of this vertex with the specified direction, and emit any number of output elements, including none.

Type Parameters:

T - the output type

Parameters:

neighborsFunction - the group reduce function to apply to the neighboring edges and vertices of each vertex.

direction - the edge direction (in-, out-, all-).

typeInfo - the explicit return type

Returns:

a DataSet containing elements of type T

Throws:

IllegalArgumentException

groupReduceOnNeighbors

public <T> DataSet<T> groupReduceOnNeighbors(NeighborsFunction<K,VV,EV,T> neighborsFunction,
                                             EdgeDirection direction)
                                      throws IllegalArgumentException

Groups by vertex and computes a GroupReduce transformation over the neighbors (both edges and vertices) of each vertex. The neighborsFunction applied on the neighbors only has access to the vertex id (not the vertex value) of the grouping vertex.

For each vertex, the neighborsFunction can iterate over all neighbors of this vertex with the specified direction, and emit any number of output elements, including none.

Type Parameters:

T - the output type

Parameters:

neighborsFunction - the group reduce function to apply to the neighboring edges and vertices of each vertex.

direction - the edge direction (in-, out-, all-).

Returns:

a DataSet containing elements of type T

Throws:

IllegalArgumentException

groupReduceOnNeighbors

public <T> DataSet<T> groupReduceOnNeighbors(NeighborsFunction<K,VV,EV,T> neighborsFunction,
                                             EdgeDirection direction,
                                             TypeInformation<T> typeInfo)
                                      throws IllegalArgumentException

Groups by vertex and computes a GroupReduce transformation over the neighbors (both edges and vertices) of each vertex. The neighborsFunction applied on the neighbors only has access to the vertex id (not the vertex value) of the grouping vertex.

For each vertex, the neighborsFunction can iterate over all neighbors of this vertex with the specified direction, and emit any number of output elements, including none.

Type Parameters:

T - the output type

Parameters:

neighborsFunction - the group reduce function to apply to the neighboring edges and vertices of each vertex.

direction - the edge direction (in-, out-, all-).

typeInfo - the explicit return type

Returns:

a DataSet containing elements of type T

Throws:

IllegalArgumentException

reduceOnNeighbors

public DataSet<Tuple2<K,VV>> reduceOnNeighbors(ReduceNeighborsFunction<VV> reduceNeighborsFunction,
                                               EdgeDirection direction)
                                        throws IllegalArgumentException

Compute a reduce transformation over the neighbors' vertex values of each vertex. For each vertex, the transformation consecutively calls a ReduceNeighborsFunction until only a single value for each vertex remains. The ReduceNeighborsFunction combines a pair of neighbor vertex values into one new value of the same type.

Parameters:

reduceNeighborsFunction - the reduce function to apply to the neighbors of each vertex.

direction - the edge direction (in-, out-, all-)

Returns:

a Dataset of Tuple2, with one tuple per vertex. The first field of the Tuple2 is the vertex ID and the second field is the aggregate value computed by the provided ReduceNeighborsFunction.

Throws:

IllegalArgumentException

reduceOnEdges

public DataSet<Tuple2<K,EV>> reduceOnEdges(ReduceEdgesFunction<EV> reduceEdgesFunction,
                                           EdgeDirection direction)
                                    throws IllegalArgumentException

Compute a reduce transformation over the edge values of each vertex. For each vertex, the transformation consecutively calls a ReduceEdgesFunction until only a single value for each edge remains. The ReduceEdgesFunction combines two edge values into one new value of the same type.

Parameters:

reduceEdgesFunction - the reduce function to apply to the neighbors of each vertex.

direction - the edge direction (in-, out-, all-)

Returns:

a Dataset of Tuple2, with one tuple per vertex. The first field of the Tuple2 is the vertex ID and the second field is the aggregate value computed by the provided ReduceEdgesFunction.

Throws:

IllegalArgumentException