public class JobGraph extends Object implements Serializable
The JobGraph is a graph of vertices and intermediate results that are connected together to form a DAG. Note that iterations (feedback edges) are currently not encoded inside the JobGraph but inside certain special vertices that establish the feedback channel amongst themselves.
The JobGraph defines the job-wide configuration settings, while each vertex and intermediate result define the characteristics of the concrete operation and intermediate data.
Constructor and Description |
---|
JobGraph()
Constructs a new job graph with no name and a random job ID.
|
JobGraph(JobID jobId,
String jobName)
Constructs a new job graph with the given name and a random job ID if null supplied as an id.
|
JobGraph(JobID jobId,
String jobName,
JobVertex... vertices)
Constructs a new job graph with the given name and a random job ID if null supplied as an id.
|
JobGraph(JobVertex... vertices)
Constructs a new job graph with no name and a random job ID if null supplied as an id.
|
JobGraph(String jobName)
Constructs a new job graph with the given name, a random job ID.
|
JobGraph(String jobName,
JobVertex... vertices)
Constructs a new job graph with the given name and a random job ID.
|
Modifier and Type | Method and Description |
---|---|
void |
addBlob(BlobKey key)
Adds the BLOB referenced by the key to the JobGraph's dependencies.
|
void |
addJar(Path jar)
Adds the path of a JAR file required to run the job on a task manager.
|
void |
addVertex(JobVertex vertex)
Adds a new task vertex to the job graph if it is not already included.
|
JobVertex |
findVertexByID(JobVertexID id)
Searches for a vertex with a matching ID and returns it.
|
boolean |
getAllowQueuedScheduling() |
List<URL> |
getClasspaths() |
Configuration |
getJobConfiguration()
Returns the configuration object for this job.
|
JobID |
getJobID()
Returns the ID of the job.
|
int |
getMaximumParallelism()
Gets the maximum parallelism of all operations in this job graph.
|
String |
getName()
Returns the name assigned to the job graph.
|
int |
getNumberOfVertices()
Returns the number of all vertices.
|
RestartStrategies.RestartStrategyConfiguration |
getRestartStrategyConfiguration()
Gets the restart strategy configuration
|
ScheduleMode |
getScheduleMode() |
long |
getSessionTimeout()
Gets the timeout after which the corresponding ExecutionGraph is removed at the
job manager after it has been executed.
|
JobSnapshottingSettings |
getSnapshotSettings()
Gets the settings for asynchronous snapshots.
|
List<BlobKey> |
getUserJarBlobKeys()
Returns a set of BLOB keys referring to the JAR files required to run this job.
|
Iterable<JobVertex> |
getVertices()
Returns an Iterable to iterate all vertices registered with the job graph.
|
JobVertex[] |
getVerticesAsArray()
Returns an array of all job vertices that are registered with the job graph.
|
List<JobVertex> |
getVerticesSortedTopologicallyFromSources() |
boolean |
hasUsercodeJarFiles()
Checks whether the JobGraph has user code JAR files attached.
|
void |
setAllowQueuedScheduling(boolean allowQueuedScheduling) |
void |
setClasspaths(List<URL> paths)
Sets the classpaths required to run the job on a task manager.
|
void |
setRestartStrategyConfiguration(RestartStrategies.RestartStrategyConfiguration restartStrategyConfiguration)
Sets the restart strategy configuration.
|
void |
setSavepointPath(String savepointPath)
Sets the savepoint path to rollback the deployment to.
|
void |
setScheduleMode(ScheduleMode scheduleMode) |
void |
setSessionTimeout(long sessionTimeout)
Sets the timeout of the session in seconds.
|
void |
setSnapshotSettings(JobSnapshottingSettings settings)
Sets the settings for asynchronous snapshots.
|
String |
toString() |
void |
uploadRequiredJarFiles(InetSocketAddress serverAddress)
Uploads the previously added user jar file to the job manager through the job manager's BLOB server.
|
public JobGraph()
public JobGraph(String jobName)
jobName
- The name of the jobpublic JobGraph(JobID jobId, String jobName)
jobId
- The id of the job. A random ID is generated, if null
is passed.jobName
- The name of the job.public JobGraph(JobVertex... vertices)
vertices
- The vertices to add to the graph.public JobGraph(String jobName, JobVertex... vertices)
jobName
- The name of the job.vertices
- The vertices to add to the graph.public JobGraph(JobID jobId, String jobName, JobVertex... vertices)
jobId
- The id of the job. A random ID is generated, if null
is passed.jobName
- The name of the job.vertices
- The vertices to add to the graph.public JobID getJobID()
public String getName()
public Configuration getJobConfiguration()
public void setRestartStrategyConfiguration(RestartStrategies.RestartStrategyConfiguration restartStrategyConfiguration)
restartStrategyConfiguration
- Restart strategy configuration to be setpublic RestartStrategies.RestartStrategyConfiguration getRestartStrategyConfiguration()
public long getSessionTimeout()
public void setSessionTimeout(long sessionTimeout)
sessionTimeout
- The timeout in secondspublic void setAllowQueuedScheduling(boolean allowQueuedScheduling)
public boolean getAllowQueuedScheduling()
public void setScheduleMode(ScheduleMode scheduleMode)
public ScheduleMode getScheduleMode()
public void addVertex(JobVertex vertex)
vertex
- the new task vertex to be addedpublic Iterable<JobVertex> getVertices()
public JobVertex[] getVerticesAsArray()
public int getNumberOfVertices()
public void setSnapshotSettings(JobSnapshottingSettings settings)
null
means that
snapshotting is not enabled.settings
- The snapshot settings, or null, to disable snapshotting.public JobSnapshottingSettings getSnapshotSettings()
public JobVertex findVertexByID(JobVertexID id)
id
- the ID of the vertex to search fornull
if no vertex with such ID could be foundpublic void setClasspaths(List<URL> paths)
paths
- paths of the directories/JAR files required to run the job on a task managerpublic void setSavepointPath(String savepointPath)
savepointPath
- The savepoint pathpublic List<JobVertex> getVerticesSortedTopologicallyFromSources() throws InvalidProgramException
InvalidProgramException
public void addJar(Path jar)
jar
- path of the JAR file required to run the job on a task managerpublic void addBlob(BlobKey key)
key
- path of the JAR file required to run the job on a task managerpublic boolean hasUsercodeJarFiles()
public List<BlobKey> getUserJarBlobKeys()
public void uploadRequiredJarFiles(InetSocketAddress serverAddress) throws IOException
serverAddress
- the network address of the BLOB serverIOException
- thrown if an I/O error occurs during the uploadpublic int getMaximumParallelism()
Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.