org.apache.reef.runtime.yarn.client

Class YarnSubmissionHelper

java.lang.Object

org.apache.reef.runtime.yarn.client.YarnSubmissionHelper

All Implemented Interfaces:

AutoCloseable

public final class YarnSubmissionHelper
extends Object
implements AutoCloseable

Helper code that wraps the YARN Client API for our purposes.

Constructor Summary

Constructors

Constructor and Description
YarnSubmissionHelper(org.apache.hadoop.yarn.conf.YarnConfiguration yarnConfiguration, REEFFileNames fileNames, ClasspathProvider classpath, YarnProxyUser yarnProxyUser, SecurityTokenProvider tokenProvider, boolean isUnmanaged)
YarnSubmissionHelper(org.apache.hadoop.yarn.conf.YarnConfiguration yarnConfiguration, REEFFileNames fileNames, ClasspathProvider classpath, YarnProxyUser yarnProxyUser, SecurityTokenProvider tokenProvider, boolean isUnmanaged, List<String> commandPrefixList)

Method Summary

Methods

Modifier and Type	Method and Description
YarnSubmissionHelper	addLocalResource(String resourceName, org.apache.hadoop.yarn.api.records.LocalResource resource) Add a file to be localized on the driver.
void	close()
int	getApplicationId()
String	getStringApplicationId()
YarnSubmissionHelper	setApplicationName(String applicationName) Set the name of the application to be submitted.
YarnSubmissionHelper	setConfigurationFilePaths(List<String> configurationFilePaths) Sets the configuration file for the job.
YarnSubmissionHelper	setDriverMemory(int megabytes) Set the amount of memory to be allocated to the Driver.
YarnSubmissionHelper	setDriverStderrPath(String driverStderrPath) Sets the Driver stderr file path.
YarnSubmissionHelper	setDriverStdoutPath(String driverStdoutPath) Sets the Driver stdout file path.
YarnSubmissionHelper	setLauncherClass(Class launcherClass) Sets the launcher class for the job.
YarnSubmissionHelper	setMaxApplicationAttempts(int maxApplicationAttempts) Sets the maximum application attempts for the application.
YarnSubmissionHelper	setPreserveEvaluators(boolean preserveEvaluators) Set whether or not the resource manager should preserve evaluators across driver restarts.
YarnSubmissionHelper	setPriority(int priority) Set the priority of the job.
YarnSubmissionHelper	setQueue(String queueName) Assign this job submission to a queue.
void	submit()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

YarnSubmissionHelper

public YarnSubmissionHelper(org.apache.hadoop.yarn.conf.YarnConfiguration yarnConfiguration,
                    REEFFileNames fileNames,
                    ClasspathProvider classpath,
                    YarnProxyUser yarnProxyUser,
                    SecurityTokenProvider tokenProvider,
                    boolean isUnmanaged,
                    List<String> commandPrefixList)
                     throws IOException,
                            org.apache.hadoop.yarn.exceptions.YarnException

Throws:

IOException

org.apache.hadoop.yarn.exceptions.YarnException

YarnSubmissionHelper

public YarnSubmissionHelper(org.apache.hadoop.yarn.conf.YarnConfiguration yarnConfiguration,
                    REEFFileNames fileNames,
                    ClasspathProvider classpath,
                    YarnProxyUser yarnProxyUser,
                    SecurityTokenProvider tokenProvider,
                    boolean isUnmanaged)
                     throws IOException,
                            org.apache.hadoop.yarn.exceptions.YarnException

Throws:

IOException

org.apache.hadoop.yarn.exceptions.YarnException

Method Detail

getApplicationId

public int getApplicationId()

Returns:

the application ID assigned by YARN.

getStringApplicationId

public String getStringApplicationId()

Returns:

the application ID string representation assigned by YARN.

setApplicationName

public YarnSubmissionHelper setApplicationName(String applicationName)

Set the name of the application to be submitted.

Parameters:

applicationName -

Returns:

setDriverMemory

public YarnSubmissionHelper setDriverMemory(int megabytes)

Set the amount of memory to be allocated to the Driver.

Parameters:

megabytes -

Returns:

addLocalResource

public YarnSubmissionHelper addLocalResource(String resourceName,
                                    org.apache.hadoop.yarn.api.records.LocalResource resource)

Add a file to be localized on the driver.

Parameters:

resourceName -

resource -

Returns:

setPriority

public YarnSubmissionHelper setPriority(int priority)

Set the priority of the job.

Parameters:

priority -

Returns:

setPreserveEvaluators

public YarnSubmissionHelper setPreserveEvaluators(boolean preserveEvaluators)

Set whether or not the resource manager should preserve evaluators across driver restarts.

Parameters:

preserveEvaluators -

Returns:

setMaxApplicationAttempts

public YarnSubmissionHelper setMaxApplicationAttempts(int maxApplicationAttempts)

Sets the maximum application attempts for the application.

Parameters:

maxApplicationAttempts -

Returns:

setQueue

public YarnSubmissionHelper setQueue(String queueName)

Assign this job submission to a queue.

Parameters:

queueName -

Returns:

setLauncherClass

public YarnSubmissionHelper setLauncherClass(Class launcherClass)

Sets the launcher class for the job.

Parameters:

launcherClass -

Returns:

setConfigurationFilePaths

public YarnSubmissionHelper setConfigurationFilePaths(List<String> configurationFilePaths)

Sets the configuration file for the job. Note that this does not have to be Driver TANG configuration. In the bootstrap launch case, this can be the set of the Avro files that supports the generation of a driver configuration file natively at the Launcher.

Parameters:

configurationFilePaths -

Returns:

setDriverStdoutPath

public YarnSubmissionHelper setDriverStdoutPath(String driverStdoutPath)

Sets the Driver stdout file path.

Parameters:

driverStdoutPath -

Returns:

setDriverStderrPath

public YarnSubmissionHelper setDriverStderrPath(String driverStderrPath)

Sets the Driver stderr file path.

Parameters:

driverStderrPath -

Returns:

submit

public void submit()
            throws IOException,
                   org.apache.hadoop.yarn.exceptions.YarnException

Throws:

IOException

org.apache.hadoop.yarn.exceptions.YarnException

close

public void close()

Specified by:

close in interface AutoCloseable