com.baidubce.services.bmr.model
Class SparkStepConfig

java.lang.Object
  extended by com.baidubce.services.bmr.model.StepConfig
      extended by com.baidubce.services.bmr.model.SparkStepConfig

public class SparkStepConfig
extends StepConfig

Represent configuration for a spark step.

A spark step can be configured with name, actionOnFailure, jar, submitOptions and arguments. The essential options are mapper, input, output and actionOnFailure, and the optional ones are name and arguments.


Constructor Summary
SparkStepConfig()
           
 
Method Summary
 SparkStepConfig withActionOnFailure(String actionOnFailure)
          Configure the action on failure for the java step.
 SparkStepConfig withAdditionalFile(String remote, String local)
          Configure the additional file for the step.
 SparkStepConfig withArguments(String arguments)
          Configure the arguments for the step.
 SparkStepConfig withJar(String jar)
          Configure the BOS path for step's .jar file.
 SparkStepConfig withName(String name)
          Configure the name of the step.
 SparkStepConfig withSubmitOptions(String submitOptions)
          Configure the submit options for the step.
 
Methods inherited from class com.baidubce.services.bmr.model.StepConfig
addAdditionalFile, addProperty, getActionOnFailure, getAdditionalFiles, getName, getProperties, getType, setActionOnFailure, setAdditionalFiles, setName, setProperties, setType
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

SparkStepConfig

public SparkStepConfig()
Method Detail

withJar

public SparkStepConfig withJar(String jar)
Configure the BOS path for step's .jar file.

Parameters:
jar - The BOS path for the step's .jar file.
Returns:
SparkStepConfig

withSubmitOptions

public SparkStepConfig withSubmitOptions(String submitOptions)
Configure the submit options for the step.

Parameters:
submitOptions - The main class for the step.
Returns:
SparkStepConfig

withArguments

public SparkStepConfig withArguments(String arguments)
Configure the arguments for the step.

Parameters:
arguments - The arguments for the step.
Returns:
SparkStepConfig

withActionOnFailure

public SparkStepConfig withActionOnFailure(String actionOnFailure)
Configure the action on failure for the java step. This property is set to enum value: "Continue": continue to execute other steps. "TerminateCluster": terminate the cluster when this step fails. "CancelAndWait": cancel the other pending steps and set the cluster's status to WAITING.

Parameters:
actionOnFailure - The action on step's failure.
Returns:
SparkStepConfig

withName

public SparkStepConfig withName(String name)
Configure the name of the step.

Parameters:
name - The name of the step.
Returns:
SparkStepConfig

withAdditionalFile

public SparkStepConfig withAdditionalFile(String remote,
                                          String local)
Configure the additional file for the step.

Parameters:
remote - The remote file of the additional file.
local - The local file of the additional file.
Returns:
SparkStepConfig


Copyright © 2018. All rights reserved.