az synapse spark job
Manage Synapse Spark batch jobs.
Commands
az synapse spark job cancel |
Cancel a Spark job. |
az synapse spark job list |
List all Spark jobs. |
az synapse spark job show |
Get a Spark job. |
az synapse spark job submit |
Submit a Spark job. |
az synapse spark job cancel
Cancel a Spark job.
az synapse spark job cancel --livy-id
--spark-pool-name
--workspace-name
[--subscription]
[--yes]
Examples
Cancel a Spark job.
az synapse spark job cancel --livy-id 1 --workspace-name testsynapseworkspace --spark-pool-name testsparkpool
Required Parameters
The id of the Spark job.
The name of the Spark pool.
The name of the workspace.
Optional Parameters
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID
.
Do not prompt for confirmation.
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Increase logging verbosity. Use --debug for full debug logs.
az synapse spark job list
List all Spark jobs.
az synapse spark job list --spark-pool-name
--workspace-name
[--from-index]
[--query-examples]
[--size]
[--subscription]
Examples
List all Spark jobs.
az synapse spark job list --workspace-name testsynapseworkspace --spark-pool-name testsparkpool
Required Parameters
The name of the Spark pool.
The name of the workspace.
Optional Parameters
Optional parameter specifying which index the list should begin from.
Recommend JMESPath string for you. You can copy one of the query and paste it after --query parameter within double quotation marks to see the results. You can add one or more positional keywords so that we can give suggestions based on these key words.
The size of the returned list.By default it is 20 and that is the maximum.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID
.
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Increase logging verbosity. Use --debug for full debug logs.
az synapse spark job show
Get a Spark job.
az synapse spark job show --livy-id
--spark-pool-name
--workspace-name
[--query-examples]
[--subscription]
Examples
Get a Spark job.
az synapse spark job show --livy-id 1 --workspace-name testsynapseworkspace --spark-pool-name testsparkpool
Required Parameters
The id of the Spark job.
The name of the Spark pool.
The name of the workspace.
Optional Parameters
Recommend JMESPath string for you. You can copy one of the query and paste it after --query parameter within double quotation marks to see the results. You can add one or more positional keywords so that we can give suggestions based on these key words.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID
.
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Increase logging verbosity. Use --debug for full debug logs.
az synapse spark job submit
Submit a Spark job.
az synapse spark job submit --executor-size {Large, Medium, Small}
--executors
--main-class-name
--main-definition-file
--name
--spark-pool-name
--workspace-name
[--archives]
[--arguments]
[--configuration]
[--language {CSharp, PySpark, Python, Scala, Spark, SparkDotNet}]
[--reference-files]
[--subscription]
[--tags]
Examples
Submit a Java Spark job.
az synapse spark job submit --name WordCount_Java --workspace-name testsynapseworkspace \
--spark-pool-name testsparkpool \
--main-definition-file abfss://testfilesystem@testadlsgen2.dfs.core.windows.net/samples/java/wordcount/wordcount.jar \
--main-class-name WordCount \
--arguments abfss://testfilesystem@testadlsgen2.dfs.core.windows.net/samples/java/wordcount/shakespeare.txt \
abfss://testfilesystem@testadlsgen2.dfs.core.windows.net/samples/java/wordcount/result/ \
--executors 2 --executor-size Small
Required Parameters
The executor size.
The number of executors.
The fully-qualified identifier or the main class that is in the main definition file.
The main file used for the job.
The Spark job name.
The name of the Spark pool.
The name of the workspace.
Optional Parameters
The array of archives.
Optional arguments to the job (Note: please use storage URIs for file arguments).
The configuration of Spark job.
The Spark job language.
Additional files used for reference in the main definition file.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID
.
Space-separated tags: key[=value] [key[=value] ...]. Use "" to clear existing tags.
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Increase logging verbosity. Use --debug for full debug logs.