I have configured the Databricks cli locally and able to connect to the Azure Databricks cluster. Link reference used for my workstation - git
$ databricks jobs list --profile dev
Say if I wanted to update only the schedule (cron expression) to a specific job which is already deployed in workspace, i don't see any option to do it using databricks CLI.
Note: In my case the the jobs are created using job definition json, is used to create the jobs in the cluster. This json doesn't have the schedule info to start with.
Is there are any options available to update only schedule, after the job is created or deployed in the workspace?
There is an option to run the command immediately, databricks jobs run-now
.
The REST API configuration https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsCreate
Databricks has release a new version of their CLI which is backward incompatible (more details on migration)
The new version is >= 0.205
(last legacy <= 0.18
)
In the new CLI the commands look like:
databricks jobs get 1234 > my-job.json
# my-job.json should contain the job_id parameter
databricks jobs reset --json @path/to/my-job.json
Original answer for legacy version of CLI.
To update the job use the databricks jobs reset
command.
Databricks Azure jobs CLI docs
Documentation claims that partial updates are possible, but whenever I try to only update the schedule it complains about parameters missing.
A way around it is to read the job settings first and then editing the job json before updating:
databricks jobs get --job-id 1234 > my-job.json
then update the my-job.json
and use it with reset
command:
databricks jobs reset --job-id 1234 --json-file my-job.json
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With