The Dataflow jobs are cluttered all over my dashboard, and I'd like to delete the failed jobs from my project. But in the dashboard, I don't see any option to delete the Dataflow job. I'm looking for something like below at least,
$ gcloud beta dataflow jobs delete JOB_ID
To delete all jobs,
$ gcloud beta dataflow jobs delete
Can someone please help me with this?
Cloud Dataflow currently does not provide a mechanism to restart a Dataflow job that has been stopped or cancelled.
Dataflow is a managed service for executing a wide variety of data processing patterns. The documentation on this site shows you how to deploy your batch and streaming data processing pipelines using Dataflow, including directions for using service features.
Unfortunately, this is not currently possible. You cannot delete a Dataflow job. This is something that you could request via the public issue tracker (I've wanted it in the past too).
gcloud dataflow jobs --help
COMMANDS
COMMAND is one of the following:
cancel Cancels all jobs that match the command line arguments. describe Outputs the Job object resulting from the Get API. drain Drains all jobs that match the command line arguments. list Lists all jobs in a particular project. run Runs a job from the specified path. show Shows a short description of the given job.
As Graham mentions, it is not possible to delete Dataflow jobs. However, note that you can filter the job list to only show the jobs you care about. For example, Status:Running,Succeeded
will exclude all failed or cancelled jobs.
On the commandline, you can use --status=(active|terminated|all)
:
gcloud beta dataflow jobs list --status=active
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With