Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to delete a gcloud Dataflow job?

The Dataflow jobs are cluttered all over my dashboard, and I'd like to delete the failed jobs from my project. But in the dashboard, I don't see any option to delete the Dataflow job. I'm looking for something like below at least,

$ gcloud beta dataflow jobs delete JOB_ID

To delete all jobs,

$ gcloud beta dataflow jobs delete

Can someone please help me with this?

like image 902
Vijin Paulraj Avatar asked Sep 04 '17 14:09

Vijin Paulraj


People also ask

How do I restart a Dataflow job?

Cloud Dataflow currently does not provide a mechanism to restart a Dataflow job that has been stopped or cancelled.

What is Dataflow job in GCP?

Dataflow is a managed service for executing a wide variety of data processing patterns. The documentation on this site shows you how to deploy your batch and streaming data processing pipelines using Dataflow, including directions for using service features.


2 Answers

Unfortunately, this is not currently possible. You cannot delete a Dataflow job. This is something that you could request via the public issue tracker (I've wanted it in the past too).

gcloud dataflow jobs --help

COMMANDS

COMMAND is one of the following:

 cancel
    Cancels all jobs that match the command line arguments.

 describe
    Outputs the Job object resulting from the Get API.

 drain
    Drains all jobs that match the command line arguments.

 list
    Lists all jobs in a particular project.

 run
    Runs a job from the specified path.

 show
    Shows a short description of the given job.
like image 195
Graham Polley Avatar answered Nov 24 '22 04:11

Graham Polley


As Graham mentions, it is not possible to delete Dataflow jobs. However, note that you can filter the job list to only show the jobs you care about. For example, Status:Running,Succeeded will exclude all failed or cancelled jobs.

Dataflow Job filter

On the commandline, you can use --status=(active|terminated|all):

gcloud beta dataflow jobs list --status=active
like image 42
Scott Wegner Avatar answered Nov 24 '22 04:11

Scott Wegner