Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I drain a pipeline from within another pipeline?

I need to programmatically drain a pipeline from within another pipeline. The DataflowPipelineJob class doesn't have a drain method implemented. Is it possible to call drain on a pipeline from with the Java API?

like image 382
Valentin Avatar asked Jan 15 '17 13:01

Valentin


1 Answers

Was able to initiate draining with the following code:

    // spawn child pipe
    DataflowPipelineRunner runner = DataflowPipelineRunner.fromOptions(options);
    DataflowPipelineJob job = runner.run(p);

    // under the some condition later, drain the spawned pipe:
    Dataflow client = com.google.cloud.dataflow.sdk.util.Transport.newDataflowClient(options).build();
    Job content = new Job();
    content.setProjectId(options.getProject());
    content.setId(job.getJobId());
    content.setRequestedState("JOB_STATE_DRAINING");
    client.projects().jobs()
            .update(options.getProject(), job.getJobId(), content)
            .execute();
like image 126
Valentin Avatar answered Nov 07 '22 02:11

Valentin