Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Measure job execution time in flink

Tags:

apache-flink

Is there any way to measure job execution time in Apache Flink when submitting the job to flink using command line?

PS. I want the flink API to give me the time rather than measuring it myself in bash by noting the start and end times

like image 208
orak Avatar asked Dec 12 '15 18:12

orak


People also ask

What is backpressure in Flink?

Flink uses backpressure to adapt the processing speed of individual operators. The operator can struggle to keep up processing the message volume it receives for many reasons. The operation may require more CPU resources than the operator has available, The operator may wait for I/O operations to complete.

What is Job Manager in Flink?

The Job manager receives jobs from clients, divides the jobs into tasks, and sends the tasks to the workers. Workers communicate statistics and results. tally to each sample generating a new DataStreams.

What is a job in Flink?

A Flink job is first in the created state, then switches to running and upon completion of all work it switches to finished. In case of failures, a job switches first to failing where it cancels all running tasks.


1 Answers

The ExecutionEnvironment.execute() method returns a JobExecutionResult object containing the job runtime.

You could for example do something like this:

// execute program
JobExecutionResult result = env.execute("My Flink Job");
System.out.println("The job took " + result.getNetRuntime(TimeUnit.SECONDS) + " to execute");
like image 127
Robert Metzger Avatar answered Sep 19 '22 15:09

Robert Metzger