Is there any way to measure job execution time in Apache Flink when submitting the job to flink using command line?
PS. I want the flink API to give me the time rather than measuring it myself in bash by noting the start and end times
Flink uses backpressure to adapt the processing speed of individual operators. The operator can struggle to keep up processing the message volume it receives for many reasons. The operation may require more CPU resources than the operator has available, The operator may wait for I/O operations to complete.
The Job manager receives jobs from clients, divides the jobs into tasks, and sends the tasks to the workers. Workers communicate statistics and results. tally to each sample generating a new DataStreams.
A Flink job is first in the created state, then switches to running and upon completion of all work it switches to finished. In case of failures, a job switches first to failing where it cancels all running tasks.
The ExecutionEnvironment.execute()
method returns a JobExecutionResult
object containing the job runtime.
You could for example do something like this:
// execute program
JobExecutionResult result = env.execute("My Flink Job");
System.out.println("The job took " + result.getNetRuntime(TimeUnit.SECONDS) + " to execute");
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With