flink - adding instrumentation

Question

I want to add NewRelic instrumentation to my flink jobs. I don't see where it's possible to pass additional classpath / other params to the bin/flink run <job> command.

The NewRelic java agent wants -javaagent:<path to jar> added to the execution path. Passing in the config file path is advisable as well.

---

Edit:

I added this line to my conf/flink-conf.yaml on all (3) cluster machines:

env.java.opts: "-javaagent:/opt/newrelic/newrelic.jar -Dnewrelic.config.file=/opt/newrelic/newrelic.yml"

When I go to start the cluster only the job manager will start. The task manager doesn't start on any of the machines.

The only way I've found to add instrumentation (so far) is to change the command line at the end of bin/flink to include the above parameters. This is fine except that it requires the session where the command was running to remain open. If you exit out, the Flink job keeps going but the NewRelic agent quits.

Answer 1

You can pass additional JVM start-up parameters via the env.java.opts config value which you can set in Flink's configuration file flink-conf.yaml.

Answer 2

First remove the quotes in the value(right side)

env.java.opts: -javaagent:/opt/newrelic/newrelic.jar -Dnewrelic.config.file=/opt/newrelic/newrelic.yml

And also make sure that you put the files in "lib" directory of flink and rewrite the command as

env.java.opts: -javaagent:lib/newrelic.jar -Dnewrelic.config.file=lib/newrelic.yml

All the files in "lib" directory will be copied to job manager and task managers and available in the relative path "./lib"