Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Local Flink config running standalone from IDE

Tags:

apache-flink

If I'd like to run a Flink app locally, directly from within Intellij but I need to specify config params (like fs.hdfs.hdfssite to set up S3 access), is there any other way to provide those config params apart from ExecutionEnvironment.createLocalEnvironment(conf)? What if I want to use StreamExecutionEnvironment.getExecutionEnvironment? Can I have a Flink config in my project and point the local app to it?

Is this the proper way to do it? Or would you set up your IDE to submit the app to a real local Flink instance?

like image 396
bibac Avatar asked Jun 10 '16 08:06

bibac


2 Answers

To create a StreamExecutionEnvironment with configuration options, use this call StreamExecutionEnvironment.createLocalEnvironment(int parallelism, Configuration configuration)

like image 137
Robert Metzger Avatar answered Dec 06 '22 16:12

Robert Metzger


I could load the flink-conf.yaml file by providing the ENV_VAR FLINK_CONF_DIR in IntelliJ Run Configuration

like image 34
Averell Avatar answered Dec 06 '22 16:12

Averell