I have installed luigi by pip command and I would like to change the port for the web UI. I tried to find the config file but I couldn't. Do I need to create one?
By default, Luigi tasks run using the Luigi scheduler. To run one of your previous tasks using the Luigi scheduler omit the --local-scheduler argument from the command.
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
worker module. The worker communicates with the scheduler and does two things: Sends all tasks that has to be run. Gets tasks from the scheduler that should be run.
You can start luigid with the --port option.
luigid --port 80
Configuration file locations are:
in increasing order of preference. You do need to create one. e.g.,
[core]
default-scheduler-host=www.example.com
default-scheduler-port=8088
In spite of the documentation saying otherwise, the port configuration from the config file is not used, at least in some versions or some circumstances
Until this is resolved, you should always use the --port
option of luigid:
luigid --port 12345
Also see https://github.com/spotify/luigi/issues/2235
For other configuration options a config file should be used. See https://luigi.readthedocs.io/en/stable/configuration.html
For a configuration global to the host you can create a file:
/etc/luigi/luigi.cfg
Make sure it is readable by the user that runs luigid and luig.
Alternatively a local configuration file that will be recognized is
luigi.cfg
which you would have to create in the current working directory.
If you want a custom config file location you could set the environment variable LUIGI_CONFIG_PATH
to the full path of your config file.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With