In the Luigi docs, the use of a luigi.Config
class is recommended for global configuration.
However, I am running into issues when using such a config class in order to pass a commandline argument to various Tasks in the pipeline.
Here's a lightweight example:
import datetime
import luigi
class HelloWorldTask(luigi.Task):
def run(self):
print("{task} says: Hello world on {date}!".format(task=self.__class__.__name__,
date=GlobalParams.date.strftime('%d-%b-%Y')))
class GlobalParams(luigi.Config):
date = luigi.DateParameter(default=datetime.date.today())
if __name__ == '__main__':
luigi.run(['HelloWorldTask', '--workers', '1', '--local-scheduler',
'--GlobalParams-date', '2018-01-01'])
The class GlobalParams
defines a DateParameter
which I would like to later reference in the run()
blocks of pipeline Tasks. However, this fails with the error,
AttributeError: 'DateParameter' object has no attribute 'strftime'
.
In the debugger, I can see that a DateParameter
object is passed to the HelloWorldTask
Task, but any attempts to extract the expected '2018-01-01'
value passed at runtime fails.
Am I misunderstanding how to use these constructs? How should I be passing a single parameter to (possibly many) Tasks?
The issue in the example code is that the GlobalParams class is not being instantiated before its parameter is accessed.GlobalParams.date.strftime('%d-%b-%Y')
should read GlobalParams().date.strftime('%d-%b-%Y')
.
This is included, but easy to overlook, in the configuration docs linked in the question.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With