MLFlow active run does not match environment run id

Question

I am trying to perform an MLFlow run but stuck with the following error after trying a lot of things.

run = mlflow.active_run()
if run:
    print("Active run_id: {}".format(run.info.run_id))
    mlflow.end_run()

mlflow.set_experiment('TNF_EXP') 
mlflow.set_tracking_uri("http://localhost:5000/") # Actual Server URI instead of localhost
experiment = mlflow.get_experiment_by_name("TNF_EXP")

with mlflow.start_run(experiment_id=experiment.experiment_id) as run:
...
...

mlflow.end_run()

Error -

File "/.../ModelTrainer.py", line 108, in train
    with mlflow.start_run(experiment_id=experiment.experiment_id) as run:
  File "/usr/local/lib/python3.6/site-packages/mlflow/tracking/fluent.py", line 207, in start_run
    "arguments".format(existing_run_id)
mlflow.exceptions.MlflowException: Cannot start run with ID e9953eb5918845bb9be1xxxxxx because active run ID does not match environment run ID. Make sure --experiment-name or --experiment-id matches experiment set with set_experiment(), or just use command-line arguments
2021/02/11 09:41:36 ERROR mlflow.cli: === Run (ID 'e9953eb5918845bb9be1xxxxxx') failed ===

I noticed I had an active run earlier so I included the first if block to end that run. The code ran successfully and I was able to log the data on MLFlow UI but now when I run it I start getting the same issue. There are no active runs found before starting a new run currently.

FYI, I am running the code on Azure server with the respective tracking URI mentioned in the code.

However the code runs fine if I include an argument --experiment-name="TNF_EXP" in the mlflow run command on the CLI

Answer 1

That is primarily because you have started a run with default experiment name and then you are trying to set the experiment_name as "TNF_EXP".

Will suggest you to make use of mlflow.run(..., experiment_name="TNF_EXP") python method then running it from the CLI.

You can find more information here.