Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to use azureml.core.runconfig.DockerConfiguration class in azureml.core.Environment or azureml.core.ScriptRunConfig class

I use Microsoft Azure Machine Learning (Azure-ml) to run my (python) experiments.

For specifying the VM and python environment I use:

from azureml.core import Environment
from azureml.core import ScriptRunConfig

# Other imports and code...

# Specify VM and Python environment:
vm_env = Environment.from_conda_specification(name='my-test-env', file_path=PATH_TO_YAML_FILE)
vm_env.docker.enabled = True
vm_env.docker.base_image = 'mcr.microsoft.com/azureml/openmpi3.1.2-cuda10.2-cudnn7-ubuntu18.04'

# Finally, use the environment in the ScriptRunConfig:
src = ScriptRunConfig(source_directory=DEPLOY_CONTAINER_FOLDER_PATH,
                      script=SCRIPT_FILE_TO_EXECUTE,
                      arguments=EXECUTE_ARGUMENTS,
                      compute_target=compute_target,
                      environment=vm_env)

I get the following warning for the line vm_env.docker.enabled = True:

'enabled' is deprecated. Please use the azureml.core.runconfig.DockerConfiguration object with the 'use_docker' param instead.

The documentation about the DockerSection Class and DockerConfiguration Class is not very clear about applying the DockerConfiguration Class.

I can't figure out how to use the azureml.core.runconfig.DockerConfiguration object. Can someone provide me with an example? Thank you!

like image 665
Stefan Avatar asked May 04 '21 14:05

Stefan


People also ask

Which SDK commands can you use to view the registered environments in your workspace?

For a registered environment, you can retrieve image details using the following code where details is an instance of DockerImageDetails (AzureML Python SDK >= 1.11) and provides all the information about the environment image such as the dockerfile, registry, and image name.

What is ScriptRunConfig?

A ScriptRunConfig packages together the configuration information needed to submit a run in Azure ML, including the script, compute target, environment, and any distributed job-specific configs. Once a script run is configured and submitted with the submit, a ScriptRun is returned.


2 Answers

The ScriptRunConfig class now accepts a docker_runtime_config argument, which is where you pass the DockerConfiguration object.

So, the code would look something like this:

from azureml.core import Environment
from azureml.core import ScriptRunConfig
from azureml.core.runconfig import DockerConfiguration

# Other imports and code...

# Specify VM and Python environment:
vm_env = Environment.from_conda_specification(name='my-test-env', file_path=PATH_TO_YAML_FILE)
vm_env.docker.base_image = 'mcr.microsoft.com/azureml/openmpi3.1.2-cuda10.2-cudnn7-ubuntu18.04'

docker_config = DockerConfiguration(use_docker=True)

# Finally, use the environment in the ScriptRunConfig:
src = ScriptRunConfig(source_directory=DEPLOY_CONTAINER_FOLDER_PATH,
                      script=SCRIPT_FILE_TO_EXECUTE,
                      arguments=EXECUTE_ARGUMENTS,
                      compute_target=compute_target,
                      environment=vm_env,
                      docker_runtime_config=docker_config)
like image 131
Erik Z Avatar answered Nov 15 '22 09:11

Erik Z


Adding another example for anyone using RunConfiguration:

Change:

run_config = RunConfiguration()
run_config.environment.docker.enabled = True

To:

run_config = RunConfiguration()
docker_config = DockerConfiguration(use_docker=True)
run_config.docker = docker_config

run_config can later be used as a parameter for e.g. PythonScriptStep.

Note that the docker attribute moved from the internal environment to the RunConfiguration directly.

like image 27
Moshe Zvi Avatar answered Nov 15 '22 09:11

Moshe Zvi