Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Install Multiple Package using Sagemaker Life Cycle configuration file

I wanted to know how I can pre-install Python packages in Sagemaker before spinning it up?

For example, I want to install Tensorfliw, LightFM, and Scikit-optimize

How can i code a Lifecycle Configuration file which will tell sagemaker to install these packages before it spins up and have it ready when I am ready to code.

I know the following resources:

  1. https://aws.amazon.com/blogs/machine-learning/customize-your-amazon-sagemaker-notebook-instances-with-lifecycle-configurations-and-the-option-to-disable-internet-access/
  2. https://docs.aws.amazon.com/sagemaker/latest/dg/nbi-add-external.html
like image 912
Kshitij Yadav Avatar asked Aug 16 '19 16:08

Kshitij Yadav


People also ask

What is life cycle configuration in SageMaker?

Lifecycle Configurations are shell scripts triggered by Amazon SageMaker Studio lifecycle events, such as starting a new Studio notebook. You can use Lifecycle Configurations to automate customization for your Studio environment.

Can I PIP install in SageMaker?

SageMaker notebooks support the following package installation tools: conda install. pip install.

What are the limitations of SageMaker?

SageMaker does not allow you to schedule training jobs. SageMaker does not provide a mechanism for easily tracking metrics logged during training. We often fit feature extraction and model pipelines. We can inject the model artifacts into AWS-provided containers, but we cannot inject the feature extractors.


1 Answers

For Tensorflow, there is an existing Conda environment(tensorflow_p36) with TensorFlow preinstalled that you can use. For other packages that aren't present by default, you can use this Lifecycle Configuration script sample to install them into the tensorflow_p36 environment.

like image 182
Jaipreet Avatar answered Sep 19 '22 17:09

Jaipreet