Menu
I have a notebook on SageMaker I would like to run every night. What's the best way to schedule this task. Is there a way to run a bash script and schedule Cron job from SageMaker?
875
To execute a notebook in Amazon SageMaker, you use a Lambda function that sets up and runs an Amazon SageMaker Processing job. The function can be invoked directly by the user or added as a target of an Amazon EventBridge rule to run on a schedule or in response to an event.
Manage interactions with the Amazon SageMaker APIs and any other AWS services needed. This class provides convenient methods for manipulating entities and resources that Amazon SageMaker uses, such as training jobs, endpoints, and input datasets in S3.
In AWS Glue, you can create a development endpoint and then create a SageMaker notebook to help develop your ETL and machine learning scripts.
Maximum number of feature definitions per feature group: 2500. Maximum Transactions per second (TPS) per API per AWS account: Soft limit of 10000 TPS per API excluding the BatchGetRecord API call, which has a soft limit of 500 TPS. Maximum size of a record: 350KB.
As of mid 2020, AWS provides several options to run a notebook as a cron job. It uses Papermill to inject parameters per run, and you can also use the CLI to run the notebook on demand.
You can:
(1) use the AWS APIs or CLI directly; (2) use a convenience package, or (3) use a Jupyter Lab extension.
See this tutorial and the Quick Start guide for examples.
75
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
