Is it possible to directly call docker run from AWS lambda

Question

I have a Java standalone application which I have dockerized. I want to run this docker everytime an object is put into S3 storage. On way is to do it via AWS batch which I am trying to avoid.

Is there a direct and easy way to call docker run from a lambda?

Answer 1

Yes and no.

What you can't do is execute docker run to run a container within the context of the Lambda call. But you can trigger a task on ECS to be executed. For this to work, you need to have a cluster set up on ECS, which means you need to pay for at least one EC2 instance. Because of that, it might be better to not use Docker, but I know too little about your application to judge that.

There are a lot of articles out there how to connect S3, Lambda and ECS. Here is a pretty in-depth article by Amazon that you might be interested in:

https://aws.amazon.com/blogs/compute/better-together-amazon-ecs-and-aws-lambda/

If you are looking for code, this repository implements what is discussed in the above article:

https://github.com/awslabs/lambda-ecs-worker-pattern

Here is a snippet we use in our Lambda function (Python) to run a Docker container from Lambda:

result = boto3.client('ecs').run_task(
    cluster=cluster,
    taskDefinition=task_definition,
    overrides=overrides,
    count=1,
    startedBy='lambda'
)

We pass in the name of the cluster on which we want to run the container, as well as the task definition that defines which container to run, the resources it needs and so on. overrides is a dictionary/map with settings that you want to override in the task definition, which we use to specify the command we want to run (i.e. the argument to docker run). This enables us to use the same Lambda function to run a lot of different jobs on ECS.

Hope that points you in the right direction.

Answer 2

Yes. It is possible to run containers out Docker images stored in Docker Hub within AWS Lambda using SCAR.

For example, you can create a Lambda function to execute a container out of the ubuntu:16.04 image in Docker Hub as follows:

scar init ubuntu:16.04

And then you can run a command or a shell-script within that container upon each invocation of the function:

scar run scar-ubuntu-16-04 whoami
SCAR: Request Id: ed5e9f09-ce0c-11e7-8375-6fc6859242f0
Log group name: /aws/lambda/scar-ubuntu-16-04
Log stream name: 2017/11/20/[$LATEST]7e53ed01e54a451494832e21ea933fca
---------------------------------------------------------------------------
sbx_user1059

You can use your own Docker images stored in Docker Hub. Some limitations apply but it can be effectively used to run generic applications on AWS Lambda. It also features a programming model for file-processing event-driven applications. It uses uDocker under the hood.