AWS Batch vs AWS CodeBuild

Question

I am very new in AWS and when I was searching something to download a code from GitHub (a python project), run it, and save the output in s3 the first service that I found was CodeBuild.

So I implement this kind of workflow using CodeBuild.

But now I have seen that AWS have a service called AWS Batch and I am wondering if I should migrate my arquitecture to AWS Batch.

Can you explain which one - AWS CodeBuild or AWS Batch - is more suitable with my case? When use AWS Batch instead of AWS CodeBuild?

Thank very much.

Answer 1

TLDR Summary: AWS Codebuild is the nicer choice for simple jobs.

---

My (reverse) experience...

I needed to run a simple job that pulls data from external api, read/write to external database, and generate a CSV report.

The job takes ~1 hour to run, so AWS Lambda is out of the picture.

After some googling, I found AWS Batch and decided to give Creating a Simple “Fetch & Run” AWS Batch Job a try.

The required steps to this "simple" job working:

• Build a Docker image with the fetch & run script
• Create an Amazon ECR repository for the image
• Push the built image to ECR
• Create a simple job script and upload it to S3
• Create an IAM role to be used by jobs to access S3
• Configure a compute environment
• Create a job queue
• Create a job definition that uses the built image
• Submit and run a job that execute the job script from S3

After spending the time to create all these resources, it did not work out of the box. I found myself debugging random things I shouldn't have to debug such as:

• Dockerfile
• entrypoint script
• ECS cluster
• EC2 instance and autoscaling group

After failing to find simple practical examples, and realizing the amount of effort required, I decided to explore other solutions.

I stumbled onto Using AWS CodeBuild to execute administrative tasks and this post.

I've used AWS Codebuild in the past for CI/CD pipeline, and thought "what the hell, lets give it a try". In a shorter amount of time, I was able to get a "codebuild job" running on a cloudwatch scheduler and codebuild slack notifications added, with less effort:

• Connect build project to your source code
• Select a runtime environment
• Create IAM role
• Create a buildspec.yml and add runtime commands

One major advantage is that CodeBuild runs tasks in a full-blown Linux environment.

Drawbacks:

• Max execution time of 8 hours

AWS Codebuild was much easier to get working for my simple job.

Sorry for the long post, just wanted to share my experience with these 2 services.