caching go modules in codebuild without custom docker image

Question

TL;DR

How can I cache my modules in codebuild using AWS provided image (Go 1.12)?

Background

I'm trying to cache go modules in codebuild using the go image (1.12) from AWS.

Trying to cache /go/pkg/mod

After digging deeper, I found that there is no /go/pkg folder in that image. Hence, when I tried to cache /go/pkg it would throw an error.

Error mounting /go/pkg/mod: symlink /codebuild/local-cache/custom//go/pkg/mod /go/pkg/mod: no such file or directory

Even after I run go mod download (which will create the /go/pkg/mod, it won't cache the folder because codebuild cannot mounted it earlier).

This is my codebuild.yml

version: 0.2

phases:
  install:
    runtime-versions:
      golang: 1.12
      nodejs: 10
    commands:
      - npm install
  build:
    commands:
      - go build -ldflags="-s -w" -o api/bin/main api/main.go
cache:
  paths:
    - /go/src/**/*
    - /go/pkg/mod/**/*

Trying to cache ./vendor

I also tried caching ./vendor folder which doesn't throw any errors in codebuild. However, I don't think it's caching anything because the build time doesn't decrease. It also says it ignores the symlink.

warning: ignoring symlink /codebuild/output/src074479210/src/github.com/kkesley/myrepo/vendor go: finding github.com/aws/aws-lambda-go v1.11.1 go: finding github.com/stretchr/testify v1.2.1 go: finding github.com/pmezard/go-difflib v1.0.0 go: finding github.com/davecgh/go-spew v1.1.0 go: finding gopkg.in/urfave/cli.v1 v1.20.0 go: downloading github.com/aws/aws-lambda-go v1.11.1 go: extracting github.com/aws/aws-lambda-go v1.11.1

This is my codebuild.yml for this version:

version: 0.2

phases:
  install:
    runtime-versions:
      golang: 1.12
      nodejs: 10
    commands:
      - npm install
      - go mod vendor
  build:
    commands:
      - go build -mod vendor -ldflags="-s -w" -o api/bin/main api/main.go
cache:
  paths:
    - /go/src/**/*
    - vendor/**/*

Question

How do you cache go modules in code build without using custom docker image? Is it possible?

Answer 1

To get it working with the default CodeBuild Ubuntu build image (I'm using v4)

1. Make sure that local caching is enabled on the CodeBuild project. Go to to edit then artifacts then make sure Custom Cache is ticked

2. Set the path to cache as /go/pkg/**.*

cache:
  paths:
    - '/go/pkg/**/*'

3. Make sure your build script includes a step before building for go mod download. Before I did this caching didn't seem to work so this seemed a key step.

Here is my full buildspec.yml for reference

version: 0.2

phases:
  install:
    runtime-versions:
      golang: latest
    commands:
      - "git config --global credential.helper '!aws codecommit credential-helper $@'"
      - "git config --global credential.UseHttpPath true"
  build:
    commands:
      - 'go mod edit -dropreplace git-codecommit.ap-southeast-2.amazonaws.com/v1/repos/xyz'
      - 'go mod download'
      # Use latest from develop for the build (test env only)
      - 'go get git-codecommit.ap-southeast-2.amazonaws.com/v1/repos/xyz@develop'
      - 'rm -rf "dist"'
      - 'cp -r "eb-template" "dist"'
      - 'env GOOS=linux GOARCH=amd64 go build -o "dist/bin/server"'
      - 'go mod edit -replace git-codecommit.ap-southeast-2.amazonaws.com/v1/repos/xyz=../xyz'
      - 'echo -n ${CODEBUILD_RESOLVED_SOURCE_VERSION} > dist/commithash'

artifacts:
  base-directory: dist
  files:
    - '**/*'

cache:
  paths:
    - '/go/pkg/**/*'