Running lambdas in localstack in gitlab-ci

Question

So I have localstack running locally (on my laptop) and can deploy serverless app to it and then invoke a Lambda.
However, I am really struggling with doing the same thing in gitlab-ci.
This is the relevant part of .gitlab-ci.yml:

integration-test:
  stage: integration-test
  image: node:14-alpine3.12
  tags:
    - docker
  services:
    - name: localstack/localstack
      alias: localstack
  variables:
    LAMBDA_EXECUTOR: docker
    HOSTNAME_EXTERNAL: localstack
    DEFAULT_REGION: eu-west-1
    USE_SSL: "false"
    DEBUG: "1"
    AWS_ACCESS_KEY_ID: test
    AWS_SECRET_ACCESS_KEY: test
    AWS_DEFAULT_REGION: eu-west-1
  script:
    - npm ci
    - npx sls deploy --stage local
    - npx jest --testMatch='**/*.integration.js'
  only:
    - merge_requests

The localstack gets started and the deployment works fine. But as soon as lambda is invoked (in an integration test), localstack is trying to create a container for the lambda to run in and that's when it fails with the following:

Lambda process returned error status code: 1. Result: . Output:\\nCannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?\\nmust specify at least one container source (.....)

I tried to set DOCKER_HOST to tcp://docker:2375 but then it fails with:

Lambda process returned error status code: 1. Result: . Output:\\nerror during connect: Post http://docker:2375/v1.29/containers/create: dial tcp: lookup docker on 169.254.169.254:53: no such host\

DOCKER_HOST set to tcp://localhost:2375 complains too:

Lambda process returned error status code: 1. Result: . Output:\\nCannot connect to the Docker daemon at tcp://localhost:2375. Is the docker daemon running?\\nmust specify at least one container source

Did anyone ever get lambdas to run within localstack within shared gitlab runners?
Thanks for your help :)

Answer 1

Running docker in docker is usually a bad idea, since it's a big security vulnerability. Granting access to local docker daemon equals granting root privileges on a runner.

If you still want to use docker installed on host to spawn containers, refer to official documentation - https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-socket-binding

which boils down to adding

volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]

to [runners.docker] section in your runner config.

The question is, why do you need docker? According to https://github.com/localstack/localstack , setting LAMBDA_EXECUTOR to local will

run Lambda functions in a temporary directory on the local machine

Which should be the best approach to your problem, and won't compromise security of your runner host.