Communicate between two containers in Google cloud build

Question

I am running my CI/CD pipeline in Google cloud build. My app has web and wget containers. I am trying to reach web from wget

Cloud build internally used cloudbuild bridge network while starting containers as steps. So I am expecting these steps to communicate using names. But its failing.

If I create my own docker bridge netwok then they communicating.

I want to know why cloudbuild network is not working as expected.

Please let me know if you know any other ways to establish communication between step containers.

cloudbuild.yaml

steps:

- name: 'gcr.io/cloud-builders/docker'
  id: Web server
  args: ["run", "-d", "--name", "mani", "manikantanr/hostname_ip"]

- name: 'gcr.io/cloud-builders/wget'
  id: wget web mani:8000
  args: ["-qO-", "http://mani:8000"]

To understand the cloudbuild internals I used few docker commands.

debug-cloudbuild.yaml

steps:

- name: 'gcr.io/cloud-builders/docker'
  id: Docker Version
  args: ["version"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker info
  args: ["info"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker volume ls
  args: ["volume", "ls"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker volume inspect homevol
  args: ["volume", "inspect", "homevol"]


- name: 'gcr.io/cloud-builders/docker'
  id: Docker network ls
  args: ["network", "ls"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker network inspect cloudbuild
  args: ["network", "inspect", "cloudbuild"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker ps before
  args: ["container", "ls", "--no-trunc"]

- name: 'gcr.io/cloud-builders/docker'
  id: Web server
  args: ["run", "-d", "--name", "mani", "manikantanr/hostname_ip"]
  # waitFor: ['-']

- name: 'gcr.io/cloud-builders/wget'
  id: wget ipinfo
  args: ["-qO-", "https://ipinfo.io"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker ps after
  args: ["container", "ls", "--no-trunc"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker inspect mani host network
  args: ["inspect", "mani"]

- name: 'gcr.io/cloud-builders/docker'
  id: Docker alpine ifconfig inside container
  args: ["run", "alpine", "ifconfig"]

- name: 'gcr.io/cloud-builders/wget'
  id: wget mani:8000
  args: ["-qO-", "http://mani:8000"]

Answer 1

I had a similar issue setting up integration tests on cloud build. I was trying to run integration tests from another builder (go-builder) against my other containers (started through docker-compose community built containers).

Without specifying any networks on docker-compose.yaml, all containers are started on the default network (https://docs.docker.com/compose/networking/). On cloud build, it creates a new network named cloudbuild_default and places all my containers there. By forcing all containers to join cloudbuild network through my docker-compose.yaml file, I was able to establish communications and run my tests against them.

#docker-compose.yaml

networks:
  default:
    external:
      name: cloudbuild

This might be an alternate configuration for you. Hope it helps

Answer 2

From the docs:

Each build step is run with its container attached to a local Docker network named cloudbuild. This allows build steps to communicate with each other and share data.

You can use docker compose and using cloudbuild network, for example:

#docker-compose.yml
app-workspace:
  ...
  network_mode: cloudbuild
db-mysql:
  ...
  network_mode: cloudbuild
...
networks:
  default:
    external:
      name: cloudbuild

Or if you are using docker run, add option --network cloudbuild.

After that, you can communicate to other services you defined in the previous step as you expect. For example:

#steps
- id: 'Ping to other container'
  name: gcr.io/cloud-builders/curl
  args: ["app-workspace:your-service-port"]

Hope this helps.