Sed failing to replace string in quotes during Gitlab CI

Question

I am trying to dynamically set the image name and tag for AWS Elastic Beanstalk in my Dockerrun.aws.json file:

"Image": {
  "Name": "IMAGETAG",
  "Update": "true"
}

with the following sed command as a script in my GitLab CI file:

sed -i.bak "s|IMAGETAG|$CONTAINER_TEST_IMAGE|" Dockerrun.aws.json && rm Dockerrun.aws.json.bak; eb deploy Production

Where $CONTAINER_TEST_IMAGE is a verified good environment variable (tested by doing echo $CONTAINER_TEST_IMAGE as a script). $CONTAINER_TEST_IMAGE contains the structure of the following content (where ... is the full id):

gitlab.company.com:123/my-group/my-project:core_7de09851...8f_testing

The problem I am facing is that sed does not work during the CI pipeline. I am failing to understand why considering if I set the environment variable locally and run the same command, it will successfully replace the value of Name to the same structure URL. This testing was done on a Macbook.

I know that it is not updating the file because the Gitlab CI log reports

WARN: Failed to pull Docker image IMAGETAG:latest, retrying...

I've tried a few things that did not work:

• Running the sed and eb deploy commands as separate scripts (two different lines in the CI file)
• Switch the variable that I am seeking to replace in Dockerrun.aws.json to <IMAGE>
• While it was at <IMAGE>, running sed -i='' "s|<IMAGE>|$CONTAINER_RELEASE_IMAGE|" Dockerrun.aws.json instead of doing the .bak and then rm'ing it (I read somewhere that sed has inconsistencies on OSX with the -i='' version)

Does anyone have any thoughts on what the issue might be and how it can be resolved?

Answer 1

There were two aspects of this that were going wrong:

1. The sed command was not executing correctly on the runner, but was working locally
2. eb deploy was ignoring the updated file

For part 1, he working sed command is:

sed -ri "s|\"IMAGETAG\"|\"$1\"|" Dockerrun.aws.json

where the line in Dockerrun.aws.json is "Name": "IMAGETAG",. sed still confuses me here so I can't explain why this one works vs the original command.

For part 2, apparently eb deploy will always look at the latest commit if it can, rather than the current working directory. Makes sense, I guess. To get around this, run the command as eb deploy --staged. You can read more about this flag on AWS's site.

Also, note that my .gitlab-ci.yml simply calls a script to run all of this rather than doing it there.

- chmod +x ./scripts/ebdeploy.sh
- ./scripts/ebdeploy.sh $CONTAINER_TEST_IMAGE