Elastic Beanstalk: log task customization on Amazon Linux 2 platforms

Question

I'm wondering how to do log task customization in the new Elastic Beanstalk platform (the one based on Amazon Linux 2). Specifically, I'm comparing:

• Old: Single-container Docker running on 64bit Amazon Linux/2.14.3
• New: Single-container Docker running on 64bit Amazon Linux 2/3.0.0

(My question actually has nothing to do with Docker as such, I'm speculating the problem exist for any of the new Elastic Beanstalk platforms).

Previously I could follow Amazon's recipe, meaning put a file into /opt/elasticbeanstalk/tasks/bundlelogs.d/ and it would then be acted upon. This is no longer true.

Has this changed? I can't find it documented. Anyone been successful in doing log task customization on the newer Elastic Beanstalk platform? If so, how?

Minimal working example

I've created a minimal working example and deployed on both platforms.

Dockerfile:

FROM ubuntu
COPY daemon-run.sh /daemon-run.sh
RUN chmod +x /daemon-run.sh
EXPOSE 80
ENTRYPOINT ["/daemon-run.sh"]

Dockerrun.aws.json:

{
  "AWSEBDockerrunVersion": "1",
  "Logging": "/var/mydaemon"
}

daemon-run.sh:

#!/bin/bash
echo "Starting daemon" # output to stdout
mkdir -p /var/mydaemon/deeperlogs
while true; do
   echo "$(date '+%Y-%m-%dT%H:%M:%S%:z')  Hello World" >> /var/mydaemon/deeperlogs/app_$$.log
   sleep 5
done

.ebextensions/mydaemon-logfiles.config:

files: 
  "/opt/elasticbeanstalk/tasks/bundlelogs.d/mydaemon-logs.conf" :
    mode: "000755"
    owner: root
    group: root
    content: |
       /var/log/eb-docker/containers/eb-current-app/deeperlogs/*.log

If I do "Full Logs" action on the old platform I would get a ZIP with my deeperlogs included inside var/log/eb-docker/containers/eb-current-app. On the new platform I don't.

Investigation

If you look on the disk you'll see that the new Elastic Beanstalk doesn't have a /opt/elasticbeanstalk/tasks folder at all, unlike the old one. Hmm.

Answer 1

On Amazon Linux 2 the folder is:

/opt/elasticbeanstalk/config/private/logtasks/bundle

The .ebextensions/mydaemon-logfiles.config should be:

files: 
  "/opt/elasticbeanstalk/config/private/logtasks/bundle/mydaemon-logs.conf":
    mode: "000644"
    owner: root
    group: root
    content: |
       /var/mydaemon/deeperlogs/*.log

container_commands:
  append_deeperlogs_to_applogs:
    command: echo -e "\n/var/log/eb-docker/containers/eb-current-app/deeperlogs/*" >> /opt/elasticbeanstalk/config/private/logtasks/bundle/applogs

The mydaemon-logfiles.config also adds deeperlogs into applogs file. Without it deeperlogs will not be included in the download log zip bundle. Which is intresting, because the folder will be in the correct location, i.e., /var/log/eb-docker/containers/eb-current-app/deeperlogs/. But without being explicitly listed in applogs, it will be skipped when zip bundle is being generated.

I tested it with single docker environment (3.0.1).

The full log bundle successful contained deeperlogs with correct log data:

Hope that this will help. I haven't found any references for that. AWS documentaiton does not document this, as it is mostly based on Amazon Linux 1, not Amazon Linux 2.