python flask application to track APM logs with Datadog on Azure Linux containers using ACR

Question

My current organization is migrating to DataDog for Application Performance Monitoring. I am deploying a Python Flask web application using docker to Azure Container Registry. After the deployment to Azure the app should be listed/available on Datadog portal.

Please note I just started learning Docker containers. There is a high chance I could do completely wrong. Please bear with me

Steps followed

Option 1: Create a docker container on local machine and push to ACR

1. Added dd-trace python library to the docker image

2. Added dd-trace run command the docker file

3. build the image

4. run the container on local

   Getting OSError: [Errno 99] Cannot assign requested address

   FROM python:3.7
   ENV VIRTUAL_ENV=/opt/venv
   RUN python -m venv $VIRTUAL_ENV
   ENV PATH="$VIRTUAL_ENV/bin:$PATH"
   
   ENV DD_API_KEY=apikeyfromdatadoghq
   ENV DD_ENV=safhire-dev
   ENV DD_LOGS_ENABLED=true
   ENV DD_LOGS_INJECTION=true
   ENV DD_SERVICE=dev-az1-pythonbusinessservice
   ENV DD_TAGS=products:myprojects
   ENV DD_TRACE_DEBUG=true
   ENV DD_TRACE_ENABLED=true
   ENV DOCKER_ENABLE_CI=true
   
   COPY /app /app
   
   
   COPY requirements.txt /
   RUN pip install --no-cache-dir -U pip
   RUN pip install --no-cache-dir -r /requirements.txt
   CMD ddtrace-run python app/main.py runserver 127.0.0.1:3000
   

Option 2: Forward logs to Azure Blob Storage but a heavy process

1. Deploy Python using Code base Linux
2. Forward the logs to a Azure Blob storage
3. Create a BlobTrigger Azure Function to forward the logs to DataDogAPI
4. I believe with this approach we can not capture APM logs but, we can capture application and console logs

Option 3: using Serilog but, my organization does not want to use third party logging framework, we have our own logging framework

Any help is highly appreciated, I am looking for a solution using Option 1. I went through the Microsoft articles, Datadog documentation but, no luck.

I setup app registrations, Manage reader permissions on Subscription, created ClientID and app secrets on Azure portal. none of them helped

Could you confirm whether is there a way to collect the APM logs on datadog with out installing agent on Azure.

Thank you in advance.

Answer 1

After few days of research and follow up with datadog support team, I am able to get the APM logs on datadog portal.

Below is my docker-compose.yml file configuration, I believe it helps someone in future

version: "3"
services:
  web:
    build: web
    command: ddtrace-run python standalone_api.py 
    volumes:
      - .:/usr/src/app
    depends_on: 
      datadog-agent:
        condition: service_healthy         
    image: pythonbusinessservice:ICDNew
    ports: 
     - 5000:5000
    environment:     
    - DATADOG_HOST=datadog-agent
    - DD_TRACE_AGENT_PORT=8126
    - DD_AGENT_HOST=datadog-agent
  datadog-agent:
    build: datadog
    image: gcr.io/datadoghq/agent:latest
    ports: 
     - 8126:8126          
    environment: 
     - DD_API_KEY=9e3rfg*****************adf3
     - DD_SITE=datadoghq.com
     - DD_HOSTNAME=pythonbusinessservice
     - DD_TAGS=env:dev      
     - DD_APM_ENABLED=true
     - DD_APM_NON_LOCAL_TRAFFIC=true
     - DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true     
     - DD_SERVICE=pythonbusinessservice 
     - DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL=true 
     - DD_CONTAINER_EXCLUDE="name:datadog-agent"      
    volumes:
     - /var/run/docker.sock:/var/run/docker.sock
     - /proc/:/host/proc/:ro
     - /opt/datadog-agent/run:/opt/datadog-agent/run:rw
     - /sys/fs/cgroup:/host/sys/fs/cgroup:ro

The Dockerfile for my python long running application

FROM python:3.7

COPY . /app
WORKDIR /app

RUN pip install -r requirements.txt

CMD ["ddtrace-run python", "/app/standalone_api.py"]

Please note, on the requirements.txt file I have ddtrace package listed