No Logging on Azure DevOps Pipeline

Question

Update:

Is it possible to add or change a command that executes a pipeline on Azure DevOps?

---

Running my program locally on Visual Studio Code, I do get outputs.

However, running my GitHub origin branch on Azure DevOps does not yield any output.

I followed a Stack Overflow answer, which references this solution to a GitHub Issue.

I have implemented the below, but Azure's Raw Logs return blank on my Python logging.

test_logging.py:

import logging

filename = "my.log"

global logger
logger = logging.getLogger()
logger.setLevel(logging.INFO)
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
open(filename, "w").close()  # empty logs
fileHandler = logging.FileHandler(filename)
fileHandler.setFormatter(formatter)
fileHandler.setLevel(logging.INFO)
logger.addHandler(fileHandler)

logger.error('TEST')

# fetch logs
with open(filename, "r") as fileHandler:
    logs = [log.rstrip() for log in fileHandler.readlines()]
open(filename, "w").close()  # empty logs
print('logs = ', logs)

>>> logs = []

host.json:

{
  "version":  "2.0",
  "logging": {
    "fileLoggingMode": "always",
    "logLevel": {
      "default": "Debug"
    }
  } 
}

---

I then tried this alternative host.json from post:

"logging": {
    "fileLoggingMode": "debugOnly",
    "logLevel": {
        "default": "None",
        "Host.Results": "Information",
        "Function": "Information",
        "Host.Aggregator": "Information"
    },
    "applicationInsights": {
        "samplingSettings": {
            "isEnabled": false,
            "maxTelemetryItemsPerSecond": 5
        }
    }
}

azure-pipeline-ontology_tagger.yaml

# ##########
# A build run against multiple Python targets
# ##########

resources:
- repo: self

variables:
  tag: '$(Build.SourceBranchName)-$(Build.BuildNumber)'
  imageName: '$(Build.Repository.Name)-ontology_tagger'
  artifactFeed: grandproject/private-sources
  repositoryUrl: private-sources
  packageDirectory: workers/ontology_tagger

trigger:
  batch: true
  branches:
    include:
    - master
    - development
    - releases/*
  paths:
    include:
    - "workers/ontology_tagger"
    exclude:
    - "workers"
    - "*.md"
pr:
  branches:
    include:
    - master
    - development
    - releases/*
  paths:
    include:
    - "workers/ontology_tagger"
    exclude:
    - "workers"
    - "*.md"

stages:
- stage: BuildWP
  displayName: Build Workers python package
  jobs:

  - job: Build
    displayName: Build Worker python image

    pool:
      name: EKS-grandproject-dev

    steps:
    - bash: env

    - task: PipAuthenticate@0
      displayName: Authenticate with artifact feed
      inputs:
        artifactFeeds: $(artifactFeed)

    - task: TwineAuthenticate@1
      displayName: Authenticate with artifact feed
      inputs:
        artifactFeed: $(artifactFeed)

    - bash: echo "##vso[task.setvariable variable=POETRY_HTTP_BASIC_AZURE_PASSWORD;isOutput=true]$(echo $PIP_EXTRA_INDEX_URL | sed -r 's|https://(.+):(.+)@.*|\2|')"
      name: "PIPAUTH"

    - task: Bash@3
      displayName: Test worker
      inputs:
        targetType: 'inline'
        workingDirectory: '$(packageDirectory)'
        script: |
          docker build . --progress plain --pull --target test \
          --build-arg POETRY_HTTP_BASIC_AZURE_PASSWORD=${PIPAUTH_POETRY_HTTP_BASIC_AZURE_PASSWORD} \
          --build-arg ATLASSIAN_TOKEN=$(ATLASSIAN_TOKEN)
    - task: Bash@3
      displayName: Build and publish package
      inputs:
        targetType: 'inline'
        workingDirectory: '$(packageDirectory)'
        script: |
          set -e
          cp $(PYPIRC_PATH) ./
          docker build . --target package --progress plain  --build-arg REPO=$(repositoryUrl)
    - task: Bash@3
      displayName: Build docker image
      inputs:
        targetType: 'inline'
        workingDirectory: '$(packageDirectory)'
        script: |
          docker build . --tag '$(imageName):$(tag)' --progress plain --pull --target production \
          --build-arg POETRY_HTTP_BASIC_AZURE_PASSWORD=${PIPAUTH_POETRY_HTTP_BASIC_AZURE_PASSWORD} \
          --label com.azure.dev.image.build.sourceversion=$(Build.SourceVersion) \
          --label com.azure.dev.image.build.sourcebranchname=$(Build.SourceBranchName) \
          --label com.azure.dev.image.build.buildnumber=$(Build.BuildNumber)
    - task: ECRPushImage@1
      displayName: Push image with 'latest' tag
      condition: and(succeeded(),eq(variables['Build.SourceBranchName'], 'master'))
      inputs:
        awsCredentials: 'dev-azure-devops'
        regionName: 'eu-central-1'
        imageSource: 'imagename'
        sourceImageName: $(imageName)
        sourceImageTag: $(tag)
        repositoryName: $(imageName)
        pushTag: 'latest'
        autoCreateRepository: true

    - task: ECRPushImage@1
      displayName: Push image with branch name tag
      condition: and(succeeded(),ne(variables['Build.SourceBranchName'], 'merge'))
      inputs:
        awsCredentials: 'iotahoe-dev-azure-devops'
        regionName: 'eu-central-1'
        imageSource: 'imagename'
        sourceImageName: $(imageName)
        sourceImageTag: $(tag)
        repositoryName: $(imageName)
        pushTag: '$(Build.SourceBranchName)'
        autoCreateRepository: true

    - task: ECRPushImage@1
      displayName: Push image with uniq tag
      condition: and(succeeded(),ne(variables['Build.SourceBranchName'], 'merge'))
      inputs:
        awsCredentials: 'dev-azure-devops'
        regionName: 'eu-central-1'
        imageSource: 'imagename'
        sourceImageName: $(imageName)
        sourceImageTag: $(tag)
        repositoryName: $(imageName)
        pushTag: $(tag)
        autoCreateRepository: true
        outputVariable: 'ECR_PUSHED_IMAGE_NAME'

Please let me know if there is anything else I should provide.

Answer 1

I think you have fundamentally mixed up some things here: the links you have provided and are following provide guidance on setting up logging in Azure Functions. However, you appear to be talking about logging in Azure Pipelines, which is an entirely different thing. So just to be clear:

Azure Pipelines run the build and deployment jobs that deploy the code you might have on your GitHub repository to Azure Functions. Pipelines are executed in Azure Pipelines agents, that can be either Microsoft- or Self-hosted. If we assume that you are executing your pipelines with Microsoft-Hosted agents, you should not assume that these agents have any capabilities that Azure Functions might have (nor that you should execute code aimed for Azure Functions in the first place). If you want do execute python code in your pipeline, you should first start looking at what python-related capabilities the hosted agents have pre-installed and work from there: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops&tabs=yaml

If you want to log something about pipeline run, you should first check the "Enable system diagnostics" option when queuing pipeline manually. For implementing more logging by yourself, do check: https://learn.microsoft.com/en-us/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=bash

For logging in Azure Functions you might want to start here: https://learn.microsoft.com/en-us/azure/azure-functions/functions-monitoring , but that would be an entirely different topic than logging in Azure Pipelines.