Azure YAML Deployment failing with "Error: Failed to deploy web package to App Service. Conflict (CODE: 409)"

Question

Hoping someone has seen this also. Our last successful build was on Dec 6th 2019. New deployment attempts are failing at the deployment step (Same issue in the Build and Release pipelines).

Error is reported as follows:

Got service connection details for Azure App Service:'OUR SITE'
Package deployment using ZIP Deploy initiated.
##[error]Failed to deploy web package to App Service.
##[error]Error: Error: Failed to deploy web package to App Service. Conflict (CODE: 409)
##[warning]Error: Failed to update deployment history. Error: Bad Request (CODE: 400)
App Service Application URL: OUR SITE

Here is our deployment YAML, sorry been at this for days now :(

# Node.js Express Web App to Linux on Azure
# Build a Node.js Express app and deploy it to Azure as a Linux web app.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/javascript

trigger:
- master

variables:

  # Azure Resource Manager connection created during pipeline creation
  azureSubscription: '[OUR SUB ID]'

  # Web app name
  webAppName: 'website-to-deploy'

  # Environment name
  environmentName: 'web-to-deploy'

  # Agent VM image name
  vmImageName: 'ubuntu-latest'

stages:
- stage: Build
  displayName: Build stage
  jobs:  
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)

    steps:
    - task: NodeTool@0
      inputs:
        versionSpec: '10.x'
      displayName: 'Install Node.js'

    - script: |
        npm install
        npm run build --if-present
        npm run test --if-present
      displayName: 'npm install, build and test'

    - task: ArchiveFiles@2
      displayName: 'Archive files'
      inputs:
        rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
        includeRootFolder: false
        archiveType: zip
        archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
        replaceExistingArchive: true

    - upload: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
      artifact: drop
    - task: AzureRmWebAppDeployment@4
      inputs:
        ConnectionType: 'AzureRM'
        azureSubscription: 'OUR SUBSCRIPTION'
        appType: 'webAppLinux'
        WebAppName: 'site-to-deploy'
        packageForLinux: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
        RuntimeStack: 'NODE|12-lts'
        StartupCommand: 'npm start'
        AppSettings: 'WE DO HAVE SOME SETTINGS'     

Please note: I have tried adding WEBSITE_WEBDEPLOY_USE_SCM = true to the configuration, this has not helped.
There have been no changes to the above build YAML that was successfully working and feeding into a release pipeline

Thanks for any help.

UPDATE : Ticket raised with MS Dev-Ops You can add system.debug: true in your variables section to get far more verbose logging.
This has allowed me to see that the AppService returns the 409 code as it believes there is a deployment already underway. If and when I get a resolution I will edit this post.

Here is the debug info:

##[debug]Encountered a retriable status code: 409. Message: 'Conflict'.
##[debug][POST]https://$web040-glndev-com:***@web040-glndev-com.scm.azurewebsites.net/api/zipdeploy?isAsync=true&deployer=VSTS_ZIP_DEPLOY
##[debug]Could not parse response: {}
##[debug]Response: undefined
##[debug]Encountered a retriable status code: 409. Message: 'Conflict'.
##[debug][POST]https://$web040-glndev-com:***@web040-glndev-com.scm.azurewebsites.net/api/zipdeploy?isAsync=true&deployer=VSTS_ZIP_DEPLOY
##[debug]Could not parse response: {}
##[debug]Response: undefined
##[debug]ZIP Deploy response: {"statusCode":409,"statusMessage":"Conflict","headers":{"transfer-encoding":"chunked","content-type":"text/plain; charset=utf-8","server":"Kestrel","date":"Wed, 15 Jan 2020 17:40:47 GMT","connection":"close"},"body":"There is a deployment currently in progress. Please try again when it completes."}
##[error]Failed to deploy web package to App Service.
##[debug]Processed: ##vso[task.issue type=error;]Failed to deploy web package to App Service.
##[debug]task result: Failed
##[error]Error: Error: Failed to deploy web package to App Service. Conflict (CODE: 409)
##[debug]Processed: ##vso[task.issue type=error;]Error: Error: Failed to deploy web package to App Service. Conflict (CODE: 409)
##[debug]Processed: ##vso[task.complete result=Failed;]Error: Error: Failed to deploy web package to App Service. Conflict (CODE: 409)
##[debug][POST]https://management.azure.com/subscriptions/7a7aad03-79b0-4118-8dd1-8ebd63716c6a/resourceGroups/appsvc_linux_centralus/providers/Microsoft.Web/sites/web040-glndev-com/config/appsettings/list?api-version=2016-08-01
##[debug]Correlation ID from ARM api call response : e4ebaf0f-7ccf-4b7b-ba4d-b70b8fd13bea
##[debug]Application Insights is not configured for the App Service. Skipping adding release annotation.

Ticket is here: 409 Unable to deploy web package

Answer 1

Restarting the App Service solved my problem.

My App Service Plan was already a paid subscription and there was no issue with the capacity. I just restarted the App Service and everything worked fine.

Answer 2

So after a lot of discussion with MS support, Here is the ticket

It transpires that the real issue is that my App Service Plan was at capacity. In development we were using the free service and there are limits. We had hit this limit and the message was really a red herring and not the real problem at all.

Anyway, I upgraded to the first tier of paid subscription and everything is now working.