Checkout git submodule from azure pipeline

Question

I am currently working on an azure pipeline. Within my main github repo (repo A), I have another github repo added as a sub module. (repo B)

My goal is to checkout the sub-module at the start of the pipeline with the following YAML:

stages:
- stage: checkout
  jobs:
  - job: checkout
    steps:
      - checkout: self
        submodules: true
        persistCredentials: true

This then attempts to checkout the sub-module, but ends with the following error:

Cloning into '/home/vsts/work/1/s/devops-scripting'...
fatal: could not read Username for 'https://github.com': terminal prompts disabled
fatal: clone of 'https://github.com/sourcerepo/devops-scripting.git' into submodule path '/home/vsts/work/1/s/devops-scripting' failed

It seems to be an issue with using an incorrect user/password - if i was pushing i could simply use supply user/pass parameters, however this doesn't seem to work for checking out.

How can i update a submodule via an azure pipeline?

Answer 1

Checkout git submodule from azure pipeline

If the github repo (repo A) and sub module (repo B) are in the same GitHub organization, then the token stored in the GitHub service connection is used to access the sources.

If not, you can instead use a custom script step to fetch submodules. First, get a personal access token (PAT) and prefix it with pat:. Next, base64-encode this prefixed string to create a basic auth token. Finally, add this script to your pipeline:

git -c http.https://<url of submodule repository>.extraheader="AUTHORIZATION: basic <BASE64_ENCODED_TOKEN_DESCRIBED_ABOVE>" submodule update --init --recursive

Be sure to replace "<BASIC_AUTH_TOKEN>" with your Base64-encoded token.

Use a secret variable in your project or build pipeline to store the basic auth token that you generated. Use that variable to populate the secret in the above Git command.

Another workaround, Use custom script to reuse access token for submodule sync:

steps:
- checkout: self
  submodules: false
  persistCredentials : true

- powershell: |
    $header = "AUTHORIZATION: bearer $(System.AccessToken)"
    git -c http.extraheader="$header" submodule sync
    git -c http.extraheader="$header" submodule update --init --force --depth=1

Check more info from this post.

Answer 2

I had the same issue. I tried with this in my YAML pipeline:

steps:
- checkout: self
  submodules: true
  persistCredentials : true

But in my Azure DevOps project, I also needed to go to: 1) Project Settings, 2) Settings. There, I disabled the option:

• Protect access to repositories in YAML pipelines
  Apply checks and approvals when accessing repositories from YAML pipelines. Also, generate a job access token that is scoped to repositories that are explicitly referenced in the YAML pipeline.

Then it worked.