Unable to push nuGet packages to GitLab with dotnet CLI due to Unauthorized error

Question

GitLab now supports nuget public and private feed repository. I've got a public project (e.g: https://gitlab.com/sunnyatticsoftware/sasw-test-support) I create an access token for my user with api and write_repository (e.g: AAABBBCCCDDD)

I create a group variable in my CI/CD: SASW_API_ACCESS_TOKEN: AAABBBCCCDDD. All normal.

Then I create the multi stage CI/CD script to build, pack and publish. When attempting to publish the nuGet package with the following: dotnet nuget push **/*.nupkg --source https://gitlab.com/api/v4/projects/17141695/packages/nuget/index.json --api-key AAABBBCCCDDD --skip-duplicate

I get the error:

info : Pushing Sasw.TestSupport.2.0.2.nupkg to 'https://gitlab.com/api/v4/projects/17141695/packages/nuget'...
info :   PUT https://gitlab.com/api/v4/projects/17141695/packages/nuget/
info :   Unauthorized https://gitlab.com/api/v4/projects/17141695/packages/nuget/ 397ms
error: Response status code does not indicate success: 401 (Unauthorized).
ERROR: Job failed: exit code 1

The documentation doesn't mention anything special, but I notice that when using the (legacy?) nuget CLI it passes a username. Dotnet CLI, however, doesn't support username, just API KEY.

Any idea why this is not working?

This is my CI/CD script:

variables:
  GITLAB_RUNNER_DOTNET_CORE: mcr.microsoft.com/dotnet/core/sdk:3.1
  NUGET_REPOSITORY: https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/nuget/index.json
  NUGET_API_KEY: $SASW_API_ACCESS_TOKEN
  NUGET_FOLDER_NAME: nupkgs
  NUGET_VERSION_SUFFIX: $SASW_PRERELEASE_SUFFIX

stages:
  - build
  - pack
  - release
  
#Docker image
image: $GITLAB_RUNNER_DOTNET_CORE

#Jobs
ci:
  stage: build
  script:
    - dotnet restore --no-cache --force
    - dotnet build --configuration Release --no-restore
    #- dotnet vstest test/*UnitTests/bin/Release/**/*UnitTests.dll
    #- dotnet vstest test/*IntegrationTests/bin/Release/**/*IntegrationTests.dll
    
pack-prerelease:
  stage: pack
  script:
    - dotnet pack *.sln --configuration Release --output $NUGET_FOLDER_NAME --version-suffix $NUGET_VERSION_SUFFIX --include-symbols -p:SymbolPackageFormat=snupkg
  artifacts:
    paths:
    - $NUGET_FOLDER_NAME
    expire_in: 1 week
  except:
    - master

pack-release:
  stage: pack
  script:
    - dotnet pack *.sln --configuration Release --output $NUGET_FOLDER_NAME
  artifacts:
    paths:
    - $NUGET_FOLDER_NAME
    expire_in: 1 week
  only:
    - master

publish-nuget:
  stage: release
  script:
    - dotnet nuget push **/*.nupkg --source $NUGET_REPOSITORY --api-key $NUGET_API_KEY --skip-duplicate

PS: The project is public, so in case it's needed have a look at: https://gitlab.com/sunnyatticsoftware/sasw-test-support/-/jobs/451080235

---

UPDATE 1: Further verbosity from my local linux console

$ dotnet nuget -v Debug push **/*.nupkg --source https://gitlab.com/api/v4/projects/17141695/packages/nuget/index.json --api-key cBwt5_hidden_ --skip-duplicate
trace: NuGet Command Line Version: 5.4.0.2
info : Pushing Sasw.TestSupport.2.0.2.nupkg to 'https://gitlab.com/api/v4/projects/17141695/packages/nuget'...
info :   PUT https://gitlab.com/api/v4/projects/17141695/packages/nuget/
info :   Unauthorized https://gitlab.com/api/v4/projects/17141695/packages/nuget/ 1159ms
error: Response status code does not indicate success: 401 (Unauthorized).
trace: System.AggregateException: One or more errors occurred. (Response status code does not indicate success: 401 (Unauthorized).)
trace:  ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 401 (Unauthorized).
trace:    at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
trace:    at NuGet.Protocol.Core.Types.PackageUpdateResource.EnsureSuccessStatusCode(HttpResponseMessage response, Nullable`1 codeNotToThrow, ILogger logger)
trace:    at NuGet.Protocol.Core.Types.PackageUpdateResource.<>c__DisplayClass23_0.<PushPackageToServer>b__0(HttpResponseMessage response)
trace:    at NuGet.Protocol.HttpSource.ProcessResponseAsync[T](HttpSourceRequest request, Func`2 processAsync, SourceCacheContext cacheContext, ILogger log, CancellationToken token)
trace:    at NuGet.Protocol.Core.Types.PackageUpdateResource.PushPackageToServer(String source, String apiKey, String pathToPackage, Int64 packageSize, Boolean noServiceEndpoint, Boolean skipDuplicate, TimeSpan requestTimeout, ILogger logger, CancellationToken token)
trace:    at NuGet.Protocol.Core.Types.PackageUpdateResource.PushPackageCore(String source, String apiKey, String packageToPush, Boolean noServiceEndpoint, Boolean skipDuplicate, TimeSpan requestTimeout, ILogger log, CancellationToken token)
trace:    at NuGet.Protocol.Core.Types.PackageUpdateResource.PushPackage(String packagePath, String source, String apiKey, Boolean noServiceEndpoint, Boolean skipDuplicate, TimeSpan requestTimeout, ILogger log, CancellationToken token, Boolean isSnupkgPush)
trace:    at NuGet.Protocol.Core.Types.PackageUpdateResource.Push(String packagePath, String symbolSource, Int32 timeoutInSecond, Boolean disableBuffering, Func`2 getApiKey, Func`2 getSymbolApiKey, Boolean noServiceEndpoint, Boolean skipDuplicate, SymbolPackageUpdateResourceV3 symbolPackageUpdateResource, ILogger log)
trace:    at NuGet.Commands.PushRunner.Run(ISettings settings, IPackageSourceProvider sourceProvider, String packagePath, String source, String apiKey, String symbolSource, String symbolApiKey, Int32 timeoutSeconds, Boolean disableBuffering, Boolean noSymbols, Boolean noServiceEndpoint, Boolean skipDuplicate, ILogger logger)
trace:    at NuGet.CommandLine.XPlat.PushCommand.<>c__DisplayClass0_1.<<Register>b__1>d.MoveNext()
trace:    --- End of inner exception stack trace ---
trace:    at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
trace:    at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
trace:    at System.Threading.Tasks.Task`1.get_Result()
trace:    at Microsoft.Extensions.CommandLineUtils.CommandLineApplication.<>c__DisplayClass56_0.<OnExecute>b__0()
trace:    at Microsoft.Extensions.CommandLineUtils.CommandLineApplication.Execute(String[] args)
trace:    at NuGet.CommandLine.XPlat.Program.MainInternal(String[] args, CommandOutputLogger log)

UPDATE 2: Alexey's answer is the proper one and up to date as per April 2021. GitLab has improved Nuget support in the last year and now it's possible to easily push packages to the repo package registry and have readacces on Nuget feed at project level or group level using deploy tokens.

Answer 1

Pushing NuGet packages built from the current repository is quite easy, following the documentation. You do not need a NuGet.config file to push packages, as the credentials can be specified for the dotnet push command. You also don't need to keep the credentials in the CI file, as the CI variables contain all the necessary temporary credentials to push packages to the project package registry.

This is a fragment from my working .gitlab-ci.yml file, which I literally copy-pasted from the docs. All the necessary information comes from CI variables, so this snippet is totally reusable.

nuget:
  stage: deploy
  image: mcr.microsoft.com/dotnet/sdk:5.0-buster-slim
  script:
    - dotnet pack -c Release -o $PWD/nuget
    - dotnet nuget add source "$CI_SERVER_URL/api/v4/projects/$CI_PROJECT_ID/packages/nuget/index.json" --name gitlab --username gitlab-ci-token --password $CI_JOB_TOKEN --store-password-in-clear-text
    - dotnet nuget push "$PWD/nuget/*.nupkg" --source gitlab
  only:
    - master
    - tags

Concerning the question about installing the package, you would, indeed, need a token. But it is not your personal access token. All you need really to let people install and restore from your feed is the repository deploy token. It doesn't have to be defined on the project level if you have multiple projects in a group, which contains several projects that have NuGet packages published. You can also create a deploy token for the whole group. The deploy token you create for this purpose only needs to have the read_package_registry permissions and it doesn't give any other rights to the users who get this token.

After creating a deploy token, the token name is used as the username, and the token itself is the password. You put both of those to the NuGet.config file, where you have your project or group feed listed.

For example:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <packageSources>
        <clear />
        <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
        <add key="myfeed" value="https://gitlab.mydomain.dev/api/v4/groups/19/-/packages/nuget/index.json" />
    </packageSources>
    <packageSourceCredentials>
        <myfeed>
            <add key="Username" value="gitlab+deploy-token-14" />
            <add key="ClearTextPassword" value="thetokenvalue" />
        </myfeed>
    </packageSourceCredentials>
</configuration>

Answer 2

I managed to push the nuget in a way that is not intuitive at all. I had to create a NuGet.Config with credential details. This is something I don't like because passing the api key to the dotnet nuget push command should be enough.

In any case, these are my steps: dotnet new nugetconfig --force to create a NuGet.Config in the root folder for my solution Then edit it to add a new source (which is linked to a specific project. Annoying..)

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <!--To inherit the global NuGet package sources remove the <clear/> line below -->
    <clear />
    <add key="nuget" value="https://api.nuget.org/v3/index.json" />
    <!--The feed URL line below. The key can be anything but it must match the section in packageSourceCredentials -->
    <add key="gitlab" value="https://gitlab.com/api/v4/projects/17141695/packages/nuget/index.json" />
  </packageSources>
  <packageSourceCredentials>
    <gitlab>
      <add key="Username" value="diegosasw" /> <!--My gitlab username-->
      <add key="ClearTextPassword" value="cBwt5HPD_hidden_" /> <!--My gitlab access key-->
    </gitlab>
  </packageSourceCredentials>
</configuration>

And then the following command would work

$ dotnet nuget push **/*.nupkg --source https://gitlab.com/api/v4/projects/17141695/packages/nuget/index.json --skip-duplicate
warn : No API Key was provided and no API Key could be found for 'https://gitlab.com/api/v4/projects/17141695/packages/nuget'. To save an API Key for a source use the 'setApiKey' command.
info : Pushing Sasw.TestSupport.2.0.3.nupkg to 'https://gitlab.com/api/v4/projects/17141695/packages/nuget'...
info :   PUT https://gitlab.com/api/v4/projects/17141695/packages/nuget/
info :   Created https://gitlab.com/api/v4/projects/17141695/packages/nuget/ 1936ms
info : Your package was pushed.

Now I can see the package available but without metadata! (not sure why). So if I add the very same NuGet.Config to any solution folder and try to install the package it works.

$ dotnet add Foo/Foo.csproj package Sasw.TestSupport
  Writing C:\Users\dmsanz\AppData\Local\Temp\tmpF8D4.tmp
info : Adding PackageReference for package 'Sasw.TestSupport' into project 'Foo/Foo.csproj'.
info : Restoring packages for D:\src\sasw\sasw-test-support\Foo\Foo.csproj...
info :   CACHE https://api.nuget.org/v3-flatcontainer/sasw.testsupport/index.json
info :   CACHE https://gitlab.com/api/v4/projects/17141695/packages/nuget/download/sasw.testsupport/index.json
info : Package 'Sasw.TestSupport' is compatible with all the specified frameworks in project 'Foo/Foo.csproj'.
info : PackageReference for package 'Sasw.TestSupport' version '2.0.3' added to file 'D:\src\sasw\sasw-test-support\Foo\Foo.csproj'.
info : Committing restore...
info : Writing assets file to disk. Path: D:\src\sasw\sasw-test-support\Foo\obj\project.assets.json
log  : Restore completed in 1.13 sec for D:\src\sasw\sasw-test-support\Foo\Foo.csproj.

It works... but I don't like it at all.

• First of all, I cannot have my NuGet.Config generic, they need to have a URL with a specific project in it. It'd be nice at least to have a nuGet feed pointing to a group rather than a project, so that the NuGet.Config can be shared between different projects.
• It's not intuitive that having an api-key to run a command, I need to delegate on a NuGet.Config to provide authentication.
• When I am a developer creating a project, I like to ship in a NuGet.Config with the details of the NuGet feed that the project needs to consume packages. So if any credentials required at all, those would be read-only credentials for other people to be able to do a dotnet restore without issues to compile the solution. But with this approach in GitLab, I need to have a NuGet.Config in my solution folder with sensitive data like an api key with write permissions just to be able to push the package from the CI/CD pipeline. That should be unacceptable. Am I forced to do some tricks in my CI/CD pipeline to create a brand new NuGet.Config file just to place there the sensitive credentials in order to publish a nuget package just because GitLab does not support a nice and clean way to push packages to a repo with a simple command that does not need NuGet.Config at all?

I hope to be wrong. Please if anybody has a better solution I'll gladly mark it as accepted answer.

Answer 3

As always, I read all the answers above way too fast. So, in hopes of helping someone else:

The solution for me was to use DEPLOY TOKENS instead of "normal" access tokens.

Read more here: https://docs.gitlab.com/ee/user/project/deploy_tokens/