Goland modules with private gitlab (ssh)

Question

Good day! I try to migrate from glide to go modules (private gitlab repos) and checkout code via ssh

I have a simple project with an import from private gitlab repo.

go.mod looks like:

module my.private.package/modtest

go 1.12

require my.private.package/statistics v1.0.0

when I try to build my app or run test I get:

go: my.private.package/[email protected]: unrecognized import path "my.private.package/statistics" (parse https://my.private.package/statistics?go-get=1: no go-import meta tags ())
go: error loading module requirements

I Tried to add settings to git config:

[url "ssh://[email protected]:9999"]
        insteadOf = https://my.private.package

But still getting this error.

Is there any way to make it work? Thank you.

Answer 1

I've dealt with Go modules and a private GitLab before. Our private GitLab has groups and subgroups. The piece you are likely missing is ~/.netrc and you may have an improper global git configuration.

I've made a GitHub gist for this. You can find it here: https://gist.github.com/MicahParks/1ba2b19c39d1e5fccc3e892837b10e21

You can find the gist pasted below:

Problem

The go command line tool needs to be able to fetch dependencies from your private GitLab, but authenticaiton is required.

This assumes your private GitLab is hosted at privategitlab.company.com.

Environment variables

The following environment variables are recommended:

export GO111MODULE=on
export GOPRIVATE=privategitlab.company.com

The above lines might fit best in your shell startup, like a ~/.bashrc.

Explanation

GO111MODULE=on tells Golang command line tools you are using modules. I have not tested this with projects not using Golang modules on a private GitLab.

GOPRIVATE=privategitlab.company.com tells Golang command line tools to not use public internet resources for the hostnames listed (like the public module proxy).

Get a personal access token from your private GitLab

To future proof these instructions, please follow this guide from the GitLab docs. I know that the read_api scope is required for Golang command line tools to work, and I may suspect read_repository as well, but have not confirmed this.

Set up the ~/.netrc

In order for the Golang command line tools to authenticate to GitLab, a ~/.netrc file is best to use.

To create the file if it does not exist, run the following commands:

touch ~/.netrc
chmod 600 ~/.netrc

Now edit the contents of the file to match the following:

machine privategitlab.company.com login USERNAME_HERE password TOKEN_HERE

Where USERNAME_HERE is replaced with your GitLab username and TOKEN_HERE is replaced with the access token aquired in the previous section.

Common mistakes

Do not set up a global git configuration with something along the lines of this:

git config --global url."[email protected]:".insteadOf "https://privategitlab.company.com"

I beleive at the time of writing this, the SSH git is not fully supported by Golang command line tools and this may cause conflicts with the ~/.netrc.

Bonus: SSH config file

For regular use of the git tool, not the Golang command line tools, it's convient to have a ~/.ssh/config file set up. In order to do this, run the following commands:

mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/config
chmod 600 ~/.ssh/config

Please note the permissions on the files and directory above are essentail for SSH to work in it's default configuration on most Linux systems.

Then, edit the ~/.ssh/config file to match the following:

Host privategitlab.company.com
  Hostname privategitlab.company.com
  User USERNAME_HERE
  IdentityFile ~/.ssh/id_rsa

Please note the spacing in the above file matters and will invalidate the file if it is incorrect.

Where USERNAME_HERE is your GitLab username and ~/.ssh/id_rsa is the path to your SSH private key in your file system. You've already uploaded its public key to GitLab. Here are some instructions.

Answer 2

What version of Go are you using? If it's Go 1.13 or later, the default is to download modules through proxy.golang.org. You can change that for a particular set of packages using the GOPRIVATE environment variable.

Here is a quote from go help module-private, which I highly recommend reading in full. Once a module is fetched directly, it should use the same git/ssh logic as before modules.

The GOPRIVATE environment variable controls which modules the go command considers to be private (not available publicly) and should therefore not use the proxy or checksum database. The variable is a comma-separated list of glob patterns (in the syntax of Go's path.Match) of module path prefixes. For example,

GOPRIVATE=*.corp.example.com,rsc.io/private

causes the go command to treat as private any module with a path prefix matching either pattern, including git.corp.example.com/xyzzy, rsc.io/private, and rsc.io/private/quux.