Create Github mirror in Artifactory

Question

How to create a github mirror in Artifactory. We need to install node-sass. We have a remote npm repository which downloads node-sass but at the end it needs some binding.node file which comes from github. Here the build fails due to proxy.

We created a remote repository to https://npm.taobao.org/mirrors/node-sass and told our build to check our remote repo for node-sass:

SASS_BINARY_SITE=https://xx/artifactory/node-sass/

This works fine. But now we are mirroring a mirror. We want to mirror the github releases immediatly but this does not work: We tried https://github.com/sass/node-sass/releases/ and https://github.com/sass/node-sass/releases/download/ but both did not work.

I read about the VCS repo but this seems to really our solution I'm afraid.

Answer 1

I spent all morning fighting this and I finally got it working. The root cause is that, by default, Artifactory tries to do an HTTP HEAD request on an artifact, which does not work on GitHub CDN S3 buckets: see this issue: RTFACT-11923

As of Artifactory 5.5.2, there is now an option in the advanced configuration of "generic" repos to ignore HTTP HEAD requests. If this is checked, a generic repo can mirror GitHub releases.

I've successfully set up a mirror with the following configuration:

1. Create a custom layout for GitHub releases:

   [orgPath]/releases/download/[baseRev]/[module].[ext]
   

2. Create a new remote repository with the "generic" type.

3. Set the URL of the new repository to https://github.com/
4. Set the repository layout and remote repository layout to the custom layout created for GitHub
5. Uncheck "List Remote Folder Items"
6. On the "Advanced" tab, under the "Others" section, check "Bypass HEAD Requests"

Once you save the repository, try something like:

http://artifactory.local/artifactory/github-releases/sass/node-sass/releases/download/v4.9.0/win32-ia32-11_binding.node

Which should download that artifact and cache it for future use. You can then set SASS_BINARY_SITE to "http://artifactory.local/artifactory/github-releases/sass/node-sass/releases/download" and that should force npm install to use the cached/proxied version in Artifactory.

In theory this will work to cache any releases on GitHub, but I've not tried it for anything else yet.