What permissions does nginx need for the Gitlab-workhorse socket?

Question

I've been trying to get this sorted all day. I'm running an omnibus install of GitLab on Ubuntu 14.04, using an existing nginx install. I can login, create projects, issues, etc. but I cannot push/pull, clone or anything to a project from the outside (not through web gui). One catch...I'm running Plesk 12.5 on this server. Here's some outputs:

Error being reported by nginx:

2016/02/10 16:00:50 [crit] 24866#0: *53 connect() to unix://var/opt/gitlab/gitlab-workhorse/socket 
failed (13: Permission denied) while connecting to upstream, client: XXX.XXX.X.X, 
server: git.example.com, 
request: "GET /namespace/project-name.git/info/refs?service=git-upload-pack HTTP/1.1", 
upstream: "http://unix://var/opt/gitlab/gitlab-workhorse/socket:/namespace/project-name.git/info/refs?service=git-upload-pack", host: "git.example.com"

The permissions on the workhorse socket:

srwxrwxrwx 1 git git 0 Feb 2 18:40 socket

I've tried to change the owner to nginx (www-data) and the group to gitlab-www but no luck. I'm not using any different directories....just not sure where to go from here. I'm so close to getting this running yet feel so far!!

Gitlab:check (yes there's an error in the IMAP settings, but I wouldn't think that is related???)

Checking GitLab Shell ...

GitLab Shell version >= 2.6.10 ? ... OK (2.6.10)
Repo base directory exists? ... yes
Repo base directory is a symlink? ... no
Repo base owned by git:git? ... yes
Repo base access is drwxrws---? ... yes
hooks directories in repos are links: ...
[... project checks all ok ...]
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Check directories and files:
        /var/opt/gitlab/git-data/repositories: OK
        /var/opt/gitlab/.ssh/authorized_keys: OK
Test redis-cli executable: redis-cli 2.8.21
Send ping to redis server: PONG
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Sidekiq ...

Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking Reply by email ...

Address formatted correctly? ... yes
IMAP server credentials are correct? ... no
  Try fixing it:
  Check that the information in config/gitlab.yml is correct
  For more information see:
  doc/incoming_email/README.md
  Please fix the error above and rerun the checks.
Init.d configured correctly? ... skipped (omnibus-gitlab has no init script)
MailRoom running? ... can't check because of previous errors

Checking Reply by email ... Finished

Checking LDAP ...

LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab ...

Git configured with autocrlf=input? ... yes
Database config exists? ... yes
Database is SQLite ... no
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory setup correctly? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
projects have namespace: ...
[... project checks all yes ...]
Redis version >= 2.8.0? ... yes
Ruby version >= 2.1.0 ? ... yes (2.1.8)
Your git bin path is "/opt/gitlab/embedded/bin/git"
Git version >= 1.7.10 ? ... yes (2.6.2)
Active users: 4

Checking GitLab ... Finished

Here's the nginx configuration for the vhost:

location /uploads/ {
    ## If you use HTTPS make sure you disable gzip compression
    ## to be safe against BREACH attack.
    gzip off;

    proxy_read_timeout      300;
    proxy_connect_timeout   300;
    proxy_redirect          off;

    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   $scheme;
    proxy_set_header    X-Frame-Options     SAMEORIGIN;

    proxy_pass https://gitlab;
}

location @gitlab {
    ## If you use HTTPS make sure you disable gzip compression
    ## to be safe against BREACH attack.
    gzip off;

    proxy_read_timeout      300;
    proxy_connect_timeout   300;
    proxy_redirect          off;

    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   $scheme;
    proxy_set_header    X-Frame-Options     SAMEORIGIN;

    #proxy_pass http://gitlab;

    # Returns 502 error if not changed to localhost
    proxy_pass http://localhost:8080;
}

location ~ ^/[\w\.-]+/[\w\.-]+/gitlab-lfs/objects {
    client_max_body_size 0;
    error_page 418 = @gitlab-workhorse;
    return 418;
}

location ~ ^/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$ {
    client_max_body_size 0;
    error_page 418 = @gitlab-workhorse;
    return 418;
}

location ~ ^/[\w\.-]+/[\w\.-]+/repository/archive {
    client_max_body_size 0;
    error_page 418 = @gitlab-workhorse;
    return 418;
}

location ~ ^/api/v3/projects/.*/repository/archive {
    client_max_body_size 0;
    error_page 418 = @gitlab-workhorse;
    return 418;
}

location ~ ^/[\w\.-]+/[\w\.-]+/builds/download {
    client_max_body_size 0;
    error_page 418 = @gitlab-workhorse;
    return 418;
}

location ~ /ci/api/v1/builds/[0-9]+/artifacts {
    client_max_body_size 0;
    error_page 418 = @gitlab-workhorse;
    return 418;
}

location @gitlab-workhorse {
    client_max_body_size 0;
    ## If you use HTTPS make sure you disable gzip compression
    ## to be safe against BREACH attack.
    gzip off;

    #   proxy_read_timeout      300;
    #   proxy_connect_timeout   300;
    #   proxy_redirect          off;

    proxy_buffering off;

    # The following settings only work with NGINX 1.7.11 or newer
    #
    # # Pass chunked request bodies to gitlab-workhorse as-is
    #proxy_request_buffering off;

    proxy_read_timeout      300;
    proxy_connect_timeout   300;
    proxy_redirect          off;

    proxy_http_version 1.1;

    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-Ssl     on;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   $scheme;
    proxy_pass http://gitlab-workhorse;

}

location ~ ^/(assets)/ {
    root /opt/gitlab/embedded/service/gitlab-rails/public;
    gzip_static on; # to serve pre-gzipped version
    expires max;
    add_header Cache-Control public;
}

location ~ / {
    root /opt/gitlab/embedded/service/gitlab-rails/public;
    try_files $uri $uri/index.html $uri.html @gitlab;
}

error_page 502 /502.html;

Lastly, here's the configuration at the server block level for the vhost:

upstream gitlab {
  server unix:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket fail_timeout=0;
}

upstream gitlab-workhorse {
  server unix://var/opt/gitlab/gitlab-workhorse/socket fail_timeout=0;
}

Looking over the docs at https://github.com/gitlabhq/omnibus-gitlab/blob/master/doc/settings/nginx.md I see this note but it doesn't specify what the permissions should be:

*Note: make sure that the webserver user has the correct permissions on all directories used by external web-server, otherwise you will receive failed (XX: Permission denied) while reading upstream errors.

Any guidance would be much appreciated.

Answer 1

Had encountered similar issue, and the log clearly suggested permission issues.

At the end of the documentation from Gitlab for NGINX configuration, http://doc.gitlab.com/omnibus/settings/nginx.html it provides the required permission settings.
sudo usermod -aG gitlab-www www-data after restarting the services it worked as expected.

Answer 2

Strangely no user was defined in my /etc/nginx/nginx.conf file and this was causing the problem you mentioned.

So I added this line in the /etc/nginx/nginx.conf file:

user www-data;

And then I ran this command:

sudo usermod -aG gitlab-www www-data

I restarted Nginx (sudo service nginx restart) and everything went fine.