Xcode server continuous integration and git lfs

Question

I created a bot in Xcode for continuous integration on my project.

I added a trigger to run prior to integration where I'm attempting to do a "git lfs pull" so that it pulls large files into the temporary directory Xcode uses to perform the build.

What is a way to get the "git lfs pull" to work in order to have the integration succeed?

Currently I cannot get it to successfully download large files. My script looks like the following:

#!/bin/bash

changeToRepo() {
    cd ${XCS_SOURCE_DIR}/My-Project-Name
}

changeToRepo
/usr/local/bin/git-lfs pull

However, the large files are not downloaded and when I check the logs for the trigger script, I see the following output.

Git LFS: (0 of 1 files) 0 B / 139.13 MB

Git LFS: (0 of 1 files) 0 B / 139.13 MB
Could not checkout file git-lfs/1.1.0 (GitHub; darwin amd64; go 1.5.1; git 258acf1) git version 2.5.4 (Apple Git-61)

$ git-lfs pull Could not checkout file

Could not write working directory file: Error opening media file. goroutine 66 [running]: github.com/github/git-lfs/lfs.Stack(0x0, 0x0, 0x0) /Users/rick/go/src/github.com/github/git-lfs/lfs/errors.go:557 +0x80 github.com/github/git-lfs/commands.logPanicToWriter(0x89a1e8, 0xc82002e018, 0x896028, 0xc82000e480) /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:184 +0xf7f github.com/github/git-lfs/commands.logPanic(0x896028, 0xc82000e480, 0x0, 0x0) /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:148 +0x421 github.com/github/git-lfs/commands.handlePanic(0x896028, 0xc82000e480, 0x0, 0x0) /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:123 +0x4e github.com/github/git-lfs/commands.LoggedError(0x896028, 0xc82000e480, 0x548060, 0x17, 0x0, 0x0, 0x0) /Users/rick/go/src/github.com/github/git-lfs/commands/commands.go:73 +0x82 github.com/github/git-lfs/commands.checkoutWithChan(0xc82012c4e0) /Users/rick/go/src/github.com/github/git-lfs/commands/command_checkout.go:202 +0x860 github.com/github/git-lfs/commands.checkoutFromFetchChan.func1(0xc82012c4e0, 0xc82018e040) /Users/rick/go/src/github.com/github/git-lfs/commands/command_checkout.go:78 +0x21 created by github.com/github/git-lfs/commands.checkoutFromFetchChan /Users/rick/go/src/github.com/github/git-lfs/commands/command_checkout.go:80 +0x439

Answer 1

Here's how I got git lfs to work in a bot. First, I edited my bot "Before Integration" trigger run script to look like this:

#!/bin/bash
export PATH="$PATH:/opt/local/bin:/opt/local/sbin:/usr/local/bin"
(cd ${XCS_SOURCE_DIR}/My-Project_Dir/My-Subfolder-Dir; sudo git lfs pull)

We export the PATH b/c git lfs lives inside of user/local/bin (these other locations may apply depending on how you installed it or what kind of server you are using). Otherwise, git lfs cannot be found.

The directory My-Project_Dir/My-Subfolder-Dir is the location of the project. The Xcode server will download my source to ${XCS_SOURCE_DIR} which is a temp folder, but the .git file pertaining to my large file lives in My-Project_Dir/My-Subfolder-Dir.

Note that the cd and git lfs commands are inside of parentheses, this is because the cd command spawns a child process. If we just had a line with cd and had git lfs on the next line, the directory would not stay changed because after the child process exits, the parent is still in the initial directory. The git lfs command will not be issued inside that new directory in the child process unless we run the commands sequentially inside of that child process.

Also note that I need to use sudo with git lfs, otherwise git lfs does not have permission. The bot is running in a different user space than my server admin space, with user name _xcsbuildd.

So, add the _xcsbuildd user to sudoers. In Terminal, use the following command:

$ sudo visudo

Find the line "%admin ALL=(ALL) ALL", press "a" to edit the file, and below it add the following line:

_xcsbuildd ALL=(ALL) NOPASSWD: ALL

Hit Esc and then write and quit by typing:

:wq 

Check to make sure the sudoers is correct:

$ sudo cat /etc/sudoers | grep _xcsbuildd
_xcsbuildd ALL=(ALL) NOPASSWD: ALL 

Answer 2

Folks, I'm not sure if there's a problem with the original solution, but if you have problems, try something like below - it worked for me with Xcode Build Server from Xcode 9.2 / didn't require any permissions/sudoing and etc. Basically it's about installing git-lfs via homebrew and executing git lfs pull right after that.

#!/bin/sh -ex
pushd "${XCS_SOURCE_DIR:?}"

rm -rf homebrew
mkdir homebrew
curl -L "https://github.com/Homebrew/brew/tarball/master" | tar xz --strip 1 -C homebrew
export PATH="$PATH:$PWD/homebrew/bin"
brew install git-lfs

cd YourCheckedOutRepository
git lfs pull

popd