Managing large binary files with Git

Question

I am looking for opinions of how to handle large binary files on which my source code (web application) is dependent. We are currently discussing several alternatives:

1. Copy the binary files by hand.
   • Pro: Not sure.
   • Contra: I am strongly against this, as it increases the likelihood of errors when setting up a new site/migrating the old one. Builds up another hurdle to take.
2. Manage them all with Git.
   • Pro: Removes the possibility to 'forget' to copy a important file
   • Contra: Bloats the repository and decreases flexibility to manage the code-base and checkouts, clones, etc. will take quite a while.
3. Separate repositories.
   • Pro: Checking out/cloning the source code is fast as ever, and the images are properly archived in their own repository.
   • Contra: Removes the simpleness of having the one and only Git repository on the project. It surely introduces some other things I haven't thought about.

What are your experiences/thoughts regarding this?

Also: Does anybody have experience with multiple Git repositories and managing them in one project?

The files are images for a program which generates PDFs with those files in it. The files will not change very often (as in years), but they are very relevant to a program. The program will not work without the files.

Answer 1

I discovered git-annex recently which I find awesome. It was designed for managing large files efficiently. I use it for my photo/music (etc.) collections. The development of git-annex is very active. The content of the files can be removed from the Git repository, only the tree hierarchy is tracked by Git (through symlinks). However, to get the content of the file, a second step is necessary after pulling/pushing, e.g.:

$ git annex add mybigfile $ git commit -m'add mybigfile' $ git push myremote $ git annex copy --to myremote mybigfile ## This command copies the actual content to myremote $ git annex drop mybigfile ## Remove content from local repo ... $ git annex get mybigfile ## Retrieve the content ## or to specify the remote from which to get: $ git annex copy --from myremote mybigfile 

There are many commands available, and there is a great documentation on the website. A package is available on Debian.