Git Annex with GitHub?

Question

I'm not very familiar with git-annex. I've just been reading the documentation trying to get my mind around it. The thought that has come to my mind, which I can't find certainty of is: Could I host a git repo on GitHub, and git push to it with git and git annex, but then set up git annex so that it pushes the binary files to like an S3 bucket, and all the source code files go to GitHub?

Answer 1

git-annex supports a variety of special remotes for storing archived content, including Amazon S3. It is possible to store git repository content (which includes symlinks to annexed content as well as any content stored directly by git alongside the annexed files, such as source code) whilst using special remotes (or other remotes where git-annex is available) to stored annexed content.

Answer 2

Yes you can do that. Simplest way is to git add the files you want to directly be in the git repo (e.g. the source code) and git annex add the large files.

You can then check in any changes to the source code files (or anything else you added with git add) to github as normal.

You can manage the storage and versioning of the large files using git annex commands. Git annex supports using AWS S3 and/or glacier for backing up the files. It can also back them up to a server you control over ssh or to an external drive (or any combination of the above). http://git-annex.branchable.com/special_remotes/

With the latest version of git annex, you can also set up automatically filters that decide which types/sizes of files to check in directly to git vs which ones to store as links in the annex. https://git-annex.branchable.com/tips/largefiles/