When exceeding the data quota, what is the best way to "split up" this github repo?

Question

I added several ~1M files to (an already large) repo. Upon the next add/commit/push, I received this error:

Git LFS: (0 of 1 files) 0 B / 28.35 KB   
batch response: http: This repository is over its data quota. Purchase more data packs to restore access.
Docs: https://help.github.com/articles/purchasing-additional-storage-and-bandwidth-for-a-personal-account/
error: failed to push some refs to 'https://github.com/USERNAME/REPO_NAME

Without paying for more data packs (which isn't feasible on my current budget, though I am doing research with an academic account with GitHub), how would one resolve this issue?

Perhaps this is a straightforward question, but I haven't been able to find the guidelines online: what is the recommended way for users to split this repo into two? I suspect one would simply create a new repo, move these files over to this, and delete these files in the previous repo (deleting all branches in the process).

Are there other options available? It's very strange----most of these files are less than 1MB....naturally if there was one large file causing the problem, then this process would be a bit more straightforward.

Answer 1

The challenge you're going to run into is that you really can't make a repository any smaller without rewriting its history. If you're OK with rewriting history, you can use git filter-branch to modify history in bulk, e.g. --index-filter 'git rm --cached --ignore-unmatch path/to/big/file' (or use --subdirectory-filter to limit the repository to a single subfolder).

I'm not certain how you would go about purging/splitting the files that have been stored on GitHub by Git-LFS, though.

It's very strange----most of these files are less than 1MB....naturally if there was one large file causing the problem, then this process would be a bit more straightforward.

If the 1MB files are uncompressible and change often (or there are just a lot of them), it's expected that the repository would get relatively large pretty quickly.