Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How large does a "large file" have to be to benefit from Git LFS?

Tags:

git

git-lfs

I'm reading about Git LFS and see time and again that it works great for "large files"

Git Large File Storage (LFS) replaces large files such as audio samples, videos[...]

Version large files—even those as large as a couple GB in size—with Git.

Git Large File Storage (LFS) is a free, open-source extension that replaces large files with text pointers inside Git and stores the contents of those files on a remote server.

Unfortunately, I don't see anywhere what a "large file" actually is. It's clear that something that takes up several gigabytes is a large file, but what about something smaller?

Will I benefit from Git LFS with "large files" as small as 50 MB? 20MB? 5MB? 1MB? Less than 1MB?

How large does a "large file" have to be to benefit from Git LFS compared to regular Git?

like image 783
Thunderforge Avatar asked Feb 27 '18 21:02

Thunderforge


People also ask

What is considered a large file in Git?

You can (or have to) decode on your own, which files you add to the repo directly and which you track with LFS. Files up to a couple MB are usually fine. If you exceed 100 MB (which btw. is the hard limit for files stored in git on Github), you should definitely use LFS.

Does Git LFS have a limit?

GitHub has a file size limit of 100mb for a single file, and oftentimes in game development we may need to push files above this limit.

When should you use Git LFS?

You should use Git LFS if you have large files or binary files to store in Git repositories. That's because Git is decentralized. So, every developer has the full change history on their computer.

What is Git LFS large file storage used for?

Git LFS (Large File Storage) is a Git extension developed by Atlassian, GitHub, and a few other open source contributors, that reduces the impact of large files in your repository by downloading the relevant versions of them lazily.


1 Answers

There is no exact threshold to define what is a large file. This is up to the user. To see if you need to store some files using Git LFS you need to understand how git works.

The most fundamental difference between Git and other source control tools (perforce, svn), is that Git stores a full snapshot of the repository on every commit. Thus when you have a large file, the snapshot contains a compressed version of this file (or a pointer to the file blob if the file wasn't changed). The repository snapshot is stored as a graph under the .git folder. Thus if the file is "large", the repository size will grow rapidly.

There are multiple criteria to determine whether to store a file using Git LFS.

  • The size of the file. IMO if a file is more than 10 MB, you should consider storing it in Git LFS

  • How often the file is modified. A large file (based on the users intuition of a large file) that changes very often should be stored using Git LFS

  • The type of the file. A non-text file that cannot be merged is elligible for Git LFS storage

Will I benefit from Git LFS with "large files" as small as 50 MB? 20MB? 5MB? 1MB? Less than 1MB?

Depending on how often the file changes, in any size mentioned you can benefit. Consider the case where you do 100 commits editing the file every time. For a 20MB file that can be compressed say to 15 MB, the repository size would increase by approximately 1.5GB if the file is not stored using Git LFS.

like image 106
yamenk Avatar answered Oct 04 '22 04:10

yamenk