Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Limiting file size in git repository

I'm currently thinking of changing my VCS (from subversion) to git. Is it possible to limit the file size within a commit in a git repository? For e. g. subversion there is a hook: http://www.davidgrant.ca/limit_size_of_subversion_commits_with_this_hook

From my experience people, especially those who are inexperienced, sometimes tend to commit files which should not go into a VCS (e. g. big file system images).

like image 632
dubbaluga Avatar asked Aug 22 '11 12:08

dubbaluga


People also ask

Do GitHub repos have a size limit?

Repositories have a hard size limit of 100GB. As of 2020, the documented size limit of each such file is 2 GB.

How do I reduce the size of a Git file?

remove the file from your project's current file-tree. remove the file from repository history — rewriting Git history, deleting the file from all commits containing it. remove all reflog history that refers to the old commit history. repack the repository, garbage-collecting the now-unused data using git gc.

How do I manage large files in Git?

Use the Git LFS extension with a repository to speed up the handling of large files. Use the Git Large File Storage (LFS) extension with an existing Bitbucket Cloud repository. Use the bfg-repo-cleaner utility to change the Git history of a repository.

Does Git LFS reduce size?

Git LFS does not compress files. Some files are compressible, and some are not. It, like Git's partial clone feature, is designed to offload most of the data to a trusted server for the purposes of making local access lighter and cheaper.


1 Answers

As I was struggling with it for a while, even with the description, and I think this is relevant for others too, I thought I'd post an implementation of how what J16 SDiZ described could be implemented.

So, my take on the server-side update hook preventing too big files to be pushed:

#!/bin/bash  # Script to limit the size of a push to git repository. # Git repo has issues with big pushes, and we shouldn't have a real need for those # # eis/02.02.2012  # --- Safety check, should not be run from command line if [ -z "$GIT_DIR" ]; then         echo "Don't run this script from the command line." >&2         echo " (if you want, you could supply GIT_DIR then run" >&2         echo "  $0 <ref> <oldrev> <newrev>)" >&2         exit 1 fi  # Test that tab replacement works, issue in some Solaris envs at least testvariable=`echo -e "\t" | sed 's/\s//'` if [ "$testvariable" != "" ]; then         echo "Environment check failed - please contact git hosting." >&2         exit 1 fi   # File size limit is meant to be configured through 'hooks.filesizelimit' setting filesizelimit=$(git config hooks.filesizelimit)  # If we haven't configured a file size limit, use default value of about 100M if [ -z "$filesizelimit" ]; then         filesizelimit=100000000 fi  # Reference to incoming checkin can be found at $3 refname=$3  # With this command, we can find information about the file coming in that has biggest size # We also normalize the line for excess whitespace biggest_checkin_normalized=$(git ls-tree --full-tree -r -l $refname | sort -k 4 -n -r | head -1 | sed 's/^ *//;s/ *$//;s/\s\{1,\}/ /g' )  # Based on that, we can find what we are interested about filesize=`echo $biggest_checkin_normalized | cut -d ' ' -f4,4`  # Actual comparison # To cancel a push, we exit with status code 1 # It is also a good idea to print out some info about the cause of rejection if [ $filesize -gt $filesizelimit ]; then          # To be more user-friendly, we also look up the name of the offending file         filename=`echo $biggest_checkin_normalized | cut -d ' ' -f5,5`          echo "Error: Too large push attempted." >&2         echo  >&2         echo "File size limit is $filesizelimit, and you tried to push file named $filename of size $filesize." >&2         echo "Contact configuration team if you really need to do this." >&2         exit 1 fi  exit 0 

Note that it's been commented that this code only checks the latest commit, so this code would need to be tweaked to iterate commits between $2 and $3 and do the check to all of them.

like image 64
eis Avatar answered Oct 17 '22 06:10

eis