Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to quickly find all git repos under a directory

Tags:

bash

The following bash script is slow when scanning for .git directories because it looks at every directory. If I have a collection of large repositories it takes a long time for find to churn through every directory, looking for .git. It would go much faster if it would prune the directories within repos, once a .git directory is found. Any ideas on how to do that, or is there another way to write a bash script that accomplishes the same thing?

#!/bin/bash  # Update all git directories below current directory or specified directory  HIGHLIGHT="\e[01;34m" NORMAL='\e[00m'  DIR=. if [ "$1" != "" ]; then DIR=$1; fi cd $DIR>/dev/null; echo -e "${HIGHLIGHT}Scanning ${PWD}${NORMAL}"; cd ->/dev/null  for d in `find . -name .git -type d`; do   cd $d/.. > /dev/null   echo -e "\n${HIGHLIGHT}Updating `pwd`$NORMAL"   git pull   cd - > /dev/null done 

Specifically, how would you use these options? For this problem, you cannot assume that the collection of repos is all in the same directory; they might be within nested directories.

top   repo1   dirA    dirB      dirC         repo1 
like image 407
Mike Slinn Avatar asked Aug 16 '12 06:08

Mike Slinn


People also ask

How do I see all repositories in git?

You can get a list of any configured remote URLs with the command git remote -v . git remote -v because -v is for verbose. git remote gives a simple list of remotes (base, origin in this case). The -v option includes the url for both fetch and push operations of each remote.

Where is the git repository directory?

git directory is a configuration file for git. Use the terminal to display the . git directory with the command ls -a . The ls command lists the current directory contents and by default will not show hidden files.

Where is the .git directory in Linux?

If you are in Linux find / -name ". git" , otherwise there is no way, they are standard directories, just use your OS file/folder find program to find . git named folders.

Can git track a folder?

Git Empty directories in Git Git doesn't track directories js. Assume you added a build step to your application and rely on the "build" directory to be there as the output directory (and you don't want to make it a setup instruction every developer has to follow), a convention is to include a ".


2 Answers

Check out Dennis' answer in this post about find's -prune option:

How to use '-prune' option of 'find' in sh?

find . -name .git -type d -prune 

Will speed things up a bit, as find won't descend into .git directories, but it still does descend into git repositories, looking for other .git folders. And that 'could' be a costly operation.

What would be cool is if there was some sort of find lookahead pruning mechanism, where if a folder has a subfolder called .git, then prune on that folder...

That said, I'm betting your bottleneck is in the network operation 'git pull', and not in the find command, as others have posted in the comments.

like image 93
Clayton Stanley Avatar answered Sep 21 '22 16:09

Clayton Stanley


Here is an optimized solution:

#!/bin/bash # Update all git directories below current directory or specified directory # Skips directories that contain a file called .ignore  HIGHLIGHT="\e[01;34m" NORMAL='\e[00m'  function update {   local d="$1"   if [ -d "$d" ]; then     if [ -e "$d/.ignore" ]; then        echo -e "\n${HIGHLIGHT}Ignoring $d${NORMAL}"     else       cd $d > /dev/null       if [ -d ".git" ]; then         echo -e "\n${HIGHLIGHT}Updating `pwd`$NORMAL"         git pull       else         scan *       fi       cd .. > /dev/null     fi   fi   #echo "Exiting update: pwd=`pwd`" }  function scan {   #echo "`pwd`"   for x in $*; do     update "$x"   done }  if [ "$1" != "" ]; then cd $1 > /dev/null; fi echo -e "${HIGHLIGHT}Scanning ${PWD}${NORMAL}" scan * 
like image 35
Mike Slinn Avatar answered Sep 21 '22 16:09

Mike Slinn