Speed up bash script that uses several find commands

Question

I have a bash script to add some files of a project to git and then synchronize that branch, as the amount of files has increased i have noticed the script has become much slower, so i want to figure out if I'm doing it the correct way.

This is the section of the script where the files are added:

echo "Adding files..."
find . -name '*.js' -exec git add {} \;
find . -name '*.html' -exec git add {} \;
find . -name '*.css' -exec git add {} \;
find . -name '*.py' -exec git add {} \;
find . -name '*.txt' -exec git add {} \;
find . -name '*.jpg' -exec git add {} \;
find . -name '*.sh' -exec git add {} \;

echo "Commit"
git commit -m "'$1'"

I'm not sure if making one single call to find would be faster than having all these separated commands, but I did it this way so it was simpler to remove some types of file or add new ones.

I would really appreciate any suggestion to make this more efficient, using the commands in a different way or using different commands is a completely acceptable answer.

Answer 1

If git supports adding multiple files in one command, the simplest thing you can do is use the + suffix for -exec:

find . -name '*.js' -exec git add {} \+

This gathers a large number of files and passes them all to the command in one command line.

So what will be executed is:

git add a.js b.js c.js d.js

instead of

git add a.js
git add b.js
git add c.js
git add d.js

If you're processing hundreds or thousands of files, this will make a big difference in execution time.

To combine all the file patterns into a single find command, use find's "or" operator:

find . \( -name '*.js' -o   \
          -name '*.html' -o \
          -name '*.css' -o  \
          -name '*.py' -o   \
          -name '*.txt' -o  \
          -name '*.jpg' -o  \
          -name '*.sh' \) -exec git add {} +

The \ before ( and ) is needed to protect them from their special shell meaning. You could equally use quotes instead: '(', ')'.

find has some complicated options and one has to go to a bit of trouble to learn them and become familiar with them but I've saved myself a lot of effort over the years by being able to reel off a complicated find command rather than struggling with filtering file names through grep and awk and so forth.

One of my current favourite patterns for scanning through a maven/subversion java project while ignoring uninteresting files is:

find . \( \( \( -iname .svn -o -iname target -o -iname classes \) -type d -prune -false \) -o \( <your filter expression> \) \) -exec grep -li xxx {} +