Linux: Move 1 million files into prefix-based created Folders

Question

I have a directory called "images" filled with about one million images. Yep.

I want to write a shell command to rename all of those images into the following format:

original: filename.jpg
new: /f/i/l/filename.jpg

Any suggestions?

Thanks,
Dan

Answer 1

for i in *.*; do mkdir -p ${i:0:1}/${i:1:1}/${i:2:1}/; mv $i ${i:0:1}/${i:1:1}/${i:2:1}/; done;

The ${i:0:1}/${i:1:1}/${i:2:1} part could probably be a variable, or shorter or different, but the command above gets the job done. You'll probably face performance issues but if you really want to use it, narrow the *.* to fewer options (a*.*, b*.* or what fits you)

edit: added a $ before i for mv, as noted by Dan

Answer 2

You can generate the new file name using, e.g., sed:

$ echo "test.jpg" | sed -e 's/^\(\(.\)\(.\)\(.\).*\)$/\2\/\3\/\4\/\1/'
t/e/s/test.jpg

So, you can do something like this (assuming all the directories are already created):

for f in *; do
   mv -i "$f" "$(echo "$f" | sed -e 's/^\(\(.\)\(.\)\(.\).*\)$/\2\/\3\/\4\/\1/')"
done

or, if you can't use the bash $( syntax:

for f in *; do
   mv -i "$f" "`echo "$f" | sed -e 's/^\(\(.\)\(.\)\(.\).*\)$/\2\/\3\/\4\/\1/'`"
done

However, considering the number of files, you may just want to use perl as that's a lot of sed and mv processes to spawn:

#!/usr/bin/perl -w
use strict;

# warning: untested
opendir DIR, "." or die "opendir: $!";
my @files = readdir(DIR); # can't change dir while reading: read in advance
closedir DIR;
foreach my $f (@files) {
    (my $new_name = $f) =~ s!^((.)(.)(.).*)$!$2/$3/$4/$1/;
    -e $new_name and die "$new_name already exists";
    rename($f, $new_name);
}

That perl is surely limited to same-filesystem only, though you can use File::Copy::move to get around that.