Split a folder into multiple subfolders in terminal/bash script

Question

I have several folders, each with between 15,000 and 40,000 photos. I want each of these to be split into sub folders - each with 2,000 files in them.

What is a quick way to do this that will create each folder I need on the go and move all the files?

Currently I can only find how to move the first x items in a folder into a pre-existing directory. In order to use this on a folder with 20,000 items... I would need to create 10 folders manually, and run the command 10 times.

ls -1  |  sort -n | head -2000| xargs -i mv "{}" /folder/

I tried putting it in a for-loop, but am having trouble getting it to make folders properly with mkdir. Even after I get around that, I need the program to only create folders for every 20th file (start of a new group). It wants to make a new folder for each file.

So... how can I easily move a large number of files into folders of an arbitrary number of files in each one?

Any help would be very... well... helpful!

Answer 1

This solution worked for me on MacOS:

i=0; for f in *; do d=dir_$(printf %03d $((i/100+1))); mkdir -p $d; mv "$f" $d; let i++; done

It creates subfolders of 100 elements each.

Answer 2

Try something like this:

for i in `seq 1 20`; do mkdir -p "folder$i"; find . -type f -maxdepth 1 | head -n 2000 | xargs -i mv "{}" "folder$i"; done

Full script version:

#!/bin/bash

dir_size=2000
dir_name="folder"
n=$((`find . -maxdepth 1 -type f | wc -l`/$dir_size+1))
for i in `seq 1 $n`;
do
    mkdir -p "$dir_name$i";
    find . -maxdepth 1 -type f | head -n $dir_size | xargs -i mv "{}" "$dir_name$i"
done

---

For dummies:

1. create a new file: vim split_files.sh
2. update the dir_size and dir_name values to match your desires
   • note that the dir_name will have a number appended
3. navigate into the desired folder: cd my_folder
4. run the script: sh ../split_files.sh