Parsing output of ls
to iterate through list of files is bad. So how should I go about iterating through list of files in order by which they were first created? I browsed several questions here on SO and they all seem to parsing ls
.
The embedded link suggests:
Things get more difficult if you wanted some specific sorting that only
ls
can do, such as ordering bymtime
. If you want the oldest or newest file in a directory, don't usels -t | head -1
-- read Bash FAQ 99 instead. If you truly need a list of all the files in a directory in order by mtime so that you can process them in sequence, switch to perl, and have your perl program do its own directory opening and sorting. Then do the processing in the perl program, or -- worst case scenario -- have the perl program spit out the filenames with NUL delimiters.Even better, put the modification time in the filename, in YYYYMMDD format, so that glob order is also mtime order. Then you don't need ls or perl or anything. (The vast majority of cases where people want the oldest or newest file in a directory can be solved just by doing this.)
Does that mean there is no native way of doing it in bash
? I don't have the liberty to modify the filename to include the time in them. I need to schedule a script in cron
that would run every 5 minutes, generate an array containing all the files in a particular directory ordered by their creation time and perform some actions on the filenames and move them to another location.
The following worked but only because I don't have funny filenames. The files are created by a server so it will never have special characters, spaces, newlines etc.
files=( $(ls -1tr) )
I can write a perl
script that would do what I need but I would appreciate if someone can suggest the right way to do it in bash
. Portable option would be great but solution using latest GNU utilities will not be a problem either.
The syntax to loop through each file individually in a loop is: create a variable (f for file, for example). Then define the data set you want the variable to cycle through. In this case, cycle through all files in the current directory using the * wildcard character (the * wildcard matches everything).
Bash Sort Files Alphabetically By default, the ls command lists files in ascending order. To reverse the sorting order, pass the -r flag to the ls -l command, like this: ls -lr . Passing the -r flag to the ls -l command applies to other examples in this tutorial.
Use the ls Command to List Directories in Bash. We use the ls command to list items in the current directory in Bash. However, we can use */ to print directories only since all directories finish in a / with the -d option to assure that only the directories' names are displayed rather than their contents.
sorthelper=();
for file in *; do
# We need something that can easily be sorted.
# Here, we use "<date><filename>".
# Note that this works with any special characters in filenames
sorthelper+=("$(stat -n -f "%Sm%N" -t "%Y%m%d%H%M%S" -- "$file")"); # Mac OS X only
# or
sorthelper+=("$(stat --printf "%Y %n" -- "$file")"); # Linux only
done;
sorted=();
while read -d $'\0' elem; do
# this strips away the first 14 characters (<date>)
sorted+=("${elem:14}");
done < <(printf '%s\0' "${sorthelper[@]}" | sort -z)
for file in "${sorted[@]}"; do
# do your stuff...
echo "$file";
done;
Other than sort
and stat
, all commands are actual native Bash commands (builtins)*. If you really want, you can implement your own sort
using Bash builtins only, but I see no way of getting rid of stat
.
The important parts are read -d $'\0'
, printf '%s\0'
and sort -z
. All these commands are used with their null-delimiter options, which means that any filename can be procesed safely. Also, the use of double-quotes in "$file"
and "${anarray[*]}"
is essential.
*Many people feel that the GNU tools are somehow part of Bash, but technically they're not. So, stat
and sort
are just as non-native as perl
.
With all of the cautions and warnings against using ls
to parse a directory notwithstanding, we have all found ourselves in this situation. If you do find yourself needing sorted directory input, then about the cleanest use of ls
to feed your loop is ls -opts | read -r name; do...
This will handle spaces in filenames, etc.. without requiring a reset of IFS
due to the nature of read
itself. Example:
ls -1rt | while read -r fname; do # where '1' is ONE not little 'L'
So do look for cleaner solutions avoiding ls
, but if push comes to shove, ls -opts
can be used sparingly without the sky falling or dragons plucking your eyes out.
let me add the disclaimer to keep everyone happy. If you like newlines
inside your filenames -- then do not use ls
to populate a loop. If you do not have newlines
inside your filenames, there are no other adverse side-effects.
Contra: TLDP Bash Howto Intro:
#!/bin/bash
for i in $( ls ); do
echo item: $i
done
It appears that SO users do not know what the use of contra means -- please look it up before downvoting.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With