How can I manipulate file names using bash and sed?

Question

I am trying to loop through all the files in a directory.

I want to do some stuff on each file (convert it to xml, not included in example), then write the file to a new directory structure.

for file in `find /home/devel/stuff/static/ -iname "*.pdf"`;
do
  echo $file;
  sed -e 's/static/changethis/' $file > newfile +".xml";
  echo $newfile;
done

I want the results to be:

$file => /home/devel/stuff/static/2002/hello.txt

$newfile => /home/devel/stuff/changethis/2002/hello.txt.xml

How do I have to change my sed line?

Answer 1

If you need to rename multiple files, I would suggest to use rename command:

# remove "-n" after you verify it is what you need
rename -n 's/hello/hi/g' $(find /home/devel/stuff/static/ -type f)

or, if you don't have rename try this:

find /home/devel/stuff/static/ -type f | while read FILE
do
    # modify line below to do what you need, then remove leading "echo" 
    echo mv $FILE $(echo $FILE | sed 's/hello/hi/g')
done

Answer 2

Are you trying to change the filename? Then

for file in /home/devel/stuff/static/*/*.txt
do
    echo "Moving $file"
    mv "$file" "${file/static/changethis}.xml"
done

Please make sure /home/devel/stuff/static/*/*.txt is what you want before using the script.

Answer 3

First, you have to create the name of the new file based on the name of the initial file. The obvious solution is:

newfile=${file/static/changethis}.xml

Second you have to make sure that the new directory exists or create it if not:

mkdir -p $(dirname $newfile)

Then you can do something with your file:

doSomething < $file > $newfile

Answer 4

I wouldn't do the for loop because of the possibility of overloading your command line. Command lines have a limited length, and if you overload it, it'll simply drop off the excess without giving you any warning. It might work if your find returns 100 file. It might work if it returns 1000 files, but it might fail if your find returns 1000 files and you'll never know.

The best way to handle this is to pipe the find into a while read statement as glenn jackman.

The sed command only works on STDIN and on files, but not on file names, so if you want to munge your file name, you'll have to do something like this:

$newname="$(echo $oldname | sed 's/old/new/')"

to get the new name of the file. The $() construct executes the command and puts the results of the command on STDOUT.

So, your script will look something like this:

find /home/devel/stuff/static/ -name "*.pdf" | while read $file
do
    echo $file;
    newfile="$(echo $file | sed -e 's/static/changethis/')"
    newfile="$newfile.xml"
    echo $newfile;
 done

Now, since you're renaming the file directory, you'll have to make sure the directory exists before you do your move or copy:

find /home/devel/stuff/static/ -name "*.pdf" | while read $file
do
    echo $file;
    newfile="$(echo $file | sed -e 's/static/changethis/')"
    newfile="$newfile.xml"
    echo $newfile;

    #Check for directory and create it if it doesn't exist
    $dirname=$(dirname "$newfile")
    if [ ! -d "$dirname" ]
    then
        mkdir -p "$dirname"
    fi

    #Directory now exists, so you can do the move
    mv "$file" "$newfile"
 done

Note the quotation marks to handle the case there's a space in the file name.

By the way, instead of doing this:

if [ ! -d "$dirname" ]
then
    mkdir -p "$dirname"
fi

You can do this:

[ -d "$dirname"] || mkdir -p "$dirname"

The || means to execute the following command only if the test isn't true. Thus, if [ -d "$dirname" ] is a false statement (the directory doesn't exist), you run mkdir.

It's a fairly common shortcut when you see shell scripts.