sed command to fix filenames in a directory

Question

I run a script which generated about 10k files in a directory. I just discovered that there is a bug in the script which causes some filenames to have a carriage return (presumably a '\n' character).

I want to run a sed command to remove the carriage return from the filenames.

Anyone knows which params to pass to sed to clean up the filenames in the manner described?

I am running Linux (Ubuntu)

Answer 1

I don't know how sed would do this, but this python script should do the trick:.

This isn't sed, but I find python a lot easier to use when doing things like these:

#!/usr/bin/env python

import os

files = os.listdir('.')

for file in files:
  os.rename(file, file.replace('\r', '').replace('\n', ''))
  print 'Processed ' + file.replace('\r', '').replace('\n', '')

It strips any occurrences of both \r and \n from all of the filenames in a given directory.

To run it, save it somewhere, cd into your target directory (with the files to be processed), and run python /path/to/the/file.py.

Also, if you plan on doing more batch renaming, consider Métamorphose. It's a really nice and powerful GUI for this stuff. And, it's free!

Good luck!

---

Actually, try this: cd into the directory, type in python, and then just paste this in:

exec("import os\nfor file in os.listdir('.'):\n  os.rename(file, file.replace('\\r', '').replace('\\n', ''))\n  print 'Processed ' + file.replace('\\r', '').replace('\\n', '')")

It's a one-line version of the previous script, and you don't have to save it.

---

Version 2, with space replacement powers:

#!/usr/bin/env python

import os

for file in os.listdir('.'):
  os.rename(file, file.replace('\r', '').replace('\n', '').replace(' ', '_')
  print 'Processed ' + file.replace('\r', '').replace('\n', '')

And here's the one-liner:

exec("import os\nfor file in os.listdir('.'):\n  os.rename(file, file.replace('\\r', '').replace('\\n', '')replace(' ', '_'))\n  print 'Processed ' + file.replace('\\r', '').replace('\\n', '');")

Answer 2

If there are no spaces in your filenames, you can do:

for f in *$'\n'; do mv "$f" $f; done

It won't work if the newlines are embedded, but it will work for trailing newlines.

If you must use sed:

for f in *$'\n'; do mv "$f" "$(echo "$f" | sed '/^$/d')"; done

Using the rename Perl script:

rename 's/\n//g' *$'\n'

or the util-linux-ng utility:

rename $'\n' '' *$'\n'

If the character is a return instead of a newline, change the \n or ^$ to \r in any places they appear above.

Answer 3

The reason you aren't getting any pure-sed answers is that fundamentally sed edits file contents, not file names; thus the answers that use sed all do something like echo the filename into a pipe (pseudo file), edit that with sed, then use mv to turn that back into a filename.

Since sed is out, here's a pure-bash version to add to the Perl, Python, etc scripts you have so far:

killpattern=$'[\r\n]' # remove both carriage returns and linefeeds
for f in *; do
    if [[ "$f" == *$killpattern* ]]; then
        mv "$f" "${f//$killpattern/}"
    fi
done

...but since ${var//pattern/replacement} isn't available in plain sh (along with [[...]]), here's a version using sh-only syntax, and tr to do the character replacement:

for f in *; do
    new="$(printf %s "$f" | tr -d "\r\n")"
    if [ "$f" != "$new" ]; then
        mv "$f" "$new"
    fi
done