How to substitute string in {} in "find (...) -exec (...) {} \;" bash command?

Question

How can I substitute a string in {} as found by bash's ´find´?

For instance I would like ? below to substitute "in" by "out":

find . -name "*.in" -exec python somescript.py {} ? \;

i.e. to execute for all "*.in" files

python somescript.py somefile.in somefile.out

Answer 1

find doesn't have a substitution feature. You need to call a shell.

find . -name "*.in" -exec sh -c 'python somescript.py "$0" "${0%.in}.out"' {} \;

$0 is the file name, ${0%.in} removes the .in suffix.

Alternatively, in bash (but not in plain sh), run shopt -s globstar to enable recursive directory expansion (and shopt -s nullglob if there's a risk that there won't be any matching .in file) and use a for loop instead of find.

for x in **/*.in; do
  python somescript.py "$x" "${x%.in}.out"
done

Answer 2

Using Brace Expansion:

find . -name "*.in" -exec bash -c 'python script.py "${0%.*}"{.in,.out}' {} \;

Using Shell Parameter Expansion:

find . -name "*.in" -exec bash -c 'python script.py "${0} ${0/.in/.out}"' {} \;

Result:

python script.py somefile.in somefile.out

Answer 3

For me, all the built-in tools were a bit cumbersome to use (did I get the shell quoting right? will it do the right thing? can I easily preview changes?).

A few years ago, I wrote rpt that will take its command line parameters, open a text editor (to let you edit the file names) and then run a command (default mv) on the old and new filenames.

Recently, I took this a step further, here's fea (For Each Argument):

#!/usr/bin/env python3
# fea: For Each Argument
# https://thp.io/2019/rpt-and-fea.html

import sys, shlex, re

_, cmd, *args = sys.argv

print('\n'.join(re.sub(r'[{](.)([^}]*?)(\1)([^}]*?)(\1)[}]',
    lambda m: shlex.quote(re.sub(m.group(2), m.group(4), arg)),
    cmd).replace('{}', shlex.quote(arg)) for arg in args))

The following replacements happen:

{} -> insert arg
{/regex/replacement/} -> run re.sub(regex, replacement) on the arg,
                         you can pick any character for "/", as long
                         as it appears at the start, middle and end
                         (to separate the regex from the replacement)

Some use cases:

# Create backup files
fea "cp {} {}.bak" *.py

# Encode WAV files with oggenc
fea "oggenc {} -o {/wav$/ogg/}" *.wav

# Decode MP3 files with mpg123
fea "mpg123 -w {/mp3$/wav/} {}" *.mp3

# Render markdown documents to HTML
fea "markdown {} > {/md$/html/}" *.md

# Fancy replacement
fea "markdown {} > {#input/(.*).md#output/\1.html#}" input/*.md

# As mentioned above, note that you need to pipe the
# fea output into a shell to execute the command
fea "cp {} {}.bak" *.py | sh -e -x

Your use case can be covered like this:

find . -name '*.in' -print0 | xargs -0 fea "python somescript.py {} {/in$/out/}"

Or if the files are just in the current folder:

fea "python somescript.py {} {/in$/out/}" *.in

Or if you are using zsh with recursive globbing:

fea "python somescript.py {} {/in$/out/}" **/*.in

If you are happy with the commands it displays, just pipe its output into sh -e -x (or bash or whatever) to execute them.

While it's possible to make a for loop with the shell (or any of the parameter/variable expansions), shell quoting and escaping is so hard to get right (for me), the fea tool makes sure it also works with weird names:

touch 'some$weird "filename.py'
touch 'and !! more.py'
touch 'why oh why?.py'
touch "this is ridiculous'.py"
fea "cp {} {}.bak" *.py | sh -e -x

See rpt and fea blog post for details.