Making Python scripts work with xargs

Question

What would be the process of making my Python scripts work well with 'xargs'? For instance, I would like the following command to work through each line of text file, and execute an arbitrary command:

cat servers.txt | ./hardware.py -m 

Essentially would like each line to be passed to the hardware.py script.

Answer 1

To make your commands work with xargs you simply need them to accept arguments. Arguments in Python are in the sys.argv list. In this way you can execute somthing like:

find . -type f -name '*.txt' -print0 | xargs -0 ./myscript.py

which might be equivalent to

./myscript.py ./foo.txt ./biz/foobar.txt ./baz/yougettheidea.txt

To make your commands work with standard input, you can also use the sys module, this time with sys.stdin, which you can treat as a file. This is more like the example you gave:

./myscript.py < somefile.txt

Answer 2

You are confusing two issues.

First, your application can receive input from stdin. This has nothing to do with xargs. In your example, all hardware.py needs to do is read sys.stdin as the input file, e.g.:

if __name__=='__main__':
    for line in sys.stdin:
         do_something(line)

If you want hardware.py to produce output that other programs down the line can use, just write to sys.stdout

Second, your application can receive input from arguments. This is where you would use xargs. For example:

xargs ./hardware.py < servers.txt # same as cat servers.txt | xargs ./hardware.py

This would pass every "word" of servers.txt (not every line) as an argument to hardware.py (possibly multiple arguments at once). This would be the same as running hardware.py word1 word2 word3 word4 ...

Python stores command line arguments in the sys.arvg array. sys.argv[0] will be the command name, and sys.argv[1:] will be all the command line arguments. However, you are usually better off processing your command line using argparse.