How do i make my awk command into a python command

Question

I have an awk command that works in bash, but im now trying to put it into a python script

I have tried both os.system, and subprocess.call both return the same error. sh: 1: Syntax error: "(" unexpected

os.system('awk \'FNR<=27{print;next} ++count%10==0{print;count}\' \'{0} > {1}\'.format(inputfile, outpufile)')

So this awk command will take the large inputfile and create an output file that leaves the first 27 lines of header, but then starting on line 28 it only takes every 10th line and puts it into the output file

Im using the .format() because it is within a python script where the input file will be different every times its run.

ive also tried

subprocess.call('awk \'FNR<=27{print;next} ++count%10==0{print;count}\' \'{0} > {1}\'.format(inputfile, outpufile)')

both come up with the same error above. What am I missing?

Answer 1

There are two main issues with your Python code:

1. format() is a python method call, it should not be put into the string of awk_cmd to execute under the shell
2. when calling format() method, braces {} are used to identify substitution target in the format string objects, they need to be escaped using {{ ... }}

See below a modified version of your code:

awk_cmd = "awk 'FNR<=7{{print;next}} ++count%10==0{{print;count}}' {0} > {1}".format(inputfile, outpufile)
os.system(awk_cmd)

Answer 2

As per the comment above, probably more pythonic (and more manageable) to directly use python.

But, if you want to use awk then one way is to format your command with your variable filenames separately.

This works using a basic test text file:

import os


def awk_runner(inputfile, outputfile):
    cmd = "awk 'FNR<=27{print;next} ++count%10==0{print;count}' " + inputfile + " > " + outputfile
    os.system(cmd)


awk_runner('test1.txt', 'testout1.txt')