Create Output file with multiple lines (Python)

Question

I have an file with a specific data I would like to pull.

The file looks like this:

DS User ID 1  
random garbage  
random garbage  
DS  N user name 1   
random garbage  
DS User ID 2   
random garbage  
random garbage  
DS  N user name 2

So far I have:

import sys  
import re  
f = open(sys.argv[1])

strToSearch = ""

for line in f:
        strToSearch += line

patFinder1 = re.compile('DS\s+\d{4}|DS\s{2}\w\s{2}\w.*|DS\s{2}N', re.MULTILINE)

for i in findPat1:  
    print(i)

My output to screen looks like this:

DS user ID 1  
DS  N user name 1  
DS user ID 2  
DS  N user name 2   

If I write to file using:

outfile = "test.dat"   
FILE = open(outfile,"a")  
FILE.writelines(line)  
FILE.close()  

Everything is pushed to a single line:

DS user ID 1DS  N user name 1DS user ID 2DS  N user name 2 

I can live with the first scenario for the ouput. Ideally though I would like to strip the 'DS' and 'DS N' from the output file and have it comma separated.

User ID 1,user name 1  
User ID 2, username 2

Any ideas on how to get this accomplished?

Answer 1

It's difficult to provide a robust solution without understanding the actual input data format, how much flexibility is allowed, and how the parsed data is going to be used.

From just the sample input/output given above, one can quickly cook up a working sample code:

out = open("test.dat", "a") # output file

for line in open("input.dat"):
    if line[:3] != "DS ": continue # skip "random garbage"

    keys = line.split()[1:] # split, remove "DS"
    if keys[0] != "N": # found ID, print with comma
        out.write(" ".join(keys) + ",")
    else: # found name, print and end line
        out.write(" ".join(keys[1:]) + "\n")

Output file will be:

User ID 1,user name 1
User ID 2,user name 2

This code can of course be made much more robust using regex if the format specification is known. For example:

import re
pat_id = re.compile(r"DS\s+(User ID\s+\d+)")
pat_name = re.compile(r"DS\s+N\s+(.+\s+\d+)")
out = open("test.dat", "a")

for line in open("input.dat"):
    match = pat_id.match(line)
    if match: # found ID, print with comma
        out.write(match.group(1) + ",")
        continue
    match = pat_name.match(line)
    if match: # found name, print and end line
        out.write(match.group(1) + "\n")

Both the examples above assumes that "User ID X" always comes before "N user name X", hence the respective trailing chars of "," and "\n".

If the order is not specific, one can store the values in a dictionary using the numeric ID as a key then print out the ID/name pair after all input has been parsed.

If you provide more info, perhaps we can be of more help.

Answer 2

print adds a newline character after the arguments, but writelines does not. So you have to write like:

file = open(outfile, "a")
file.writelines((i + '\n' for i in findPat1))
file.close()

The writelines statement can also be written as:

for i in findPat1:
    file.write(i + '\n')