How to delete specific strings from a file?

Question

I have a data file (unstructured, messy file) from which I have to scrub specific list of strings (delete strings).

Here is what I am doing but with no result:

infile = r"messy_data_file.txt"
outfile = r"cleaned_file.txt"

delete_list = ["firstname1 lastname1","firstname2 lastname2"....,"firstnamen lastnamen"]
fin=open(infile,"")
fout = open(outfile,"w+")
for line in fin:
    for word in delete_list:
        line = line.replace(word, "")
    fout.write(line)
fin.close()
fout.close()

When I execute the file, I get the following error:

NameError: name 'word' is not defined

Answer 1

The readlines method returns a list of lines, not words, so your code would only work where one of your words is on a line by itself.

Since files are iterators over lines this can be done much easier:

infile = "messy_data_file.txt"
outfile = "cleaned_file.txt"

delete_list = ["word_1", "word_2", "word_n"]
with open(infile) as fin, open(outfile, "w+") as fout:
    for line in fin:
        for word in delete_list:
            line = line.replace(word, "")
        fout.write(line)

Answer 2

To remove the string within the same file, I used this code

f = open('./test.txt','r')
a = ['word1','word2','word3']
lst = []
for line in f:
    for word in a:
        if word in line:
            line = line.replace(word,'')
    lst.append(line)
f.close()
f = open('./test.txt','w')
for line in lst:
    f.write(line)
f.close()

Answer 3

Based on your comment "I am double clicking the .py file. It seems to invoke the python application which disappears after a couple of seconds. I dont get any error thought" I believe your issue is the script is not finding the input file. That is also why you are not getting any output. When you double click on it... I actually can't recall where the interpreter is going to look but I think it's where the python.exe is installed.

Use a fully qualified path like so.

# Depends on your OS
infile = r"C:\tmp\messy_data_file.txt"
outfile = r"C:\tmp\cleaned_file.txt"

infile = r"/etc/tmp/messy_data_file.txt"
outfile = r"/etc/tmp/cleaned_file.txt"

Also, for your sanity, run it from the command-line instead of double clicking. It'll be much easier to catch errors/output.