Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Creating interruptible process in python

I'm creating a python script of which parses a large (but simple) CSV.

It'll take some time to process. I would like the ability to interrupt the parsing of the CSV so I can continue at a later stage.

Currently I have this - of which lives in a larger class: (unfinished)

Edit:

I have some changed code. But the system will parse over 3 million rows.

def parseData(self)
    reader = csv.reader(open(self.file))
    for id, title, disc in reader:
        print "%-5s %-50s %s" % (id, title, disc)
        l = LegacyData()
        l.old_id = int(id)
        l.name = title
        l.disc_number = disc
        l.parsed = False
        l.save()

This is the old code.

def parseData(self):
        #first line start
        fields = self.data.next()
        for row in self.data:
            items = zip(fields, row)
            item = {}
            for (name, value) in items:
                item[name] = value.strip()
            self.save(item)

Thanks guys.

like image 692
Glycerine Avatar asked Sep 28 '26 04:09

Glycerine


1 Answers

If under linux, hit Ctrl-Z and stop the running process. Type "fg" to bring it back and start where you stopped it.

like image 104
user318904 Avatar answered Sep 30 '26 18:09

user318904