EPFImporter creates table but imports nothing

Question

I'm using Apple's EPFImporter tool http://www.apple.com/itunes/affiliates/resources/documentation/epfimporter.html

It's a Python script that will take space separated EPF file lists and import them into my database.

Here's what I have:

Braden-Keiths-MacBook-Pro:~ bradenkeith$ ./EPFImporter.py /Users/bradenkeith/Downloads/popularity20120314 

Here's what the CLI returns:

2012-03-14 22:12:28,748 [INFO]: Beginning import for the following directories:
    /Users/bradenkeith/Downloads/popularity20120314 
2012-03-14 22:12:28,748 [INFO]: Importing files in /Users/bradenkeith/Downloads/popularity20120314 
2012-03-14 22:12:28,749 [INFO]: Starting import of /Users/bradenkeith/Downloads/popularity20120314... 
2012-03-14 22:12:28,749 [INFO]: Beginning full ingest of epf_application_popularity_per_genre (2000491 records) 
2012-03-14 22:14:28,774 [INFO]: ...at record 1797000... 
2012-03-14 22:16:02,152 [INFO]: Full ingest of epf_application_popularity_per_genre took 0:03:33.402408 
2012-03-14 22:16:02,196 [INFO]: Import of popularity20120314 completed at: 12-03-14 22:16:02 
2012-03-14 22:16:02,196 [INFO]: Total import time for popularity20120314: 0:03:33.44 
2012-03-14 22:16:02,196 [INFO]: Total import time for all directories: 0:03:33.44

The tool was capable of creating the database. It just won't add any of the entries to the database. It obviously sees 2mil+ records, and spends the time combing through them... deleting and merging the blank tables... but it's just that - tables are still blank. I thought possibly it's a permissions thing with mySQL. I double checked and made sure everything was granted to the user account I was using. Still nothing.

Any ideas of what this might could be?

Answer 1

You can get it to work by altering EPFIngester.py according to:

1. Find function

   def _populateTable(self, tableName, resumeNum=0,
   isIncremental=False, skipKeyViolators=False):
   

2. In the function, within the while loop, find row:

   cur = conn.cursor()
   

3. Under it insert:

   cur.connection.autocommit(True)
   

The altered source should look like:

...
cur = conn.cursor()
cur.connection.autocommit(True)
colVals = unicode(", ".join(stringList), 'utf-8')
....

Answer 2

The EPFImporter was made in 2010. At the time, the latest version of MySQLdb set autocommit to true. The version of MySQLdb you are using is most likely a newer version where autocommit is set to false.