Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

openpyxl writing large file memory issue

I'm trying to create a 75col by 650k rows document using openpyxl write only workbook, which is said to bear near constant memory footprint, but after a while I get 17.2GB memory usage in activity monitor, here's the code I'm using, am I doing something wrong?

def testOPENPYXL():
    wb = openpyxl.Workbook(write_only=True)
    ws = wb.create_sheet()
    for irow in range(650000):
        ws.append(['%d' % i for i in range(75)])
    path = os.path.expanduser("~/Desktop/test/test.xlsx")
    wb.save(path)
like image 909
Eugene Avatar asked Dec 08 '16 16:12

Eugene


1 Answers

The simple solution is to install lxml we have a shim in openpyxl that mimics lxml's streaming writer but it isn't as memory efficient.

like image 183
Charlie Clark Avatar answered Sep 20 '22 01:09

Charlie Clark