My Python for loop is causing a MemoryError. How can I optimize this?

Question

I'm trying to compile a list of all the MAC address Apple devices will have. oui.txt tells me Apple has been assigned 77 MAC ranges to use. These ranges come in the form of:

00:00:00
00:11:11
etc...

This leaves me the last three HEX digits to append. That's 16^6. A total of 1291845632 Apple MAC addresses.

The problem I'm having is writing a program to create a list of these MAC addresses. Here's my current code:

import re

apple_mac_range = []
apple_macs      = []

# Parse the HTML of http://standards.ieee.org/cgi-bin/ouisearch to get the MACs
with open('apple mac list', 'r') as f:
    for line in f.readlines():

        match = re.search(r'[\w\d]{2}-[\w\d]{2}-[\w\d]{2}', line)

        if match:
            apple_mac_range.append(match.group().split('-'))

for mac in apple_mac_range:
    for i in range(1, 1291845633):
        print i

This gives me a MemoryError... How can I optimize it?

Answer 1

range(1, 1291845633) creates a list of 1,291,845,632 elements (several GB) all at once. Use xrange(1, 1291845633) instead and it will generate elements as you need them instead of all at once.

Regardless, it looks like you want something more like this:

for mac in apple_mac_range: 
    for i in xrange(16777216): 
        print mac, i 

Of course it's quite likely that a list of 1.3e+9 MAC addresses will not be very useful. If you want to see if a given MAC address is an Apple device, you should just check to see if the 3-byte prefix is in the list of 77. If you're trying to do access control by giving a router or something a list of all possible MAC addresses, it's unlikely that the device will accept 1.3e+9 items in its list.