Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Speed up python's struct.unpack

Tags:

I am trying to speed up my script. It basically reads a pcap file with Velodyne's Lidar HDL-32 information and allows me to get X, Y, Z, and Intensity values. I have profiled my script using python -m cProfile ./spTestPcapToLas.py and it is spending the most amount of time in my readDataPacket() function calls. In a small test (80 MB file) the unpacking portion takes around 56% of the execution time.

I call the readDataPacket function like this (chunk refers to the pcap file):

packets = []
for packet in chunk:
    memoryView = memoryview(packet.raw())
    udpDestinationPort = unpack('!h', memoryView[36:38].tobytes())[0]

    if udpDestinationPort == 2368:
        packets += readDataPacket(memoryView)

The readDataPacket() function itself is defined like this:

def readDataPacket(memoryView):
    firingData = memoryView[42:]    
    firingDataStartingByte = 0    
    laserBlock = []

    for i in xrange(firingBlocks):
        rotational = unpack('<H', firingData[firingDataStartingByte+2:firingDataStartingByte+4])[0]        
        startingByte = firingDataStartingByte+4
        laser = []
        for j in xrange(lasers):   
            distanceInformation = unpack('<H', firingData[startingByte:(startingByte + 2)])[0] * 0.002
            intensity = unpack('<B', firingData[(startingByte + 2)])[0]   
            laser.append([distanceInformation, intensity])
            startingByte += 3
        firingDataStartingByte += 100
        laserBlock.append([rotational, laser])

    return laserBlock

Any ideas on how I can speed up the process? By the way, I am using numpy for the X, Y, Z, Intensity calculations.