Scapy python script consuming too much cpu while sniffing

Question

I have coded a wireless probe sniffer using python + scapy. I'm want to use this script in openwrt routers.

Everytime it captures a probe request from nearby devices, the information is send to a webservice. (mac, power and probe).

My problem is the high consumption of CPU. The script runs quite good in my laptop (it takes between 50-70% of cpu), but when I run it in an openwrt router (400mhz cpu,16 ram) it takes 99%. It's a well known bug in scapy the lost of packets with high loads (I have tested at the same time the script in my laptop and in the router and the router do not catched all the available packets in the air)

I already made some optimizations to the code, but I think there's more room for improvement.

This is the script.

#!/usr/bin/python
from scapy.all import *
import time
import thread
import requests
from datetime import datetime

PROBE_REQUEST_TYPE=0
PROBE_REQUEST_SUBTYPE=4
buf={'arrival':0,'source':0,'dest':0,'pwr':0,'probe':0}
uuid='1A2B3'

def PacketHandler(pkt):
    global buf
    if pkt.haslayer(Dot11):
        if pkt.type==PROBE_REQUEST_TYPE and pkt.subtype == PROBE_REQUEST_SUBTYPE:
            arrival= int(time.mktime(time.localtime()))
            try:
                extra = pkt.notdecoded
            except:
                extra=None

            if extra!=None:
                signal_strength = -(256-ord(extra[-4:-3]))
            else:
                signal_strength = -100

            source = pkt.addr2
            dest= pkt.addr3
            pwr=signal_strength
            probe=pkt.getlayer(Dot11).info

            if buf['source']!=source and buf['probe']!=probe:
                print 'launch %r %r %r' % (source,dest,probe)
                buf={'arrival':arrival,'source':source,'dest':dest,'pwr':pwr,'probe':probe}

                try:
                    thread.start_new_thread(exporter,(arrival,source,dest,pwr,probe))
                except:
                    print 'Error launching the thread %r' % source

def exporter (arrival,source,dest,pwr,probe):
    global uuid
    urlg='http://webservice.com/?arrival='+str(arrival)+'&source='+str(source)+'&dest='+str(dest)+'&pwr='+str(pwr)+'&probe='+str(probe)+'&uuid='+uuid
    try:
        r=requests.get(urlg)
        print r.status_code
        print r.content
    except:
        print 'ERROR in Thread:::::: %r' % source







def main():
    print "[%s] Starting scan"%datetime.now()
    sniff(iface=sys.argv[1],prn=PacketHandler,store=0)

if __name__=="__main__":
    main()

[UPDATE]

After a lot of reading and deep searching (it seems not many people have found full solution to the same issue or something similar). I have found you can filter directly from the sniff function, so I've added a filter to just catch probe requests.

def main():
    print "[%s] Starting scan"%datetime.now()
    sniff(iface=sys.argv[1],prn=PacketHandler, filter='link[26] = 0x40',store=0)

In my laptop runs really smooth, using between 1%-3% of cpu and catching most of the available packets in the air.

but when I run this on the router, the script throws an error and crash.

Traceback (most recent call last):
  File "snrV2.py", line 66, in <module>
    main()
  File "snrV2.py", line 63, in main
    sniff(iface=sys.argv[1],prn=PacketHandler, filter='link[26] = 0x40',store=0)
  File "/usr/lib/python2.7/site-packages/scapy/sendrecv.py", line 550, in sniff
    s = L2socket(type=ETH_P_ALL, *arg, **karg)
  File "/usr/lib/python2.7/site-packages/scapy/arch/linux.py", line 460, in __init__
    attach_filter(self.ins, filter)
  File "/usr/lib/python2.7/site-packages/scapy/arch/linux.py", line 132, in attach_filter
    s.setsockopt(SOL_SOCKET, SO_ATTACH_FILTER, bpfh)
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 99] Protocol not available

I have tried using bpf filter syntax (the same used in tcpdump http://biot.com/capstats/bpf.html) and it is supposed you can also use it in scapy but I get a filter syntax error.

Sniff fucntion:

def main():                                                                                                                                           
    print "[%s] Starting scan"%datetime.now()                                                                                                         
    sniff(iface=sys.argv[1],prn=PacketHandler, filter='type mgt subtype probe-req', store=0) 

error:

Traceback (most recent call last):
  File "snrV2.py", line 66, in <module>
    main()
  File "snrV2.py", line 63, in main
    sniff(iface=sys.argv[1],prn=PacketHandler, filter='type mgt subtype probe-req', store=0)
  File "/usr/lib/python2.7/site-packages/scapy/sendrecv.py", line 550, in sniff
    s = L2socket(type=ETH_P_ALL, *arg, **karg)
  File "/usr/lib/python2.7/site-packages/scapy/arch/linux.py", line 460, in __init__
    attach_filter(self.ins, filter)
  File "/usr/lib/python2.7/site-packages/scapy/arch/linux.py", line 120, in attach_filter
    raise Scapy_Exception("Filter parse error")
NameError: global name 'Scapy_Exception' is not defined

In the router I have installed the last version of scapy and tcpdump. Now I really don't know what to do.

Answer 1

I encountered a similar error (socket.error: [Errno 99] Protocol not available) when I tried to use sniff() with a filter on my NETGEAR WNDR4300.

After a lot of searching on Google, I found that the reason is that the Linux kernel of my router does not enable CONFIG_PACKET. It is mentioned in the Scapy installation guide, as follows:

• Make sure your kernel has Packet sockets selected (CONFIG_PACKET)
• If your kernel is < 2.6, make sure that Socket filtering is selected CONFIG_FILTER)

If you set CONFIG_PACKET=y when you compile the kernel, then it will enable the BPF for underlying socket.