Python Sniffing from Black Hat Python book

Question

import socket
import os
import struct
import sys
from ctypes import *

# host to listen on
host   = sys.argv[1]

class IP(Structure):

    _fields_ = [
        ("ihl",           c_ubyte, 4),
        ("version",       c_ubyte, 4),
        ("tos",           c_ubyte),
        ("len",           c_ushort),
        ("id",            c_ushort),
        ("offset",        c_ushort),
        ("ttl",           c_ubyte),
        ("protocol_num",  c_ubyte),
        ("sum",           c_ushort),
        ("src",           c_ulong),
        ("dst",           c_ulong)
    ]

    def __new__(self, socket_buffer=None):
        return self.from_buffer_copy(socket_buffer)    

    def __init__(self, socket_buffer=None):

        # map protocol constants to their names
        self.protocol_map = {1:"ICMP", 6:"TCP", 17:"UDP"}

        # human readable IP addresses
        self.src_address = socket.inet_ntoa(struct.pack("<L",self.src))
        self.dst_address = socket.inet_ntoa(struct.pack("<L",self.dst))

        # human readable protocol
        try:
            self.protocol = self.protocol_map[self.protocol_num]
        except:
            self.protocol = str(self.protocol_num)

# create a raw socket and bind it to the public interface
if os.name == "nt":
    socket_protocol = socket.IPPROTO_IP 
else:
    socket_protocol = socket.IPPROTO_ICMP

sniffer = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket_protocol)

sniffer.bind((host, 0))

# we want the IP headers included in the capture
sniffer.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)

# if we're on Windows we need to send some ioctls
# to setup promiscuous mode
if os.name == "nt":
    sniffer.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)

try:
    while True:

        # read in a single packet
        raw_buffer = sniffer.recvfrom(65565)[0]

        # create an IP header from the first 20 bytes of the buffer
        ip_header = IP(raw_buffer[0:20])

        print "Protocol: %s %s -> %s" % (ip_header.protocol, ip_header.src_address, ip_header.dst_address)

except KeyboardInterrupt:
    # if we're on Windows turn off promiscuous mode
    if os.name == "nt":
        sniffer.ioctl(socket.SIO_RCVALL, socket.RCVALL_OFF)

This is code from the book Black Hat Python. This code is supposed to sniff with raw sockets and display information from the IP header. It works fine for me on Windows (using Windows 8.1 64bit). When I attempt to run this on linux (Kali linux 1.1.0-amd64) I get the following error

ValueError: Buffer size too small (20 instead of at least 32 bytes)

To get around that I added 12 spaces to the buffer like this

ip_header = IP(raw_buffer[0:20]+' '*12)

When I do that I get the following error

struct.error: 'L' format requires 0 <= number <= 4294967295

This occurs on the line

self.src_address = socket.inet_ntoa(struct.pack("<L",self.src))

I have tried changing the symbol before the L to > and ! and I tried it with just L all of them give me the same issue. I also tried wrapping self.src in ntohs like so

self.src_address = socket.inet_ntoa(struct.pack("<L",socket.ntohs(self.src)))

I'm thinking this has something to do with endianness but I'm not sure. Any help would be greatly appreciated.

NOTE: On windows you have to run as an administrator and on linux you have to run as a superuser because of the raw sockets. If you run this on linux open up another terminal and ping www.google.com so you can generate some ICMP packets for it to capture.

EDIT: I have also tried reversing the buffer with

ip_header = IP(raw_buffer[0:20][::-1]+' '*12)

EDIT 2: I did try both 65535 and 65534 on the below line before doing any of the other items I listed here.

raw_buffer = sniffer.recvfrom(65565)[0]

EDIT 3: This worked on an ubuntu machine running python 2.7.6 and my kali distro was 2.7.3 so I decided to get the latest version of python on my kali box which happens to be 2.7.9. Still no luck.

I placed the following code to the new function in my structure to view the buffer size

print sizeof(self)

On my Ubuntu and windows machines it was 20 however on my kali machine it was 32

Answer 1

#raw_buffer = sniffer.recvfrom(65565)[0]
raw_buffer = sniffer.recvfrom(65535)[0]

IP paket size is (2^16) - 1

The problem is with 32 vs 64 bit systems.
ip_header = IP(raw_buffer[:20]) works on x86 Ubuntu.
ip_header = IP(raw_buffer[:32]) works on amd64 CentOS 6.6 Python 2.6.6
ip_header = IP(raw_buffer) works in both.

You have to change these,

("src",           c_ulong),
("dst",           c_ulong)  

self.src_address = socket.inet_ntoa(struct.pack("<L",self.src))
self.dst_address = socket.inet_ntoa(struct.pack("<L",self.dst))

into

("src",           c_uint32),
("dst",           c_uint32)  

self.src_address = socket.inet_ntoa(struct.pack("@I",self.src))
self.dst_address = socket.inet_ntoa(struct.pack("@I",self.dst))

'@I' is unisigned int in native order. because c_ulong is 4 bytes in i386 and 8 in amd64. Check the following,

struct.calcsize('@BBHHHBBHLL')  

is 20 in i386 and 32 in amd64 which is size of _fields_. In actual it's 28 bytes in amd64 plus 4 bytes padded for word alignment.

ip_header = IP(raw_buffer[:20]) now works correctly independent of platforms.

Answer 2

So it's a 64/32 bit problem. The fact that it needed 32 bytes instead of 20 means that the struct was not packed correctly. "c_ulong" is 64 bits in 64 bit linux, and was being mapped that way in the "IP" class.

The IP header is 20 bytes + optional fields. The source and destination ip addresses end by byte 20, which is what the current IP structure is picking up. (if you want the options, you're going to have to parse those out by hand).

I looked up the UDP bit fields and directly set them into the class "IP". Looking at the ctypes docs, integer types can be mapped to limit the number of bits.

class IP(Structure):

    _fields_ = [
        ("ihl",           c_ubyte, 4),
        ("version",       c_ubyte, 4),
        ("tos",           c_ubyte, 8),
        ("len",           c_ushort, 16),
        ("id",            c_ushort, 16),
        ("offset",        c_ushort, 16),
        ("ttl",           c_ubyte, 8),
        ("protocol_num",  c_ubyte, 8),
        ("sum",           c_ushort, 16),
        ("src",           c_uint, 32),
        ("dst",           c_uint, 32),
    ]

If you sum the bit offsets, they sum to 160. 160/8 = 20 bytes, which is what ctypes packs this struct to.

Running that on a ping yields something that looks acceptable.

Protocol: ICMP 127.0.0.1 -> 127.0.0.1
Protocol: ICMP 127.0.0.1 -> 127.0.0.1
Protocol: ICMP 127.0.0.1 -> 127.0.0.1
Protocol: ICMP 127.0.0.1 -> 127.0.0.1

Further, packet size is a function of MTU (or Maximum Transfer Unit), so if you plan on running this on ethernet, the limiting factor is the MTU of the frame. Larger packets will get fragmented in the tcp/ip stack before being pushed out of the ethernet port.

$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:00:00:ff:ff:ff  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1

Also, this question should help clarify the issue on why some platforms have different sized ints and longs:

What is the bit size of long on 64-bit Windows?

As an alternative, I have found that dpkt is a rather good library for decoding/encoding ip packets, unless you specifically need to use or want ctypes.

https://code.google.com/p/dpkt/