HTTP GET packet sniffer in Scapy

Question

I am trying to code a simple sniffer in Scapy, which only prints HTTP packets with GET method only. Here's the code:

#!/usr/bin/python
from scapy.all import *

def http_header(packet):
        http_packet=str(packet)
        if http_packet.find('GET'):
                print GET_print(packet)
        print packet
def GET_print(packet1):
        print "***************************************GET PACKET****************************************************"
        print packet1

        print "*****************************************************************************************************"


sniff(iface='eth0',prn=http_header)

Here is the output:

*****************************************************************************************************
None
T��Г
     )�pEa��@@���h��#/��t
                             �}LGku���U
oTE��I(��Ͻ�9qi���S��?��
                          XuW�F=���-�k=X:�
***************************************GET PACKET****************************************************
T��Г
     )�pE���@@���h��#/��t
                               ʪLGku����
oTE��I�K��AH�*�e��>�v1#D�(mG5T�o�?��8��喷╭���Ի�"�KT^�'�mB���]�����k>
                                                                                �_x�X�����8V?�Ǽw/�Z�=���N�À��\r�����)+}���l�c�9��j;���h��5�T�9Hۖ/O��)��P
         މY�qf爂�%�_`��6x��5D�I3���O�
t��tpI#�����$IC��E��
                     �G�
J��α���=�]��vһ���b5^|P��DK�)uq�2��ț�w�
                    tB������y=���n�i�r�.D6�kI�a���6iC���c'��0dPqED�4����[�[��hGh̃��~|Y/�>`\6yP  Dq١?T��Mѵ���f�;���Җ��Ǵ  gY���di�_x�8|
eo�p�xW9��=���vŅYe�}�T�ۨɑy�^�C
-�_(�<�{����}�������r
$��J�k-�9����}�Ϡf�27��QKԛ�`�GY�8��Sh���Y@8�E9�Rϔ�&a�/vkф��6�DF`�/9�I�d( ��-��[A
                                                                                     ��)pP��y\ռj]���8�_���vf�b����I7�������+�P<_`
*****************************************************************************************************

What I am expecting is:

GET / HTTP/1.1
    Host: google.com
    User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20140722 Firefox/24.0 Iceweasel/24.7.0
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Accept-Language: en-US,en;q=0.5
    Accept-Encoding: gzip, deflate
    Cookie: PREF=ID=758a20b5fbd4eac9:U=2b2dedf6c84b001f:FF=0:TM=1412150291:LM=1415430021:S=Q-QemmrLqsSsEA9i; NID=67=mRdkPVhtImrOTLi5I1e5JM22J7g26jAcdiDEjj9C5q0H5jj0DWRX27hCM7gLJBeiowW-8omSv-1ycH595SW2InWX2n1JMMNh6b6ZrRsZ9zOCC2a-vstOQnBDSJu6K9LO
    Connection: keep-alive

What can I do to get my expected output?

Answer 1

You need to use the sprintf function of the packet instead of printing the packet itself. You also need to split the string returned from it and join it back together with newline characters, otherwise it spits it out all on one line:

#!/usr/bin/python
from scapy.all import *

def http_header(packet):
        http_packet=str(packet)
        if http_packet.find('GET'):
                return GET_print(packet)

def GET_print(packet1):
    ret = "***************************************GET PACKET****************************************************\n"
    ret += "\n".join(packet1.sprintf("{Raw:%Raw.load%}\n").split(r"\r\n"))
    ret += "*****************************************************************************************************\n"
    return ret

sniff(iface='eth0', prn=http_header, filter="tcp port 80")

I also added a filter for TCP port 80, but this could be removed if you need to.

Example output:

***************************************GET PACKET****************************************************
'GET /projects/scapy/doc/usage.html HTTP/1.1
Host: www.secdev.org
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.65 Safari/537.36
Referer: https://www.google.co.uk/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6
If-None-Match: "28c84-48498d5654df67640-gzip"
If-Modified-Since: Mon, 19 Apr 2010 15:44:17 GMT

'
*****************************************************************************************************

Pierre points out that you can do away with the http_header function entirely by using the lfilter argument to sniff(). I took the liberty of making the code a little more succinct at the same time:

#!/usr/bin/python
from scapy.all import *

stars = lambda n: "*" * n

def GET_print(packet):
    return "\n".join((
        stars(40) + "GET PACKET" + stars(40),
        "\n".join(packet.sprintf("{Raw:%Raw.load%}").split(r"\r\n")),
        stars(90)))

sniff(
    iface='eth0',
    prn=GET_print,
    lfilter=lambda p: "GET" in str(p),
    filter="tcp port 80")

Answer 2

EDIT:

Please note that Scapy-http is now DEPRECATED and is included in Scapy 2.4.3+. Use import scapy.layers.http or load_layer("http") to enable it.

Answer:

There's a scapy http module that you can install by running pip install scapy-http. Once that is installed, you can import it by running import scapy_http.http. This is separate from your scapy module but adds functionality to scapy so you still need to import scapy as you usually would.

Once imported, change your filter line to

sniff(iface="eth0",
prn=GET_print,
lfilter= lambda x: x.haslayer(scapy_http.http.HTTPRequest))

I removed the filter="tcp and port 80" option because, using the http lfilter will return all HTTP Request queries regardless of port, except SSL for the obvious reason that it cannot be sniffed under the usual circumstances. You may want to keep the filter option for performance reasons.