Python Scapy wrpcap - How do you append packets to a pcap file?

Question

I have some software that can emulate things like BER and delays on the network. I need a way to test the BER module of the software to make sure it actually works correctly. My solution is to create a program that sends out raw Ethernet frames with the type field set to an unused type. Inside the Ethernet frame is just random bits. For each frame sent out I need to log the frame to a pcap file. On the other side of the network link will be a receiving application that simply writes every packet it sees to its own pcap log. After the test is done running the two pcap logs will be compared to get the BER.

I'm using the python module Scapy and so far its done everything that I need. I can send out raw Ethernet frames with random data and see them in Wireshark. However, I don't know how to get the wrpcap() method to append to the pcap file, instead of overwriting. I know I can write a list of packets to wrpcap, but this application needs to be able to run for an indefinite amount of time and I don't want to have to wait until the application quits to write all of packets sent to the hard drive. As that would be a lot to store in memory, and if something happened I would have to start the test all over from scratch.

My question is: How do I append to a pcap file using scapy instead of overwriting the pcap file? Is it even possible? If not then what module can do what I need?

While looking for something with Scapy's capabilities I ran into dpkt, but I didn't find a lot of documentation for it. Can dpkt do what I'm asking and if so where can I get some good documentation for it?

Answer 1

For posterity, PcapWriter or RawPcapWriter looks to be the easier way to deal with this in scapy 2.2.0. Couldn't find much documentation other than browsing the source though. A brief example:

from scapy.utils import PcapWriter

pktdump = PcapWriter("banana.pcap", append=True, sync=True)

...
pktdump.write(pkt)
...

Answer 2

There is a way to do what you want, but it means either:

• [Memory hog with one big pcap]: Read the existing pcap from disk with rdpcap() into a scapy PacketList() and then writing frames to the PacketList as they are received. You can selectively save intermediate PacketList to the pcap at will, but I don't think there is anything like an append capability in scapy's wrpcap(). As you mentioned, this technique also means that you are keeping the entire PacketList in memory until completion.

• [Glue individual pcap files together]: Only keep small snapshots of packets in memory... you should save pcap snapshots on a per-X-minute basis to disk, and then aggregate those individual files together when the script finishes.

You can combine pcap files in linux with mergecap from the wireshark package... The following command will combine pak1.pcap and pak2.pcap into all_paks.pcap:

mergecap -w all_paks.pcap pak1.pcap pak2.pcap

As for dpkt, I looked through their source, and it might be able to incrementally write packets, but I can't speak for how stable or maintained their code base is... it looks a bit neglected from the commit logs (last commit was January 9th 2011).