NEPacketTunnelProvider Sniffer iOS

Question

As I recently found this paper describing a sniffing mechanism for iOS using Apple's NEPacketTunnelProvider Extension, I got curious and it made me want to understand it from a technical point of view. As I usually don't work at a deep network layer like that, I'm not able to comprehend it in the detail I'd like to. As Charles Proxy for iOS must do something very similar without requiring supervised devices, I assume the approach which the author of the paper presented in 2016 might be still working nowadays.

The author claimed that "Everything like IP packet parsing, building an IP packet or parsing a DNS response had to be implemented ourselves." As I want to fully understand that, I tried to build it myself. I build a NetworkExtension and a message loop for the packetFlow of the NEPacketTunnelProvider. I was able to obtain the ip datagrams and tried to parse them. I used unsigned integers of the corresponding size for the source and target ip, the transport protocol and ip version, but I'm unsure how to handle the treat the payload. My parser uses the ptr.load(fromByteOffset: <offset>, as:<DataType>.self) where ptr is a UnsafeRawPointer to access the packet flow information. Since the data might exceed the storage of UInt64, I don't know how to access and store the payload in a proper way.

Furthermore, I figured that the source IP is always 192.168.20.1 (set as my interface's NEIPv4Settings address) and my target ip is always 192.168.2.1 (my dummy NEDNSSettings server). This leads me to my first questions: Are those DNS queries? Will the datagram packet claim any further information about the actual target? Would that mean that I have to somehow execute the request to the DNS server and reroute the packet to the target which I will obtain from that DNS query?

The next step would be to implement a TCP / UDP handling, right? My current parsing approach is able to distinguish between UDP, TCP and ICMP (even though I don't have investigated in the last one yet). Therefore, I'd iterate over the datagrams and lookup whether they require a UPD or TCP session/connection and transfer the datagram. The problem I currently see their from a conceptional point of view: How do I know which source/target port to use for TCP/UPD connections/sessions? As far as I know, this information is not part of the IP Packet itself (since it's rather some information we need on transport layer level, not on network layer level).

Additionally, I found a project called Specht on github. It uses a self-written library called NEKit which somehow also uses the NEPacketTunnelProvider approach. When I understand their approach correctly, they managed to somehow build a local proxy server by writing some observer mechanisms in order to handle the requests, but since I'm relatively new to networking and swift, I'm not sure whether I understand that completely correct or whether I just haven't find all those TCP/UDP and/or DNS logic. Is this project comparable to the approach of the paper and charles proxy?

One last question: Charles proxy is in most cases able to show the hostname of the target. I'm currently just able to see destination ip addresses (which aren't real destination ip addresses, but the address of my DNS server). How am I able to see the hostname as human readable text? Does Charles do a nslookup somehow? Does Charles obtain that information out of the datagrams?

I know it's quite ambitious of me with much missing knowledge in this topic, to build something similar for test reasons, but I'm still motivated to look deeper into that topic and also have the feeling that I have understand already some key points, but unfortunately not enough to solve the puzzle... Maybe you're able to give me some more hints to get a better understanding. If there might be even an easier way to archive a similar behavior (to see outgoing connections on hostname level), I'd be interested in these as well :-)

Answer 1

I've published a Beta Proxyman iOS (website) - a Network Sniffer by using NEPacketTunnelProvider and Network Extension, so I might have experienced to answer some of your questions.

IP Package, IP Diagram, DNS, How to parse it?

Luckily, there is another way to set up a NEPacketTunnelProvider to provide you with an HTTP Message, not IP Package (it's too low-level, and you have to deal with the Parser, DNS, ...)

HTTP Message is easier to parse because there are plenty of reliable libraries (e.g. http-parser from nodeJS)

1. How to build a Network Sniffer on iOS?

It's a complicated question to answer, I would break it into small chunks:

MitM / Proxy Server

Firstly, you need a working MitM Proxy Server, which is capable of proxying and intercepting the HTTP/HTTPS Traffic. You can implement it by using SwiftNIO or CocoaAsyncSocket.

How does it work?

In general, the data flow might look like this:

The Internet -> iPhone -> Your Network Extension (VPN) -> Forward to your Local Proxy Server (in the Network Extension) -> Mitm/Proxy Server starts intercepting or monitoring the traffic -> Save to a local database (in Shared Container Group) -> Forward again to the destination server.

From the main app, you can receive the data by reading the local database.

The reason why we need a local database is that the Network Extension and the Main app are two different processes, so they could not communicate directly like a normal app.

Show me the code?

In the Network extension, let start a Proxy Server at Host:Port, then init the NetworkSetting, like the sample:

    private func initTunnelSettings(proxyHost: String, proxyPort: Int) -> NEPacketTunnelNetworkSettings {
    let settings: NEPacketTunnelNetworkSettings = NEPacketTunnelNetworkSettings(tunnelRemoteAddress: "127.0.0.1")

    /* proxy settings */
    let proxySettings: NEProxySettings = NEProxySettings()
    proxySettings.httpServer = NEProxyServer(
        address: proxyHost,
        port: proxyPort
    )
    proxySettings.httpsServer = NEProxyServer(
        address: proxyHost,
        port: proxyPort
    )
    proxySettings.autoProxyConfigurationEnabled = false
    proxySettings.httpEnabled = true
    proxySettings.httpsEnabled = true
    proxySettings.excludeSimpleHostnames = true
    proxySettings.exceptionList = [
        "192.168.0.0/16",
        "10.0.0.0/8",
        "172.16.0.0/12",
        "127.0.0.1",
        "localhost",
        "*.local"
    ]
    settings.proxySettings = proxySettings

    /* ipv4 settings */
    let ipv4Settings: NEIPv4Settings = NEIPv4Settings(
        addresses: [settings.tunnelRemoteAddress],
        subnetMasks: ["255.255.255.255"]
    )
    ipv4Settings.includedRoutes = [NEIPv4Route.default()]
    ipv4Settings.excludedRoutes = [
        NEIPv4Route(destinationAddress: "192.168.0.0", subnetMask: "255.255.0.0"),
        NEIPv4Route(destinationAddress: "10.0.0.0", subnetMask: "255.0.0.0"),
        NEIPv4Route(destinationAddress: "172.16.0.0", subnetMask: "255.240.0.0")
    ]
    settings.ipv4Settings = ipv4Settings

    /* MTU */
    settings.mtu = 1500

    return settings
}

Then start a VPN,

let networkSettings = initTunnelSettings(proxyHost: ip, proxyPort: port)

// Start
setTunnelNetworkSettings(networkSettings) { // Handle success }

Then forward the package to your local proxy server:

let endpoint = NWHostEndpoint(hostname: proxyIP, port: proxyPort)
self.connection = self.createTCPConnection(to: endpoint, enableTLS: false, tlsParameters: nil, delegate: nil)

    packetFlow.readPackets {[weak self] (packets, protocols) in
        guard let strongSelf = self else { return }
        for packet in packets {
            strongSelf.connection.write(packet, completionHandler: { (error) in
            })
        }

        // Repeat
        strongSelf.readPackets()
    }

From that, your local server can receive the packages then forwarding to the destination server.

Don't forget to save all traffic log to the local database, then notifying the main app to reload it.

One last question: Charles proxy is in most cases able to show the hostname of the target. I'm currently just able to see destination ip addresses (which aren't real destination ip addresses, but the address of my DNS server). How am I able to see the hostname as human readable text? Does Charles do a nslookup somehow? Does Charles obtain that information out of the datagrams?

Since we don't deal with IP Package, we don't need to implement the DNS Resolver. If you need a DNS, you can config like the following code:

        let dnsSettings = NEDNSSettings(servers: ["8.8.8.8", "1.1.1.1"])
    settings.dnsSettings = dnsSettings

As we receive the HTTP Message package, you can get hostname for free (From the Request's URL or Host Header)

Hope that my answer could help you.