How does Android VpnService protect fd work?

Question

According to https://developer.android.com/reference/android/net/VpnService.html#protect(int)

After protecting a socket from VPN connections, data sent through this socket will go directly to the underlying network, so its traffic will not be forwarded through the VPN. This method is useful if some connections need to be kept outside of VPN. For example, a VPN tunnel should protect itself if its destination is covered by VPN routes. Otherwise its outgoing packets will be sent back to the VPN interface and cause an infinite loop. This method will fail if the application is not prepared or is revoked.

I know Android is built upon Linux, what Linux mechanisms or utils are used behind this function, to make just data sent through this socket bypass VPN, but all others go through VPN?

Answer 1

In short, Android VPNService protecting fd working by policy routing, all packets go through protected fd will be marked with a special fwmark, all packets with this fwmark will bypass VPN.

Key code snippets listed as below:

// android/frameworks/base/core/java/android/net/VpnService.java

 /**
     * Protect a socket from VPN connections. After protecting, data sent
     * through this socket will go directly to the underlying network,
     * so its traffic will not be forwarded through the VPN.
     * This method is useful if some connections need to be kept
     * outside of VPN. For example, a VPN tunnel should protect itself if its
     * destination is covered by VPN routes. Otherwise its outgoing packets
     * will be sent back to the VPN interface and cause an infinite loop. This
     * method will fail if the application is not prepared or is revoked.
     *
     * <p class="note">The socket is NOT closed by this method.
     *
     * @return {@code true} on success.
     */
    public boolean protect(int socket) {
        return NetworkUtils.protectFromVpn(socket);
    }


// android/frameworks/base/core/java/android/net/VpnService.java

    /**
     * Protect {@code fd} from VPN connections.  After protecting, data sent through
     * this socket will go directly to the underlying network, so its traffic will not be
     * forwarded through the VPN.
     */
    public static boolean protectFromVpn(FileDescriptor fd) {
        return protectFromVpn(fd.getInt$());
    }

// android/system/netd/server/FwmarkServer.cpp

    fwmark.permission = permission;

    if (setsockopt(*socketFd, SOL_SOCKET, SO_MARK, &fwmark.intValue,
                   sizeof(fwmark.intValue)) == -1) {
        return -errno;
    }


// android/system/netd/include/Fwmark.h 

union Fwmark {
    uint32_t intValue;
    struct {
        unsigned netId          : 16;
        bool explicitlySelected :  1;
        bool protectedFromVpn   :  1;
        Permission permission   :  2;
    };
    Fwmark() : intValue(0) {}
};

static const unsigned FWMARK_NET_ID_MASK = 0xffff;

And an example of routing policy, after an app with VPN service turned on:

root@CP8692:/ # ip rule
0:  from all lookup local 
10000:  from all fwmark 0xc0000/0xd0000 lookup legacy_system 
11000:  from all iif tun0 lookup local_network 
12000:  from all fwmark 0xc0072/0xcffff lookup tun0 
12000:  from all fwmark 0x0/0x20000 uidrange 0-99999 lookup tun0 
13000:  from all fwmark 0x10063/0x1ffff lookup local_network 
13000:  from all fwmark 0x10071/0x1ffff lookup wlan0 
13000:  from all fwmark 0x10072/0x1ffff uidrange 0-0 lookup tun0 
13000:  from all fwmark 0x10072/0x1ffff uidrange 0-99999 lookup tun0 
14000:  from all oif wlan0 lookup wlan0 
14000:  from all oif tun0 uidrange 0-99999 lookup tun0 
15000:  from all fwmark 0x0/0x10000 lookup legacy_system 
16000:  from all fwmark 0x0/0x10000 lookup legacy_network 
17000:  from all fwmark 0x0/0x10000 lookup local_network 
19000:  from all fwmark 0x71/0x1ffff lookup wlan0 
21000:  from all fwmark 0x72/0x1ffff lookup wlan0 
22000:  from all fwmark 0x0/0xffff lookup wlan0 
23000:  from all fwmark 0x0/0xffff uidrange 0-0 lookup main 
32000:  from all unreachable