How to calculate HttpWebRequest spent outbound and inbound internet traffic

Question

I have the below function to fetch a page. My question is i want to calculate how much internet connection is spent

Both inbound (download) and outbound traffic (sent)

How can i do that ? Thank you

My function

 public static string func_fetch_Page(string srUrl, int irTimeOut = 60,
    string srRequestUserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0",
    string srProxy = null)
    {
        string srBody = "";
        string srResult = "";
        try
        {
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(srUrl);

            request.Timeout = irTimeOut * 1000;
            request.UserAgent = srRequestUserAgent;
            request.KeepAlive = true;
            request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";

            WebHeaderCollection myWebHeaderCollection = request.Headers;
            myWebHeaderCollection.Add("Accept-Language", "en-gb,en;q=0.5");
            myWebHeaderCollection.Add("Accept-Encoding", "gzip, deflate");

            request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;

            using (WebResponse response = request.GetResponse())
            {
                using (Stream strumien = response.GetResponseStream())
                {
                    using (StreamReader sr = new StreamReader(strumien))
                    {
                        srBody = sr.ReadToEnd();
                        srResult = "success";
                    }
                }
            }
        }
        catch ()
        {

        }

        return srBody;
    }

C# .net 4.5 WPF application

@Simon Mourier how do i calculate spent traffic

public static long long_GlobalDownload_KByte = 0;
public static long long_GlobalSent_KByte = 0;

Time  _timer_fetch_download_upload = new Timer(getDownload_Upload_Values, null, 0, 100 * 1000);

public static void getDownload_Upload_Values(Object state)
{
    using (Process p = Process.GetCurrentProcess())
    {
        foreach (var cnx in TcpConnection.GetAll().Where(c => c.ProcessId == p.Id))
        {
            Interlocked.Add(ref long_GlobalDownload_KByte, Convert.ToInt64(cnx.DataBytesIn));
            Interlocked.Add(ref long_GlobalSent_KByte, Convert.ToInt64(cnx.DataBytesOut));
        }
    }
}

Answer 1

One Windows API can give you this information: GetPerTcpConnectionEStats for IPV4 connections and the associated IPV6 GetPerTcp6ConnectionEStats function. Note you need to use SetPerTcpConnectionEStats before being able to get any stat, and that usually needs admin rights...

To get the list of all connections, you can use the GetExtendedTcpTable function. It can also give you the connection's process id which is very useful.

These are native APIs not that easy to use, but I have created a TcpConnection class that wraps all this. It's available here in a small WPF app called IPStats: https://github.com/smourier/IPStats

So, the difficulty here is to link your .NET HttpWebRequest to a TcpConnection from the list of connections. You can't get any stat before the connection exists, but once you have the connection created, you can get the corresponding one with a code like this:

    static IEnumerable<TcpConnection> GetProcessConnection(IPEndPoint ep)
    {
        var p = Process.GetCurrentProcess();
        return TcpConnection.GetAll().Where(c => ep.Equals(c.RemoteEndPoint) && c.ProcessId == p.Id);
    }

    HttpWebRequest req = ...

    // this is how you can get the enpoint, or you can also built it by yourself manually
    IPEndPoint remoteEndPoint;
    req.ServicePoint.BindIPEndPointDelegate += (sp, rp, rc) =>
        {
            remoteEndPoint = rp;
            return null;
        };
    // TODO: here, you need to connect, so the connection exists
    var cnx = GetProcessConnection(remoteEndPoint).FirstOrDefault();

    // access denied here means you don't have sufficient rights
    cnx.DataStatsEnabled = true;

    // TODO: here, you need to do another request, so the values are incremented
    // now, you should get non-zero values here
    // note TcpConnection also has int/out bandwidth usage, and in/out packet usage.
    Console.WriteLine("DataBytesIn:" + cnx.DataBytesIn);
    Console.WriteLine("DataBytesOut:" + cnx.DataBytesOut);

    // if you need all connections in the current process, just do this
    ulong totalBytesIn = 0;
    ulong totalBytesOut = 0;
    Process p = Process.GetCurrentProcess();
    foreach (var cnx in TcpConnection.GetAll().Where(c => c.ProcessId == p.Id))
    {
        totalBytesIn += cnx.DataBytesIn;
        totalBytesOut += cnx.DataBytesOut;
    }

There are 3 drawbacks:

• you need to be admin to enable data stats;
• you can't get the stats for the very first request. This may not be a problem depending on your context;
• the matching between the HttpWebRequest's connection and the the TCP connection can be trickier to determine because you can have more than one connection to a remote endpoint, even in the same process. The only way to distinguish is to determine the local endpoint (especially the port) and extend the GetProcessConnection with this local endpoint. Unfortunately, there is no easy way to do this. Here is a related answer about this: How do I get the local port number of an HttpWebRequest?

Update: if you want to continuously monitor all connections for a given process, I've written a ProcessTcpConnections utility class that remembers all connections and sums their usage. It would be used like this (in a console app sample):

class Program
{
    static void Main(string[] args)
    {
        ProcessTcpConnections p = new ProcessTcpConnections(Process.GetCurrentProcess().Id);
        Timer timer = new Timer(UpdateStats, p, 0, 100);

        do
        {
            // let's activate the network so we measure something...
            using (WebClient client = new WebClient())
            {
                client.DownloadString("http://www.example.com");
            }
            Console.ReadKey(true); // press any key to download again
        }
        while (true);
    }

    private static void UpdateStats(object state)
    {
        ProcessTcpConnections p = (ProcessTcpConnections)state;
        p.Update();
        Console.WriteLine("DataBytesIn:" + p.DataBytesIn + " DataBytesOut:" + p.DataBytesOut);
    }
}

public class ProcessTcpConnections : TcpConnectionGroup
{
    public ProcessTcpConnections(int processId)
        : base(c => c.ProcessId == processId)
    {
        ProcessId = processId;
    }

    public int ProcessId { get; private set; }
}

public class TcpConnectionGroup
{
    private List<TcpConnectionStats> _states = new List<TcpConnectionStats>();
    private Func<TcpConnection, bool> _groupFunc;

    public TcpConnectionGroup(Func<TcpConnection, bool> groupFunc)
    {
        if (groupFunc == null)
            throw new ArgumentNullException("groupFunc");

        _groupFunc = groupFunc;
    }

    public void Update()
    {
        foreach (var conn in TcpConnection.GetAll().Where(_groupFunc))
        {
            if (!conn.DataStatsEnabled)
            {
                conn.DataStatsEnabled = true;
            }

            TcpConnectionStats existing = _states.Find(s => s.Equals(conn));
            if (existing == null)
            {
                existing = new TcpConnectionStats();
                _states.Add(existing);
            }
            existing.DataBytesIn = conn.DataBytesIn;
            existing.DataBytesOut = conn.DataBytesOut;
            existing.LocalEndPoint = conn.LocalEndPoint;
            existing.RemoteEndPoint = conn.RemoteEndPoint;
            existing.State = conn.State;
            existing.LastUpdateTime = DateTime.Now;
        }
    }

    public ulong DataBytesIn
    {
        get
        {
            ulong count = 0; foreach (var state in _states) count += state.DataBytesIn; return count;
        }
    }

    public ulong DataBytesOut
    {
        get
        {
            ulong count = 0; foreach (var state in _states) count += state.DataBytesOut; return count;
        }
    }

    private class TcpConnectionStats
    {
        public ulong DataBytesIn { get; set; }
        public ulong DataBytesOut { get; set; }
        public IPEndPoint LocalEndPoint { get; set; }
        public IPEndPoint RemoteEndPoint { get; set; }
        public TcpState State { get; set; }
        public DateTime LastUpdateTime { get;  set; }

        public bool Equals(TcpConnection connection)
        {
            return LocalEndPoint.Equals(connection.LocalEndPoint) && RemoteEndPoint.Equals(connection.RemoteEndPoint);
        }
    }
}

Answer 2

You can create a proxy using FiddlerCore (just a single dll to reference) and set a WebProxy to your HttpWebRequest to count the sent and received bytes

public class WebConnectionStats
{
    static int _Read = 0;
    static int _Written = 0;

    public static void Init(bool registerAsSystemProxy = false)
    {
        Fiddler.FiddlerApplication.OnReadRequestBuffer += (s, e) => Interlocked.Add(ref _Written, e.iCountOfBytes);
        Fiddler.FiddlerApplication.OnReadResponseBuffer += (s, e) => Interlocked.Add(ref _Read, e.iCountOfBytes);
        Fiddler.FiddlerApplication.Startup(8088, registerAsSystemProxy, true);
    }

    public static int Read
    {
        get { return _Read; }
    }

    public static int Written
    {
        get { return _Written; }
    }
}

---

WebConnectionStats.Init(); //call this only once

var client = HttpWebRequest.Create("http://stackoverflow.com") as HttpWebRequest;
client.Proxy = new WebProxy("127.0.0.1", 8088);
var resp = client.GetResponse();
var html = new StreamReader(resp.GetResponseStream()).ReadToEnd();

Console.WriteLine("Read: {0}   Write: {1}", WebConnectionStats.Read, 
                                            WebConnectionStats.Written);

PS1: This counts will not include the Tcp Headers' length

PS2: You can get more info about Fiddler core here