Sockets vs HttpListener

Question

I am working on creating a proxy which listens on a particular port for incoming connections. The connections are usually Http Requests (GET/POST).Can't decide should I choose HttpListener or Sockets. I will be modifying the HttpRequests in the proxy and then relaying it to the final destination.

When do you prefer HttpListener over Sockets. What are the benefits of each ?

Answer 1

• Advantages of HttpListener
  • Allows port sharing with other processes using http.sys (including those using HttpListener or IIS, so long as the prefixes are unique)
  • Http.sys uses I/O Completion Ports for you; it's very difficult to set that up yourself without using TcpListener
  • HttpListener parses the HTTP headers and creates the HTTP headers for the response
  • Gives you a set of classes that are very much like those in ASP.Net that expose headers, cookies, etc.
• Disadvantages of HttpListener
  • MAJOR - Localhost client sending large payloads via HTTP causes several additional copies of those payloads to be made compared to TcpListener or Mono's HttpListener on Windows. Sending of large (e.g. more than 5 MB) POST bodies between processes on the same machine should be avoided if maximum throughput per CPU is required. Consider using memory mapped files (essentially shared memory) instead. These extra copies are not a problem when the client and server are on different machines.
  • Does not allow explicit control of how long a TCP socket is left open for; http.sys manages this for you (this isn't usually a huge disadvantage, but it's just something to be aware of)
  • Any bugs in the header/cookie classes will bite you; such as the issue where setting multiple cookies will cause HttpListener to return a single Set-Cookie header with multiple cookies in it, which IE will handle but Chrome will completely ignore (you have to then hand-roll the cookies and add your own Set-Cookie header to work-around this)
• Advantages of TcpListener
  • Like HttpListener, this implements I/O Completion Ports for you
  • If you want to do something once for a connection (like check auth), you can do it since you know when the socket is opened; however, you might not want to rely on this for other reasons (proxies and load balancers may be sending requests from multiple users over the same socket)
• Disadvantages of TcpListener vs. HttpListener
  • You have to supply an HTTP header parser
  • You have to validate that the HTTP request is valid
• Advantages of Rolling this 100% on Your Own
  • None that I can think of, compared to using TcpListener
• Disadvantages of Rolling this 100% on Your Own
  • You will likely introduce lots of unnecessary/incorrect locks in your network layer code, which will not cause any problems when testing the performance of a single request at a time, but will cause the performance under load to really suffer. It'll take a while to find these and fix them.
  • We did roll this on our own on a project and we achieved the same performance as HttpListener with many more months of coding and a resulting code base that was unmaintainable. We ripped this out and replaced all the custom code with HttpListener without any negative performance impact and with substantially less code to maintain.

November 2016 Update: HttpListener works very well when receiving traffic from remote machines. However, HttpListener is a poor choice, performance-wise, for large payloads when the client is on localhost. The reason for this is partly because HttpListener uses http.sys, which is a kernel module, and apparently the implementation causes several additional copies of the body data to be made. For example, copying 100 MB from one array to another array in C# takes 35 ms while POSTing 100 MB from a local client to HttpListener takes 350 ms (yes, 10x longer). Switching to Mono's HttpListener in this case drops the time to 250 ms. Using TcpListener with a Socket client takes 180 ms. Swapping out the usage of sockets for a CircularBuffer using MemoryMappedFiles takes 55 ms. Thus HttpListener doesn't work so well for large payloads when accessed from the local machine. Note: if you try to reproduce this test you need to take care to measure the time difference. Stern starting to send the data and when the array is completing populated in the receiving process as some of the Send/Write methods return almost immediately without sending any of the data (which you can prove by sending an array of zeros and then writing ones from the back of the array towards be front; you will receive nearly all ones, proving that the data was not sent until long after the function returned).

More details on the local machine performance deficit of HttpListener vs Mono's HttpListener from one of my colleagues here: https://www.linkedin.com/pulse/http-inefficiency-dominika-blach