Low Latency Networking Technqiues and Silver-Bullets

Question

After some basic googling of low-latency networking I've come up with the following list of things programmers and system designers should consider when embarking on low latency networking:

1. The design of the hardware, systems and protocols have to be considered together

2. Develop protocols using UDP instead of TCP and implement simple ack-nak, resend logic at the application level

3. Reduce the number of context switches (preferably to zero) for the process or thread that consumes and packetizes data off the wire

4. Use the best selector for the OS (select, kqueue, epoll etc)

5. Use good quality NICs and Switches with large amounts of on-board buffer (fifo)

6. Use multiple NICs, specifically for down-stream and up-stream data flows

7. Reduce the number of IRQs being generated by other devices or software (in short remove them if they are not required)

8. Reduce the usage of mutexes and conditions. Instead where possible use Lock-Free programming techniques. Make use of the architecture's CAS capabilities. (Lock-Free containers)

9. Consider single threaded over multi-threaded designs - context switches are very expensive.

10. Understand and properly utilize your architecture's cache system (L1/L2, RAM etc)

11. Prefer complete control over memory management, rather than delegating to Garbage Collectors

12. Use good quality cables, keep the cables as short as possible, reduce the number of twists and curls

My question: I was wondering what other things fellow SOers believe are important when embarking on low latency networking.

Feel free to critique any of the above points

Answer 1

Cable quality is usually kind of a red herring. I'd think more about connecting up a network analyzer to see whether you're getting enough re-transmissions to care about. If you're getting very many, try to isolate where they're happening, and replace the cable(s) that is/are causing the problem. If you're not getting errors that result in re-transmissions, then the cable has (virtually) no effect on latency.

Large buffers on NICs and (especially) switches won't, themselves, reduce latency. In fact, to truly minimize latency, you normally want to use the smallest buffers you can, not larger ones. Data sitting in a buffer instead of being processed immediately increases latency. Truthfully, it's rarely worth worrying about, but still. If you really want to minimize latency (and care a lot less about bandwidth) you'd be better off using a hub than a switch (kind of hard to find anymore, but definitely low latency as long as network congestion is low enough).

Multiple NICs can help bandwidth a lot, but their effect on latency is generally pretty minimal.

Edit: My primary advice, however, would be to get a sense of scale. Reducing a network cable by a foot saves you about a nanosecond -- on the same general order as speeding up packet processing by a couple of assembly language instructions.

Bottom line: Like any other optimization, to get very far you need to measure where you're getting latency before you can do much to reduce it. In most cases, reducing wire lengths (to use one example) won't make enough difference to notice, simply because it's fast to start with. If something starts out taking 10 microseconds, nothing you can do is going to speed it up any more than 10 microseconds, so unless you have things so fast that 10 us is a significant percentage of your time, it's not worth attacking.