lost packets only occur in the first run

Question

I'm trying to improve the performance of my multicast application (in order to reduce its packet loss) which is working on a huge network

My experiments show that in the first run of the application there is some lost packets. But when I run the application again after the previous run (sometimes with a little delay also) there is no packet loss. Although when I re-run the application after a long delay (for example 20 minutes or so) I see the packet loss again.

And when I checked their timestamps, I saw that the lost packets were mostly the packets which were sent at the beginning. So it seems like the switches or routers need some warm up! or something (I don't know how to call this phenomena).

I've checked the tcpdump results and the number of packets that were received by receiver application was exactly the same number of packets which were received by network cart.

And I've already tried the following tricks: 1- change affinity of the process on the different CPU cores and scheduling policy 2- changing priority of the socket descriptor

changing priority of the socket descriptor already made it better (reduced number of lost packets) but after setting the priority to high, again there were some lost packets)

// For example
MulticastSender multicast_sender;
multicast_sender.init();

// Here I need a function in order to find out the switch is already warmed up or not


while (some condition)
{
    ////

    multicast_sender.send(something);

    ////
}

I was wondering isn't any possible way to add some code in order to find out whether the switch (or router) is already warmed up! enough?

Answer 1

Well, if your switches & routers are part of a dynamic multicast network (e.g., based on PIM, IGMP), which do require a certain path setup (depending on a type of a m-cast network various processes may affect setting the path between a sender and receivers), and you don't have a control over your m-cast network configuration (so that you could fix it, or setup a static path) then there's not much you could do.

Even if you do have a control access, not all events causing a m-cast packet loss could be "fixed" (some of them are intrinsic to m-casting). E.g., like if your m-cast is running in a PIM-sparse mode, then a switchover from a shared tree to the shortest tree may cause the loss. When a Rendezvous-Point (RP found on sparse mode m-casts) finishes a sender's registration process, some packet loss is inevitable, etc.

Check with your networks admins - see if they can help you setting up a "static" path between your sender (which is not always a possibility) and receivers, or at least minimize the packet loss (there are certain tweaks in m-casting).

From the application point of view: if you also writing the receiver's part - then it's best to employ some feedback mechanism indicating packet loss and possible re-transmit of the content.