Is there an advantage in choosing either loop as an outer loop?

Question

I am extending an existing logging library. It is a system with two sides: The frontend is where tasks write their log messages into, the backend is where an application can plug listeners into which forward those messages to different sinks. The backend used to be one hard-wired listener, I am now extending this for flexibility. The code is to be used exclusively on embedded devices, where high performance (measured in number of bytes forwarded per millisecond) is a very important design and implementation objective.

For performance reasons, messages are buffered, and forwarding is done in a background task. That task fetches a chunk of messages from a queue, formats them all, and then passes them to the listeners via registered functions. Those listeners will filter messages, and will only write those to their sink that pass the filter criterion.

Given this, I end up having N notification functions (the listeners) to send M messages to, a rather classic N*M problem. Now I have two possibilities: I can loop over the messages, and then loop over the notification functions passing the message to each one.

for(m in formatted_messages) 
  for(n in notification_functions)
    n(m);

void n(message)
{
    if( filter(message) )
      write(message);
}

Or I could loop over all the notification functions, and pass them all the messages I have at once:

for(n in notification_functions)
    n(formatted_messages);

void n(messages)
{
  for(m in messages)
    if( filter(m) )
      write(m);
}

Is there any fundamental considerations regarding which design is more likely to allow a higher number of messages to be processed per time slice? (Note how this question determines the listener's interface. This isn't a micro-optimization question, but one about how to make a design that does not hinder performance. I can measure only much later, and redesigning the listener interface then will be costly.)

Some considerations I have already made:

• Those listeners need to write the messages somewhere, which is rather expensive, so the function calls by themselves might not be too important performance-wise.
• In 95% of all cases, there will only be one listener.

Answer 1

Is there any fundamental considerations regarding which design is more likely to allow a higher number of messages to be processed per time slice?

In general, the main considerations with this often boil down to two main things.

1. If one of your loops is looping over objects which can potentially have good memory locality (such as looping over an array of values), keeping that portion in the inner loop can potentially keep the objects within the CPU cache, and improve performance.

2. If you plan to try to parallelize the operation, keeping the "larger" (in terms of count) collection in the outer loop allows you to parallelize the outer loop effectively, and not cause over subscription of threads, etc. It's typically simpler and cleaner to parallelize an algorithm at the outer level, so designing the loops with the potentially larger parallel "blocks" of work at the outer loop can simplify this, if it's a possibility later.

Those listeners need to write the messages somewhere, which is rather expensive, so the function calls by themselves might not be too important performance-wise.

This will probably completely negate any benefits of moving one loop outside of the other.

In 95% of all cases, there will only be one listener.

If this is the case, I would likely put the listener loop at the outer scope, unless you plan to parallelize this operation. Given that this is going to run in a background thread on an embedded device, parallelizing is unlikely, so having the listener loop as the outer loop should reduce the overall instruction count (it effectively becomes a loop over M operations, instead of M loops over a single operation).