Techniques to avoid minimal scope inefficiency with complex objects in loops in C++?

Question

Question First

Is there an elegant solution in C++ to prevent one from having to declare complex object variables that are only used within a loop outside of the loop for efficiency reasons?

Detailed explanation

A colleague has raised an interesting point wrt. to our code policy, which states (paraphrased): always use minimal scope for variables and declare the variable at the first initialization.

Coding Guide Example:

// [A] DO THIS
void f() {
  ...
  for (int i=0; i!=n; ++i) {
    const double x = calculate_x(i);
    set_squares(i, x*x);
  }
  ...
}

// [B] DON'T do this:
void f() {
  int i;
  int n;
  double x;
  ...
  for (i=0; i!=n; ++i) {
    x = calculate_x(i);
    set_squares(i, x*x);
  }
  ...
}

This is all nice and well, and there's certainly nothing wrong with this, until you move from primitive types to objects. (for a certain kind of interface)

Example:

// [C]
void fs() {
  ...
  for (int i=0; i!=n; ++i) {
    string s;
    get_text(i, s); // void get_text(int, string&);
    to_lower(s);
    set_lower_text(i, s);
  }
  ...
}

Here, the string s will be destructed, it's memory release every loop cycle and then every cycle the get_text function will have to newly allocate the memory for the s buffer.

It would be clearly more efficient to write:

  // [D]
  string s;
  for (int i=0; i!=n; ++i) {
    get_text(i, s); // void get_text(int, string&);
    to_lower(s);
    set_lower_text(i, s);
  }

as now the allocated memory in the s buffer will be preserved between loop runs and it is very likely that we'll save on allocations.

Disclaimer: Please note: Since this is loops and we're talking memory allocations, I do not consider it premature optimization to think about this problem generally. Certainly there are cases and loops where the overhead wouldn't matter; but n has the nagging tendency to be larger that the Dev initially expects and the code has the nagging tendency to be run in contexts where performance does matter.

Anyway, so now the more efficient way for the "general" loop construct is to violate code locality and declare complex objects out of place, "just in case". This makes me rather uneasy.

Note that I consider writing it like this:

// [E]
void fs() {
  ...
  {
    string s;
    for (int i=0; i!=n; ++i) {
      get_text(i, s); // void get_text(int, string&);
      to_lower(s);
      set_lower_text(i, s);
    }
  }
  ...
}

is no solution as readability suffers even more!

Thinking further, the interface of the get_text function is non-idiomatic anyway, as out params are so yesterday anyway and a "good" interface would return by value:

  // [F]
  for (int i=0; i!=n; ++i) {
    string s = get_text(i); // string get_text(int);
    to_lower(s);
    set_lower_text(i, s);
  }

Here, we do not pay double for memory allocation, because it is extremely likely that s will be constructed via RVO from the return value, so for [F] we pay the same in allocation overhead as in [C]. Unlike the [C] case however, we can't optimize this interface variant.

So the bottom line seems to be that using minimal scope (can) hurt performance and using clean interfaces _{I at least consider return by value a lot cleaner than that out-ref-param stuff} will prevent optimization opportunities -- at least in the general case.

The problem isn't so much that one would have to forgo clean code for efficiency sometimes, the problem is that as soon as Devs start to find such special cases, the whole Coding Guide (see [A], [B]) looses authority.

The question now would be: see first paragraph

Answer 1

It would be clearly more efficient to write: [start of example D ...]

I doubt this bit. You're paying for default construction to begin with outside the loop. Within the loop, there is a possibility that get_text calls reallocate buffer (depends on how your get_text and the string is defined). Note that this for some runs you may actually see an improvement (say in the case where you get progressively shorter strings) and for some (where the string lengths go up by about a factor of 2 at every iteration) a huge hit in performance.

It makes perfect sense to hoist invariants out of your loop should they pose a bottleneck (which a profiler will tell you). Otherwise, go for code that is idiomatic.

Answer 2

I'd either:

• make an exception to the rule for these heavyweights. like 'D' and note that you can restrict the scope as desired.
• permit a helper function (the string could also be a parameter)
• and if you really didn't like those, you could declare a local in your for loop's scope using a multi-element object which held your counter/iterator and the temporary. std::pair<int,std::string> would be one option, although a specialized container could reduce the syntactic noise.

(and the out parameter would be faster than RVO-style in many cases)

Answer 3

Depends on the implementation of get_text.

If you can implement it so it reuses the space allocated in the string object most of the time, then definitely declare the object outside the loop to avoid new dynamic memory allocation at each loop iteration.

Dynamic allocation is expensive (best single-threaded allocators will need about 40 instructions for a single allocation, multi-threading adds overhead and not all allocators are "best"), and can fragment memory.

(BTW, std::string typically implements so called "small string optimization", which avoids dynamic allocation for small strings. So if you know most of your strings will be small enough, and the implementation of std::string won't change, you could theoretically avoid dynamic allocation even when constructing a new object in each iteration. This would be very fragile however, so I'd recommend against it.)

---

In general case, it all depends on how your objects and functions that use them are implemented. If you care about performance, you'll have to deal with these kinds of "abstraction leaks" on case-by-case basis. So, pick your battles wisely: measure and optimize bottlenecks first.