Why are non-type template arguments used? [duplicate]

Question

I have read many questions and answers, but this question caught my eye the most; it and its answers are helpful but I still feel that I do not fully understand the uses and the theories behind using non-type template arguments. They provide many useful examples as to when it is used, but none really shed light on the theories behind the non-type template arguments.

I am interested in knowing not specifically when in examples, but more broadly why I would be inclined to use a non-type template argument over just a regular function argument.

I understand that they are determined at compile-time and that every new call creates a new function with the determined value for the non-type template argument. So why exactly would I want to create many different instances of the same function when I could just input the parameter I want into a function and have the same result - supposedly - with only one compiled function.

In essence, why should I be inclined to do #1, instead of #2, as according to the final section of this page of cplusplus.com?

#1:

template <class T, int N>
T fixed_multiply (T val)
{
  return val * N;
}

int main() {
  std::cout << fixed_multiply<int,2>(10) << '\n';
  std::cout << fixed_multiply<int,3>(10) << '\n';
}

#2:

template <class T>
T fixed_multiply (T val, int N)
{
  return val * N;
}

int main() {
  std::cout << fixed_multiply<int>(10, 2) << '\n';
  std::cout << fixed_multiply<int>(10, 3) << '\n';
}

In addition, would there be any performance benefits or the such of either one? Are there any common applications that could benefit from using non-type template arguments, or is this a highly technical parameter used in specific applications to yield specific results?

Edit: This got marked as a duplicate for some reason, the first paragraph explains my reasons for a similar question.

Answer 1

They are useful when you want (or need) the compiler to make certain compile-time optimizations while keeping code well-factored. I'll briefly list a few examples:

Branch elimination

void doSomething(bool flag) {
  if (flag) {
    //whatever
  }
}

template <bool flag>
void doSomething() {
 if (flag) {
   //whatever
}

}

In the above example, if you always know the value of flag at compile time when you call doSomething, then you can avoid the cost of a branch without manually creating doSomethingTrue and doSomethingFalse functions which may require you to repeat a lot of code. This trick is useful when you want to factor code between send and recv sides in network code, for example, and your code sits deep down in a performance-critical stack.

Avoid dynamic memory management

void someFunction(size_t size, float* inout) {
  std::vector<float> tmp(size);
  //do something with inout and tmp and then store result in inout
}

template <size_t N>
void someFunction(float *inout) {
  float tmp[N]; //or std::array if you like
  //do something with inout and tmp and store result in inout
}

In the above example, the second version will perform much better if it is stuck inside a critical loop.

Allow special optimizations

Consider the loop:

for (size_t i = 0; i < N; ++i) {
  //some arithmetic
}

If the compiler knows that N is 4 or 8, it may be able to replace your arithmetic with equivalent SIMD instructions, or at least unroll your loop more intelligently than it could if N were a runtime argument. This sort of thing is commonplace in the GPU programming (CUDA) world.

Template meta-programming

When you do some non-trivial code generation with templates, sometimes you must iterate over things at compile time. You need template non-type arguments to be able to do this. Argument packs and std::tuple are one common case, but there are many more, and hard to exemplify here.

To answer your question of when you'd make something a template parameter if you have a choice: code that takes dynamic (runtime) arguments is always more flexible, so you should only convert something to be a template argument if you can justify it with a substantial gain in performance. On the other hand if you find yourself writing much duplicated code over a set of enumerations, you should probably try and factor with templates.