Code organization across files that has to deal with template functions and inlining

Question

I'm maintaining a large library of template classes that perform algebraic computations based on either float or double type. Many of the classes have accessor methods (getters and setters) and other functions that run small amounts of code, therefore such functions need to be qualified as inline when the compiler locates their definitions. Other member functions, in contrast, contain sophisticated code and thus would better be called rather than inlined.

A substantial part of the function definitions are located in headers, actually in .inl files included by headers. But there are also many classes whose function definitions happily live in .cpp files by means of explicit instantiation for float and double, which is rather a good thing to do in case of a library (here explained why). And finally, there is a considerable number of classes whose function definitions are broken across .inl files (accessor methods) and .cpp files (constructors, destructors, and heavy computations), which makes them all pretty difficult to maintain.

I would have all my class implementations in .inl files only if I knew a reliable way to prevent some functions from being inlined, or in .cpp files if inline keyword could strongly suggest compiler to inline some of the functions, which, of course, it does not. I would really prefer all the function definitions in the library to reside in .cpp files, but since accessor methods are used extensively throughout the library, I have to make sure they are inlined whenever referenced, not called.

So, in this connection, my questions are:

1. Does it make any sense to mark the definition of a template function with inline in view of the fact that, as I've recently learnt here, it is going to be automatically qualified as inline by the compiler regardless of whether it's marked with inline or not?

2. And most importantly, since I would like to have the definitions of all the member functions of a template class gathered together in a single file, either it's .inl or .cpp (using explicit instantiation in case of .cpp), preferably still being able to hint the compiler (MSVC and GCC) which of the functions should be inlined and which shouldn't, sure if such thing is possible with template functions, how can I achieve this or, if there is really no way (I hope there is), what would be the most optimal compromise?

----------

EDIT1: I knew that inline keyword is just a suggestion to the compiler to inline a function.

EDIT2: I really do know. I like making suggestions to the compiler.

EDIT3: I still know. It's not what the question is about.

----------

In view of some new information, there is also third question that goes hand in hand with the second one.

3. If compilers are so smart these days that they can make better choices about which function should be inlined and which should be called and are capable of link-time code generation and link-time optimization, which effectively allows them looking into a .cpp-located function definition at link time to decide its fate about being inlined or called, then maybe a good solution would be simply moving all the definitions into respective .cpp files?

----------

So what's the conclusion?

First of all, I'm grateful to Daniel Trebbien and Jonathan Wakely for their structured and well-founded answers. Upvoted both but had to choose just one. None of the given answers, however, presented an acceptable solution to me, so the chosen answer happened to be the one that helped me slightly more than others in making the final decision, the details of which are explained next for anyone who's interested.

Well, since I've always been valuing the performance of code more than how much convenient it is to maintain and develop, it appears to me that the most acceptable compromise would be to move all the accessor methods and other lightweight member functions of each of the template classes into the .inl file included by the respective header, marking these functions with inline keyword in an attempt to provide the compiler with a good hint (or with a keyword for inline forcing), and move the rest of the functions into the respective .cpp file.

Having all member function definitions located in .cpp files would hinder inlining of lightweight functions while unleashing some problems with link-time optimization, as has been ascertained by Daniel Trebbien for MSVC (in an older stage of development) and by Jonathan Wakely for GCC (in its current stage of development). And having all function definitions located in headers (or .inl files) doesn't outweigh the summary benefit of having the implementation of each class sorted into .inl and .cpp files combined with a bonus side effect of this decision: it would ensure that only the code of primitive accessor methods is visible to a client of the library, while more juicy stuff is hidden in the binaries (ensuring this wasn't a major reason, however, but this plus was obvious for anyone who is familiar with software libraries). And any lightweight member function that doesn't need to be exposed by the include files of the library and is used privately by its class can have its definition in the .cpp file of the class, while its declaration/definition is spiced with inline to encourage the inline status of the function (don't know yet whether the keyword should be in both places or just one in this particular case).

Answer 1

In short: Put the template code in a header file. Use compiler-specific forceinline or noinline keywords if the optimizer fails to make good decisions about inlining.

---

You can and should put definitions of template members into header files. This ensures that the compiler has access to the definition at the point of use when it finds out what the actual template parameters are, and is able to perform implicit instantiaion.

The inline keyword has very little impact on templates, since template functions are already exempted from the single definition requirement (The One Definition Rule still requires that all definitions be the same). It is a hint to the compiler that the function should be inlined. And you can omit it as a hint to the compiler to not inline the function. So use it that way. But the optimizer will still look at other factors (function size) and make its own choice on inlining.

Some compilers have special keywords, like __attribute__(always_inline) or __declspec(noinline) to override the optimizer's choice.

Mostly, though, the compiler is smart enough not to inline "complex code that makes more sense as a function call". You shouldn't have to worry about it, just let the optimizer do its thing.

Portable inlining control isn't beneficial, because the trade-offs of inlining are very platform-specific. The optimizers should already be aware of those platform-specific tradeoffs, and if you do feel the need to override the compiler's choice, do so on a per-platform basis.

Answer 2

1. Does it make any sense to mark the definition of a template function with inline in view of the fact that, as I've recently learnt, it is going to be automatically qualified as inline by the compiler regardless of whether it's marked with inline or not? Is the behavior compiler-specific?

I think you are referring to the fact that a member function defined in its class definition is always an inline function. This is per the C++ Standard, and has been since the first publication:

9.3 Member functions

...

A member function may be defined (8.4) in its class definition, in which case it is an inline member function (7.1.2)

So, in the following example, template <typename FloatT> my_class<FloatT>::my_function() is always an inline function:

template <typename FloatT>
class my_class
{
public:
    void my_function() // `inline` member function
    {
        //...
    }
};

template <>
class my_class<double> // specialization for doubles
{
public:
    void my_function() // `inline` member function
    {
        //...
    }
};

However, by moving the definition of my_function() outside of the definition of template <typename FloatT> my_class<FloatT>, it is not automatically an inline function:

template <typename FloatT>
class my_class
{
public:
    void my_function();
};

template <typename FloatT>
void my_class<FloatT>::my_function() // non-`inline` member function
{
    //...
}

template <>
void my_class<double>::my_function() // non-`inline` member function
{
    //...
}

In the latter example, it does make sense (as in, it's not redundant) to use the inline specifier with the definitions:

template <typename FloatT>
inline void my_class<FloatT>::my_function() // `inline` member function
{
    //...
}

template <>
inline void my_class<double>::my_function() // `inline` member function
{
    //...
}

---

2. And most importantly, since I would like to have the definitions of all the member functions of a template class gathered together in a single file, either it's .inl or .cpp (using explicit instantiation in case of .cpp), preferably still being able to hint the compiler (MSVC and GCC) which of the functions should be inlined and which shouldn't, sure if such thing is possible with template functions, how can I achieve this or, if there is really no way (I hope there is), what would be the most optimal compromise?

As you know, the compiler may elect to inline a function, whether or not it has the inline specifier; the inline specifier is just a hint.

There is no standard way to force inlining or prevent inlining; however, most C++ compilers support syntactic extensions for accomplishing just that. MSVC supports a __forceinline keyword to force inlining and #pragma auto_inline(off) to prevent it. G++ supports always_inline and noinline attributes for forcing and preventing inlining, respectively. You should refer to your compiler's documentation for details, including how to enable diagnostics when the compiler is unable to inline a function as requested.

If you use those compiler extensions, then you should be able to hint to the compiler whether a function is inlined or not.

In general, I recommend to have all "simple" member function definitions gathered together in a single file (usually the header), by which I mean, if the member function does not require very many more #includes above the set of #includes required to define the classes/templates. Sometimes, for example, a member function definition will require #include <algorithm>, but it is unlikely that the class definition requires <algorithm> to be included in order to be defined. Your compiler is able to skip over function definitions that it does not use, but the larger number of #includes can noticeably lengthen compile times, and it is unlikely that you will want to inline these non-"simple" functions anyway.

---

3. If compilers are so smart these days that they can make better choices about which function should be inlined and which should be called and are capable of link-time code generation and link-time optimization, which effectively allows them looking into a .cpp-located function definition at link time to decide its fate about being inlined or called, then maybe a good solution would be simply moving all the definitions into respective .cpp files?

If you place all of your function definitions into CPP files, then you will be relying on LTO for mostly all function inlining. This may not be what you want for the following reasons:

1. At least with MSVC's LTCG, you give up the ability to force inlining (See inline, __inline, __forceinline.)
2. If the CPP files are linked to a shared library, then programs linking with the shared libraries will not benefit from LTO inlining of library functions. This is because the compiler intermediate language (IL)—the input to LTO—has been discarded and is not available in the DLL or SO.
3. If Under The Hood: Link-time Code Generation is still correct, "calls to functions in static libraries can't be optimized".
4. The linker would be performing all inlining, which might be a lot slower than having the compiler perform some inlining at compile time.
5. The compiler's LTO implementation might have bugs that cause it to not inline certain functions.
6. Use of LTO might impose certain limitations on projects using your library. For example, according to Under The Hood: Link-time Code Generation, "precompiled headers and LTCG are incompatible". The /LTCG (Link-time Code Generation) MSDN page has other notes, such as "/LTCG is not valid for use with /INCREMENTAL".

If you keep the likely-to-be-inlined function definitions in the header files, then you could use both compiler inlining and LTO. On the other hand, moving all function definitions into CPP files will restrict compiler inlining to only within the translation units.