hiding implementation details by reducing number of populated headers

Question

I am developing a library. I have interface class that would be called from outside.

I have also an internal engine that should not be called from outside.

As I read here and there, I should hide the internal engine class and not even populate its header. Since I have the following structure:

interface.hpp:

#include "engine.hpp" 
class interface{
private:
    enigne eng;
};

interface.cpp:

#include "engine.hpp"
//code that uses member variables and functions from eninge 

engine.hpp:

class engine{};

---

To solve the problem of populating "engine.hpp", I should change the code to:

interface.hpp:

class engine;
class interface{
private:
    some_smart_pointer<enigne> eng_ptr;
};

interface.cpp:

#include "engine.hpp"
//code that uses member variables and functions from eninge 

enigne.hpp:

class engine{};

This solved the problem. However, from now engine is allocated dynamically. All its member variables are in the free store.

I can not understand that I have to change my design and allocate engine on the free store for solving the problem of hiding implementation details. Is there a better solution?

P.S. I am not asking about why this solution works. I know it's about knowing the size of engine class is mandatory if I would leave it on stack. My question is about asking for a different design that may solve the problem.

EDIT:

Both interface and engine have member variables.

Answer 1

You're using the PIMPL idiom. The other way to hide implementation is to use interfaces, i.e. an abstract base class and a factory function:

interface.hpp:

class interface{
public:
    virtual ~interface(){}
    virtual void some_method() = 0;
};

// the factory function
static some_smart_pointer<interface> create();

interface.cpp:

#include "interface.hpp"
#include "engine.hpp"

class concrete : public interface{
public:
    virtual void some_method() override { /* do something with engine */ }
private:
    engine eng;
};

some_smart_pointer<interface> create(){ return new concrete; }

main.cpp

#include "interface.hpp"

int main()
{
    auto interface = create();
    interface->some_method();

    return 0;
}

The drawback here is that you must allocate dynamically interface instead of engine.

More discussion about PIMPL and interfaces here and here

EDIT:

Based on STL containers and Howard Hinnant's stack allocator, a third way to avoid variables in the free store can be:

interface.hpp:

class interface{
public:
    interface();
    ~interface();

    // I disable copy here, but you can implement them
    interface(const interface&) = delete;
    interface& operator=(interface&) = delete;

private:
    engine* eng;
};

interface.cpp:

#include "interface.hpp"
#include "engine.hpp"
#include "short_alloc.h"

#include <map>

namespace
{
    std::map<
        interface*,
        engine,
        std::default_order<interface*>,
        // use a stack allocator of 200 bytes
        short_alloc<std::pair<interface*, engine>, 200>
    > engines;
}

interface::interface():
    eng(&engines[this])
        // operator[] implicitly creates an instance and returns a reference to it
        // the pointer gets the address
{
}

interface::~interface()
{
    // destroy the instance
    engines.erase(this);
}

Answer 2

I can not understand that I have to change my design and allocate engine on the free store for solving the problem of hiding implementation details.

You seem to already have explained it yourself:

I know it's about knowing the size of engine class

Indeed. If the interface had a member of engine type, then it would necessarily need to know the size of that member - otherwise the size of interface itself couldn't be known. The size of an incomplete type can not be known. Defining the member type would solve that but conflicts with your desire to hide the implementation.

Is there a better solution?

There is no better solution. PIMPL with free store - which is the pattern that you currently use - is as good as it gets.

Technically, PIMPL doesn't require you to use the free store. You can store the object anywhere you like. You could use static storage. But that would limit the number of instances that you can have. You can even allocate a buffer of memory as a member of interface, but you need to hard code the size of such buffer and it must match the size (and alignment) of engine.

I would categorise both of these theoretical suggestions as kludges - the latter in particular. Static storage may be worth it if your profiling suggests that the overhead of that particular extra allocation is significant and forgoing PIMPL is not an option.