Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Memory not released when emptying std::vector inside a user-defined class

We are having some memory issues when std::vector is a field of a class. We fill this vector with a lot of data which, at a certain point of the program needs to be released. However, even though the vector capacity is zero, the memory is not released or completely released.

Here you have a simplified version of our program. As you can see, class Foo has only one field: a std::vector<int>. If we create a std::vector<Foo> and fill it with Foo objects, when we empty the vector inside each object, the memory is not completely released.

We have measured the memory usage using the activity monitor and you can see, next to each log line, the amount of Bytes used in each stage. Moreover, we have added another version where we don't use class Foo objects and, in that case, memory is released perfectly.

#include <iostream>
#include <vector>

class Foo {

public:
    std::vector<int> container;
};

int main() {
    int n1 = 1000;
    int n2 = 100000;
    {
        std::vector<Foo> foos;

        std::cerr << "starting" << std::endl; // 160 KiB
        std::cin.get();

        for (int i = 0; i < n1; i++){
            Foo foo;
            foo.container.assign(n2, 666);
            foos.push_back(foo);
        }

        std::cerr << "foos filled" << std::endl; // 382.1 MiB
        std::cin.get();

        for (unsigned int i = 0; i < foos.size(); i++){
            std::vector<int>().swap(foos[i].container);
        }

        std::cerr << "foos emptied" << std::endl; // 195.7 MiB
        std::cin.get();
    }
    std::cerr << "foos destroyed?" << std::endl; // 296 KiB
    std::cin.get();

    {
        std::vector<std::vector<int> > foos;

        std::cerr << "starting" << std::endl; // 296 KiB
        std::cin.get();

        {
            std::vector<int> aux;
            aux.assign(n2, 666);
            foos.assign(n1, aux);
        }

        std::cerr << "foos filled" << std::endl; // 382.1 MiB
        std::cin.get();

        for (unsigned int i = 0; i < foos.size(); ++i) {
            std::vector<int>().swap(foos[i]);
        }

        std::cerr << "foos emptied" << std::endl; // 708 KiB
        std::cin.get();
    }

    std::cerr << "foos destroyed?" << std::endl; // 708 KiB
    std::cin.get();


    return 0;
}

If it helps, we're using g++ 4.8.4 under Ubuntu 14.04 64-bit. The specific memory occupancy varies depending on whether we use C++11 or C++98, but the same phenomenon occurs in both cases.

Any ideas as to what's happening and how to recover that memory, forcibly if need be?

EDIT: Note that memory is mostly returned indeed when we destroy all objects of the Foo class, but in our real-world problem we still need the rest of the contents of the Foo-analogue class.

like image 750
rafapages Avatar asked Jul 07 '16 12:07

rafapages


1 Answers

Memory is released to user space memory allocator from runtime C++/C libraries. In general it doesn't mean that user space allocator will return this memory back to OS. User space allocator allocates memory from kernel by blocks. This blocks further are sliced on your requests through new/malloc. When you free/delete this sliced blocks, they are returned to user space allocator, not the kernel. And when user space allocator will be able to return allocated block of memory back to kernel is known only by user space allocator.

like image 175
user1641854 Avatar answered Sep 23 '22 18:09

user1641854