Using a custom Lua allocator to count memory usage, but its result is different from collectgarbage('count')

Question

I'm trying to track the lua memory usage in our project recently, and I came across the idea of using lua_Alloc custom allocator for this task. Well, the allocator code looks easy, and it seems works fine.

But soon, this small function encountered two challenges:
1.the result it outputs is quite different from the value given by collectgarbage('count');
2.Assume current memory usage is M bytes, then we input some nil references and call gc, the memory usage will turn out to be larger than M bytes. e.g.: A return, B return, C return, ..., collectgarbage()

Well, I heard that if you use lua correctly, there will be no mem-leak, so I assume I was doing something wrong in counting the memory usage. Please help me figure it out. Thanks in advance.

The compilable code is attached below:

extern "C"
{
#include "lua.h"
#include <lauxlib.h>
#include <lualib.h>
};

#include <cstdio>
#include <string>
#include <cstdlib>

using namespace std;

struct Tracker 
{
    size_t m_usage;
}g_tracker;

void*
Allocator(void* ud, void* ptr, size_t osize, size_t nsize)
{
    Tracker* pTracker = (Tracker*)ud;
    void* pRet = NULL;
    if( nsize == 0 )
    {
        pTracker->m_usage -= osize;
        //printf("Free %d bytes; ", osize);
        free(ptr);
    }
    else
    {
        pTracker->m_usage -= osize;
        //printf("first Free %d bytes; ", osize);
        pTracker->m_usage += nsize;
        //printf("then alloc %d bytes; ", nsize);
        pRet = realloc(ptr, nsize);
    }

    //printf("current usage: %d bytes\n", pTracker->m_usage);
    return pRet;
}

int main()
{
    lua_State* L = lua_newstate(Allocator, &g_tracker );
    luaL_openlibs(L);

    char buf[4096];
    while(true)
    {
        fgets(buf, 4096, stdin);                        

        if( strlen(buf) > 1 && 0 != luaL_dostring(L, buf) )
        {
            const char* errmsg = lua_tostring(L, -1);
            printf("%s\n", errmsg);
        }

        printf("current usage: %d bytes \n", g_tracker.m_usage);
    }
}

input sequence about #2:

press enter
current usage: 18867 bytes
a=nil; b=nil; c=nil;
current usage: 19311 bytes
collectgarbage()
current usage: 18900 bytes
d=nil; e=nil; f=nil;
current usage: 19345 bytes
collectgarbage()
current usage: 18900 bytes
collectgarbage()
current usage: 18900 bytes  
for a = 1, 1000, 1 do b = nil end; collectgarbage()
current usage: 19206 bytes
collectgarbage()
current usage: 18900 bytes
for i = 1, 1000, 1 do X = {}; for j = 1, 1000, 1 do X[j] = j end ; X = nil; collectgarbage() ; end
current usage: 19391 bytes
collectgarbage()
current usage: 18900 bytes

Answer 1

As stated in the Lua documentation collectgarbage('collect') returns a value in kilobytes. 1KiB == 1024 Bytes. Therefore, when collectgarbage says that you're using 19.4775390625KiB, that translates to 19945 bytes. Which is exactly equal to one of many values your allocator got.

Your memory checking function will only ever match what Lua gets at exactly one moment: the moment that collectgarbage returns. Any Lua allocations/deallocations that happens before it or after it (such as those necessary for the call to print in order to display the value) will throw your count off.

In short, you're expecting byte accuracy when such accuracy is just not possible. At least, not with the code you have there.

In general, you shouldn't worry about it.