valgrind | Invalid read of size 8 | Address 0x7a41270 is 0 bytes inside a block of size 4 alloc'd

Question

since several days I am trying to find a memory bug in my software.
It is quite complex, but well-structured and formatted (in my opinion).
The bug appears on a 64 Bit machine running Debian GNU/Linux.
(... not on my 32 Bit machine -- void*/int mixup?)
I'd be really glad if you could help me out.

Source code snippets follow at the end of this message.
If that does not suffice, you may want to checkout all source from:
http://savannah.nongnu.org/svn/?group=cybop

Follow the instructions of the INSTALL file to compile and run.

Running the software as follows works fine:

enter code herechristian@deneb:/home/project/cybop/examples$ ../src/controller/cyboi

Running it with an option brings memcheck errors in valgrind:
christian@deneb:/home/project/cybop/examples$ ../src/controller/cyboi --help

I've run memcheck using this command line:
christian@deneb:/home/project/cybop/examples$ valgrind --log-file=memcheck.log --tool=memcheck --read-var-info=yes --track-origins=yes ../src/controller/cyboi --help

Here is the memcheck output:

==20072== Memcheck, a memory error detector
==20072== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==20072== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==20072== Command: ../src/controller/cyboi --help
==20072== Parent PID: 18978
==20072== 
==20072== Invalid read of size 8
==20072==    at 0x40AF0E: decode_utf_8 (utf_8_decoder.c:298)
==20072==    by 0x42A53D: optionalise_argument (argument_optionaliser.c:110)
==20072==    by 0x42A786: optionalise (optionaliser.c:107)
==20072==    by 0x401507: main (cyboi.c:168)
==20072==  Address 0x7a41270 is 0 bytes inside a block of size 4 alloc'd
==20072==    at 0x402894D: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20072==    by 0x404C52: allocate_array (array_allocator.c:66)
==20072==    by 0x40A0FC: allocate_item (item_allocator.c:62)
==20072==    by 0x42A4D2: optionalise_argument (argument_optionaliser.c:85)
==20072==    by 0x42A786: optionalise (optionaliser.c:107)
==20072==    by 0x401507: main (cyboi.c:168)
==20072== 
==20072== Invalid read of size 8
==20072==    at 0x40AF31: decode_utf_8 (utf_8_decoder.c:300)
==20072==    by 0x42A53D: optionalise_argument (argument_optionaliser.c:110)
==20072==    by 0x42A786: optionalise (optionaliser.c:107)
==20072==    by 0x401507: main (cyboi.c:168)
==20072==  Address 0x7a41270 is 0 bytes inside a block of size 4 alloc'd
==20072==    at 0x402894D: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20072==    by 0x404C52: allocate_array (array_allocator.c:66)
==20072==    by 0x40A0FC: allocate_item (item_allocator.c:62)
==20072==    by 0x42A4D2: optionalise_argument (argument_optionaliser.c:85)
==20072==    by 0x42A786: optionalise (optionaliser.c:107)
==20072==    by 0x401507: main (cyboi.c:168)
==20072== 
==20072== 
==20072== HEAP SUMMARY:
==20072==     in use at exit: 0 bytes in 0 blocks
==20072==   total heap usage: 34 allocs, 34 frees, 8,232 bytes allocated
==20072== 
==20072== All heap blocks were freed -- no leaks are possible
==20072== 
==20072== For counts of detected and suppressed errors, rerun with: -v
==20072== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 4 from 4)

Running the software as follows:
christian@deneb:/home/project/cybop/examples$ ../src/controller/cyboi --knowledge ui_control/run.cybol

... produces these errors:

cyboi: malloc.c:3096: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size)

= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. Aborted

Any help would be really appreciated. I don't really know anymore on which place to debug.

Source code follows:

//
// cyboi.c
//

// The cybol knowledge file path item.
void* k = *NULL_POINTER_STATE_CYBOI_MODEL;

// Allocate cybol knowledge file path item.
allocate_item((void*) &k, (void*) NUMBER_0_INTEGER_STATE_CYBOI_MODEL, (void*) WIDE_CHARACTER_TEXT_STATE_CYBOI_TYPE);

// Optionalise command line argument options.
optionalise((void*) &m, k, (void*) LOG_LEVEL, (void*) LOG_OUTPUT, (void*) p1, (void*) &p0);

//
// optionaliser.c
//

while (*TRUE_BOOLEAN_STATE_CYBOI_MODEL) {

    compare_integer_greater_or_equal((void*) &b, (void*) &j, p5);

    if (b != *FALSE_BOOLEAN_STATE_CYBOI_MODEL) {

        break;
    }

    optionalise_argument(p0, p1, p2, p3, p4, p5, (void*) &j);

    // Increment loop variable.
    j++;
}

//
// argument_optionaliser.c
//

// The command line argument option as multibyte character array.
void* od = *NULL_POINTER_STATE_CYBOI_MODEL;
int oc = *NUMBER_0_INTEGER_STATE_CYBOI_MODEL;
// The option, value as wide character item.
void* ow = *NULL_POINTER_STATE_CYBOI_MODEL;
// The option, value as wide character item data, count.
void* owd = *NULL_POINTER_STATE_CYBOI_MODEL;
void* owc = *NULL_POINTER_STATE_CYBOI_MODEL;

// Allocate option, value wide character item.
allocate_item((void*) &ow, (void*) NUMBER_0_INTEGER_STATE_CYBOI_MODEL, (void*) WIDE_CHARACTER_TEXT_STATE_CYBOI_TYPE);

// Get command line argument option.
// Example: "--loglevel"
copy_array_forward((void*) &od, p4, (void*) POINTER_STATE_CYBOI_TYPE, (void*) PRIMITIVE_STATE_CYBOI_MODEL_COUNT, (void*) VALUE_PRIMITIVE_STATE_CYBOI_NAME, p6);

if (od != *NULL_POINTER_STATE_CYBOI_MODEL) {

    // Get command line argument option count (number of characters).
    oc = strlen((char*) od);

    // Decode multibyte character option into wide character.
    decode_utf_8(ow, od, (void*) &oc);

} else {

    log_write((void*) stdout, L"Error: Could not optionalise command line argument. The command line argument option is null.\n");
}

//
// utf_8_decoder.c
//

// The destination item data, count, size.
void* dd = *NULL_POINTER_STATE_CYBOI_MODEL;
void* dc = *NULL_POINTER_STATE_CYBOI_MODEL;
void* ds = *NULL_POINTER_STATE_CYBOI_MODEL;
// The new destination size.
int nds = *NUMBER_0_INTEGER_STATE_CYBOI_MODEL;

// Get destination item count, size.
copy_array_forward((void*) &dc, p0, (void*) POINTER_STATE_CYBOI_TYPE, (void*) PRIMITIVE_STATE_CYBOI_MODEL_COUNT, (void*) VALUE_PRIMITIVE_STATE_CYBOI_NAME, (void*) COUNT_ITEM_STATE_CYBOI_NAME);
copy_array_forward((void*) &ds, p0, (void*) POINTER_STATE_CYBOI_TYPE, (void*) PRIMITIVE_STATE_CYBOI_MODEL_COUNT, (void*) VALUE_PRIMITIVE_STATE_CYBOI_NAME, (void*) SIZE_ITEM_STATE_CYBOI_NAME);

// Initialise new destination size.
calculate_integer_add((void*) &nds, p2);

// Reallocate destination item.
reallocate_item(p0, (void*) &nds, (void*) WIDE_CHARACTER_TEXT_STATE_CYBOI_TYPE);

// Set locale.
// CAUTION! This setting IS NECESSARY for UTF-8 character conversion
// with restartable multibyte conversion functions like "mbsnrtowcs"
// and "wcsnrtombs" to work correctly.
// The return value is not used; this is a global setting.
char* loc = setlocale(LC_CTYPE, "");

// Get destination item data.
copy_array_forward((void*) &dd, p0, (void*) POINTER_STATE_CYBOI_TYPE, (void*) PRIMITIVE_STATE_CYBOI_MODEL_COUNT, (void*) VALUE_PRIMITIVE_STATE_CYBOI_NAME, (void*) DATA_ITEM_STATE_CYBOI_NAME);

// Initialise error number.
// It is a global variable/ function and other operations
// may have set some value that is not wanted here.
//
// CAUTION! Initialise the error number BEFORE calling the function
// that might cause an error.
errno = *NUMBER_0_INTEGER_STATE_CYBOI_MODEL;

// Converts the multibyte character string into a wide character string.
// Returns the number of wide characters
// successfully converted, except in the case of an encoding error.
//
// CAUTION! The multibyte source character string does NOT need to be
// null-terminated, since the third parametre already indicates its count.
//
// CAUTION! Hand over the NEW destination size as fourth parametre,
// since it indicates the maximum number of characters to be converted
// and conversion would break too early if that parametre was too small.
//
// CAUTION! The fifth parametre may be NULL. In this case, a static
// anonymous state only known to the function internally is used instead.
// It just indicates where conversion is started.
int n = mbsnrtowcs((wchar_t*) dd, (const char**) &sd, *((size_t*) sc), *((size_t*) ds), (mbstate_t*) *NULL_POINTER_STATE_CYBOI_MODEL);

if (n >= *NUMBER_0_INTEGER_STATE_CYBOI_MODEL) {

    // Set destination count to the number of WIDE characters converted.
    copy_integer(dc, (void*) &n);

} else {

    if (errno == EILSEQ) {

        fwprintf(stdout, L"TEST ERROR EILSEQ errno: %i\n", errno);
        log_message_terminated((void*) ERROR_LEVEL_LOG_CYBOI_MODEL, (void*) L"Could not decode utf-8. The input string contains an invalid multibyte sequence.");

    } else {

        fwprintf(stdout, L"TEST ERROR UNKNOWN errno: %i\n", errno);
        log_message_terminated((void*) ERROR_LEVEL_LOG_CYBOI_MODEL, (void*) L"Could not decode utf-8. An unknown error occured.");
    }
}

//
// item_allocator.c
//

/**
 * Allocates the item.
 *
 * @param p0 the item (pointer reference)
 * @param p1 the size
 * @param p2 the type
 */
void allocate_item(void* p0, void* p1, void* p2) {

    if (p0 != *NULL_POINTER_STATE_CYBOI_MODEL) {

        void** i = (void**) p0;

        log_message_terminated((void*) DEBUG_LEVEL_LOG_CYBOI_MODEL, (void*) L"Allocate item.");

        // Allocate item.
        allocate_array(p0, (void*) ITEM_STATE_CYBOI_MODEL_COUNT, (void*) POINTER_STATE_CYBOI_TYPE);

        // The data, count, size.
        void* d = *NULL_POINTER_STATE_CYBOI_MODEL;
        void* c = *NULL_POINTER_STATE_CYBOI_MODEL;
        void* s = *NULL_POINTER_STATE_CYBOI_MODEL;

        // Allocate data, count, size.
        allocate_array((void*) &d, p1, p2);
        allocate_array((void*) &c, (void*) PRIMITIVE_STATE_CYBOI_MODEL_COUNT, (void*) INTEGER_NUMBER_STATE_CYBOI_TYPE);
        allocate_array((void*) &s, (void*) PRIMITIVE_STATE_CYBOI_MODEL_COUNT, (void*) INTEGER_NUMBER_STATE_CYBOI_TYPE);

        // Initialise count, size.
        // The data does NOT have to be initialised and remains empty.
        // The count is set to zero, since the model does not contain any elements yet.
        // The size is set to the value that was handed over as argument.
        copy_integer(c, (void*) NUMBER_0_INTEGER_STATE_CYBOI_MODEL);
        copy_integer(s, p1);

        // Set data, count, size.
        copy_array_forward(*i, (void*) &d, (void*) POINTER_STATE_CYBOI_TYPE, (void*) PRIMITIVE_STATE_CYBOI_MODEL_COUNT, (void*) DATA_ITEM_STATE_CYBOI_NAME, (void*) VALUE_PRIMITIVE_STATE_CYBOI_NAME);
        copy_array_forward(*i, (void*) &c, (void*) POINTER_STATE_CYBOI_TYPE, (void*) PRIMITIVE_STATE_CYBOI_MODEL_COUNT, (void*) COUNT_ITEM_STATE_CYBOI_NAME, (void*) VALUE_PRIMITIVE_STATE_CYBOI_NAME);
        copy_array_forward(*i, (void*) &s, (void*) POINTER_STATE_CYBOI_TYPE, (void*) PRIMITIVE_STATE_CYBOI_MODEL_COUNT, (void*) SIZE_ITEM_STATE_CYBOI_NAME, (void*) VALUE_PRIMITIVE_STATE_CYBOI_NAME);

    } else {

        log_message_terminated((void*) ERROR_LEVEL_LOG_CYBOI_MODEL, (void*) L"Could not allocate item. The item is null.");
    }
}

//
// array_allocator.c
//

/**
 * Allocates the array.
 *
 * @param p0 the array (pointer reference)
 * @param p1 the size
 * @param p2 the type
 */
void allocate_array(void* p0, void* p1, void* p2) {

    if (p0 != *NULL_POINTER_STATE_CYBOI_MODEL) {

        void** a = (void**) p0;

        log_message_terminated((void*) DEBUG_LEVEL_LOG_CYBOI_MODEL, (void*) L"Allocate array.");

        // The memory area.
        size_t ma = *NUMBER_0_INTEGER_STATE_CYBOI_MODEL;

        // Determine type (type) size.
        determine_size((void*) &ma, p2);

        // Calculate memory area.
        calculate_integer_multiply((void*) &ma, p1);

        // A minimal space in memory is always allocated,
        // even if the requested size is zero.
        // In other words, a handle to the new instance is always returned.
        *a = malloc(ma);

        // Initialise array elements with null pointer.
        //
        // CAUTION! Initialising with zero is essential, since cyboi
        // frequently tests variables for null pointer values.
        memset(*a, *NUMBER_0_INTEGER_STATE_CYBOI_MODEL, ma);

    } else {

        log_message_terminated((void*) ERROR_LEVEL_LOG_CYBOI_MODEL, (void*) L"Could not allocate array. The array is null.");
    }
}

//
// size_determiner.c
//

/**
 * Determines the size of the given type.
 *
 * @param p0 the size
 * @param p1 the type
 */
void determine_size(void* p0, void* p1) {

    if (p1 != *NULL_POINTER_STATE_CYBOI_MODEL) {

        int* t = (int*) p1;

        // CAUTION! Do NOT call the logger here.
        // It uses functions causing circular references.
        // log_message_terminated((void*) DEBUG_LEVEL_LOG_CYBOI_MODEL, (void*) L"Determine size.");

        //
        // datetime
        //

        if (*t == *DATETIME_STATE_CYBOI_TYPE) {

            // CAUTION! This type IS NEEDED, e.g. when DEEP copying a part.
            // It is actually a pointer array, of which each
            // pointer references a structure element.
            copy_integer(p0, (void*) POINTER_TYPE_SIZE);

        //
        // element
        //

        } else if (*t == *PART_ELEMENT_STATE_CYBOI_TYPE) {

            // CAUTION! This type IS NEEDED, e.g. when DEEP copying a part
            // or when setting the references of a part
            // for rubbish (garbage) collection.
            // It is actually a pointer array, of which each
            // pointer references a structure element.
            copy_integer(p0, (void*) POINTER_TYPE_SIZE);

        //
        // logicvalue
        //

        } else if (*t == *BOOLEAN_LOGICVALUE_STATE_CYBOI_TYPE) {

            copy_integer(p0, (void*) SIGNED_INTEGER_INTEGRAL_TYPE_SIZE);

        //
        // number
        //

        } else if (*t == *COMPLEX_NUMBER_STATE_CYBOI_TYPE) {

            // CAUTION! This type IS NEEDED, e.g. when DEEP copying a part.
            // It is actually a pointer array, of which each
            // pointer references a structure element.
            copy_integer(p0, (void*) POINTER_TYPE_SIZE);

        } else if (*t == *DOUBLE_NUMBER_STATE_CYBOI_TYPE) {

            copy_integer(p0, (void*) DOUBLE_REAL_TYPE_SIZE);

        } else if (*t == *FRACTION_NUMBER_STATE_CYBOI_TYPE) {

            // CAUTION! This type IS NEEDED, e.g. when DEEP copying a part.
            // It is actually a pointer array, of which each
            // pointer references a structure element.
            copy_integer(p0, (void*) POINTER_TYPE_SIZE);

        } else if (*t == *INTEGER_NUMBER_STATE_CYBOI_TYPE) {

            copy_integer(p0, (void*) SIGNED_INTEGER_INTEGRAL_TYPE_SIZE);

        } else if (*t == *UNSIGNED_LONG_NUMBER_STATE_CYBOI_TYPE) {

            copy_integer(p0, (void*) UNSIGNED_LONG_INTEGER_INTEGRAL_TYPE_SIZE);

        //
        // pointer
        //

        } else if (*t == *POINTER_STATE_CYBOI_TYPE) {

            copy_integer(p0, (void*) POINTER_TYPE_SIZE);

        //
        // text
        //

        } else if (*t == *CHARACTER_TEXT_STATE_CYBOI_TYPE) {

            copy_integer(p0, (void*) SIGNED_CHARACTER_INTEGRAL_TYPE_SIZE);

        } else if (*t == *WIDE_CHARACTER_TEXT_STATE_CYBOI_TYPE) {

            copy_integer(p0, (void*) WIDE_CHARACTER_INTEGRAL_TYPE_SIZE);

        } else {

            // CAUTION! Do NOT call the logger here.
            // It uses functions causing circular references.
            // log_message_terminated((void*) WARNING_LEVEL_LOG_CYBOI_MODEL, (void*) L"Could not determine size. The type is unknown.");
        }

    } else {

        // CAUTION! Do NOT call the logger here.
        // It uses functions causing circular references.
        // log_message_terminated((void*) ERROR_LEVEL_LOG_CYBOI_MODEL, (void*) L"Could not determine size. The type is null.");
    }
}

Answer 1

On line 201 in utf_8_decoder.c, you have

fwprintf(stdout, L"TEST decoder *ds: %i\n", *((int*) ds));

so you treat ds as an int* and that works fine. But on the lines lines 298 and 300 that were indicated by valgrind,

fwprintf(stdout, L"TEST decoder *ds pre mbsnrtowcs as size_t: %i\n", *((size_t*) ds));

int n = mbsnrtowcs((wchar_t*) dd, (const char**) &sd, *((size_t*) sc), *((size_t*) ds), (mbstate_t*) *NULL_POINTER_STATE_CYBOI_MODEL);

you treat ds as a size_t*. An int typically has four bytes, but a size_t has eight bytes on any self-respecting 64-bit system.