Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Skip/avoid alignment padding bytes when calculating struct checksum

Is there a generic way to skip/avoid alignment padding bytes when calculating the checksum of a C struct?

I want to calculate the checksum of a struct by summing the bytes. The problem is, the struct has alignment padding bytes which can get random (unspecified) values and cause two structs with the identical data to get different checksum values.

Note: I'm mainly concerned about maintainability (adding/removing/modifying fields without the need to update the code) and reusability, not about portability (the platform is very specific and unlikely to change).

Currently, I found a few solutions, but they all have disadvantages:

  1. Pack the struct (e.g. #pragma pack (1)). Disadvantage: I prefer to avoid packing for better performance.
  2. Calculate checksum field by field. Disadvantage: The code will need to be updated when modifying the struct and requires more code (depending on the number of fields).
  3. Set to zero all struct bytes before setting values. Disadvantage: I cannot fully guarantee that all structs were initially zeroed.
  4. Arrange the struct fields to avoid padding and possibly add dummy fields to fill padding. Disadvantage: Not generic, the struct will need to be carefully rearranged when modifying the struct.

Is there a better generic way?

Calculating checksum example:

unsigned int calcCheckSum(MyStruct* myStruct)
{
    unsigned int checkSum = 0; 
    unsigned char* bytes = (unsigned char*)myStruct;
    unsigned int byteCount = sizeof(MyStruct);
    for(int i = 0; i < byteCount; i++)
    {
        checkSum += bytes[i];
    }
    return checkSum;
}
like image 407
Eliahu Aaron Avatar asked Jul 03 '19 15:07

Eliahu Aaron


People also ask

How can we avoid Structure padding?

In Structure, sometimes the size of the structure is more than the size of all structures members because of structure padding. Note: But what actual size of all structure member is 13 Bytes. So here total 3 bytes are wasted. So, to avoid structure padding we can use pragma pack as well as an attribute.

Why do structs use padding?

Padding aligns structure members to "natural" address boundaries - say, int members would have offsets, which are mod(4) == 0 on 32-bit platform. Padding is on by default.

Why does the compiler sometimes insert padding between fields?

The compiler will insert a padding byte after the char to ensure short int will have an address multiple of 2 (i.e. 2 byte aligned).


2 Answers

Is there a generic way to skip/avoid alignment padding bytes when calculating the checksum of a C struct?

There is no such mechanism on which a strictly conforming program can rely. This follows from

  1. the fact that C implementations are permitted to lay out structures with arbitrary padding following any member or members, for any reason or none, and

  2. the fact that

    When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that correspond to any padding bytes take unspecified values.

    (C2011, 6.2.6.1/6)

The former means that the standard provides no conforming way to guarantee that a structure layout contains no padding, and the latter means that in principle, there is nothing you can do to control the values of the padding bytes -- even if you initially zero-fill a structure instance, any padding takes indeterminate values as soon as you assign to that object or to any of its members.

In practice, it is likely that any of the approaches you mention in the question will do the job where the C implementation and the nature of the data permit. But only (2), computing the checksum member by member, can be used by a strictly-conforming program, and that one is not "generic" as I take you to mean that term. This is what I would choose. If you have many distinct structures that require checksumming, then it might be worthwhile to deploy a code generator or macro magic to help you with maintaining things.

On the other hand, your most reliable way to provide for generic checksumming is to exercise an implementation-specific extension that enables you to avoid structures containing any padding (your (1)). Note that this will tie you to a specific C implementation or implementations that implement such an extension compatibly, that it may not work at all on some systems (such as those where misaligned access is a hard error), and that it may reduce performance on other systems.

Your (4) is an alternative way to avoid padding, but it would be a portability and maintenance nightmare. Nevertheless, it could provide for generic checksumming, in the sense that the checksum algorithm wouldn't need to pay attention to individual members. But note also that this also places a requirement for initialization behavior analogous to (3). That would come cheaper, but it would not be altogether automatic.

In practice, C implementations do not wantonly modify padding bytes, but they don't necessarily go out of their way to preserve them, either. In particular, even if you zero-filled rigorously, per your (3), padding is not guaranteed to be copied by whole-structure assignment or when you pass or return a structure by value. If you want to do any of those things then you need to take measures on the receiving side to ensure zero-filling, and requires member-by-member attention.

like image 87
John Bollinger Avatar answered Oct 22 '22 18:10

John Bollinger


This sounds like an XY problem. Computing a checksum for a C object in memory is not usually a meaningful operation; the result is dependent on the C implementation (arch/ABI if not even the specific compiler) and C does not admit a fault-tolerance programming model able to handle the possibility of object values changing out from under you due to hardware faults of memory-safety errors. Checksums make sense mainly for serialized data on disk or in transit over a network where you want to guard against data corruption in storage/transit. And C structs are not for serialization (although they're commonly abused for it). If you write proper serialization routines, you then can just do the checksum on the serialized byte stream.

like image 26
R.. GitHub STOP HELPING ICE Avatar answered Oct 22 '22 17:10

R.. GitHub STOP HELPING ICE