Effects of __attribute__((packed)) on nested array of structures?

Question

The Problem

I'm working on sending a raw structure over a network to a known program on the other side, but have to worry about the silently introduced memory used for aligning the structures (other issues like endianness are covered). What I'm working with is something like:

typedef struct __attribute__((packed))
{
   uint16_t field1;
   uint16_t field2;
   uint16_t field3;
} packed_1_s;

typedef struct __attribute__((packed))
{
   uint16_t fieldA;
   uint16_t fieldB;
   packed_1_s structArray[10];
} packed_2_s;

typedef struct __attribute__((packed))
{
   uint16_t fieldX;
   packed_2_s fieldY;
   uint8_t arrayZ[20];
} data_s;

I understand that normally the packed_1_s structure could/would have additional space allocated for every instance of the structure to fill it out to the compiler's favored size (dependent on the hardware it's being built for), and that favored size can be anywhere from 2 bytes to 64 bytes (most recently). Normally if I had a single instance of packed_1_s in packed_2_s there would be no problem, but I'm given to understand there's some differences when you try to put elements in an array.

Attempted Solutions

The gcc documentation seems to suggest that by simply including the packed attribute in the packed_2_s definition, the fields, even if they're arrays, will all be as tightly packed as possible and won't add space to the packed_2_s structure to align the elements of the array. The documentation on the align() attribute though suggests that arrays are handled differently than other fields and need the align/packed attribute set on the field directly for it to modify the extra spacing added to match the specified alignment (or lack thereof). I tried setting the packed attribute on both the structArray field, and when that didn't work, did a test by setting the packed attribute on arrayZ in the code above:

packed_1_s structArray[10] __attribute__((packed));

uint8_t arrayZ[20] __attribute__((packed));

Both attempts gave me a compiler warning that the packed attribute wasn't understood in this context and would be skipped (good thing I build with "-Wall").

I hoped a way to get around the issue would be to use attribute align(1), indicating a desired alignment of 1 byte which is comparable to the packed attribute, but documentation says the align() attribute can only increase the alignment and packed should be used to decrease the alignment.

Considerations

From what I can determine from the GCC documentation, it seems like there's 3 major cases of additional memory being inserted.

1. Structure contents may have additional memory allocated to the structure itself to change the spacing between fields.
   Effectively, the definition of the memory map of the structure contents within the structure may change (though not the order of the elements).
2. Structures may have additional memory allocated to them to fill them out to a more efficient overall size. This is generally intended so other variables coming after a declaration of one of their instances doesn't fall within the same "block" as the structure instance where a "block" is defined by the system/compiler.
3. Arrays, whether within a structure or not, may have additional memory added to them to shift the elements to an efficient alignment.

As far as I can tell, the packed attribute can be used to affect the structures and block the additional memory added in case 1 and 2 above, but there doesn't seem to be a way to handle case 3 above on my compiler(s).

The Question

Is there any way to guarantee that the data_s structure will have absolutely no additional space added to it or any of its sub-structures so I don't have compiler dependent shifts in the memory map? Am I misunderstanding the cases where the compiler can insert space to intentionally shift the memory map?

EDIT

I discussed some of the issues with my local guru and it sounds like I have some misunderstanding of case 3 above. The elements in the array don't have space inserted between them, but the additional space to guarantee they align correctly is added to the structure itself. Apparently this suggests things like "sizeof(structureOnlyContaining_uint32_t)" won't always return "4" since additional space may be added to align the uint32_t datatype on the compiler being used. The result is that there are really only 2 cases:

1. Larger offsets in between fields in the memory map of the structure.
   The space between the fields can be modified to align each field. This can be changed using the packed or align() attributes.
2. Structure end padding. The size for a structure, as returned by sizeof(), can be modified so arrays of the structures end up correctly aligned for the system. This allows all systems to assume that the start of structures will always be aligned, causing issues if they aren't. This seems unaffected by the pack or align attributes.

Because of the new case 2, elements of an array in a structure don't necessarily obey the packed or align() attributes specified on the structure, though the start of the array and the field immediately following the array do.

My question is then about how to deal with the structArray in packed_2_s since the size of the array as a whole cannot be guaranteed purely by the packed attribute. Is there some way to guarantee the fixed size of the structArray field as a whole? It should be noted that I can't increase the size of the packed_1_s too much since the data_s struct needs to be kept as small as possible (its replacing Audio/Video data in a streaming scenario).

Answer 1

Note the following points about __attribute__((packed)):

• When packed is used in a structure declaration, it will compress its fields such, such that, sizeof(structure) == sizeof(first_member) + ... + sizeof(last_member).

• Here, an array is just one member of the struct. Packing the containing structure of an array will not change the array's size. In fact, the size of (any) array is always sizeof(element) * number_of_elements.

• Similarly, packing a containing structure of an inner structure will not change the size of the inner structure. The size of a structure is completely determined by its declaration, and is the same no matter where you use.

• Packing a structure will make its required alignment one byte (i.e. it can be placed anywhere in memory).

• Packing will introduce alignment issues when accessing the fields of a packed structure. The compiler will account for that when the the fields are accessed directly, but not when they are accessed via pointers. Of course, this does not apply to fields with required alignment one (such as char's or other packed structures). See my answer to a similar question, which includes a program demonstrating the problem with accessing members via pointers.

Finally, to answer the question,

Is there any way to guarantee that the data_s structure will have absolutely no additional space added to it or any of its sub-structures so I don't have compiler dependent shifts in the memory map?

Yes. Declare the structure as packed, and also all structures that it contains, recursively.

Also note that the packed attribute applies to a structure declaration, and not to a type. There's no such thing as packed version of a structure that is declared non-packed. When you use a structure somewhere, it (its members) will be packed if and only if the structure itself was declared packed. This is kind of implied by the fact that the size of a structure is completely determined by its declaration.

UPDATE: For some reason you're still confused about arrays. The solution I provided (declare all structures packed) works with arrays too. For example:

struct elem_struct {
    uint32_t x;
} __attribute__((packed));
// packed guarantees that sizeof(struct elem_struct) = sizeof(uint32_t) = 4

struct array_struct {
    struct elem_struct arr[10];
} __attribute__((packed));
// packed guarantees that sizeof(struct array_struct) =
// = sizeof(struct elem_struct[10]) = 10 * sizeof(struct elem_struct)
// = 10 * 4 = 40

The two additional points you made about arrays are true - but only when the structures are not packed. Packing forces the fields of the struct to be continuous, and this does create alignment issues which, if no packing was used, would be solved by inserting empty space between members and padding the struct (see the point I already raised about alignment).