Safe to pass struct without repr(C) to C API?

Question

New question per Shepmaster's advice in comments

Suppose a library defines a struct without #[repr(C)]. If the struct needs to be passed to a C API, is there a safe way to do it? Should I be worried that the Rust compiler will change the memory layout of the struct in a way the C API doesn't expect?

I've seen some libraries do this. In my experience, they use mem::transmute when it's time to pass the struct to a C function. Does transmute somehow eliminate the need for #[repr(C)]?

If I were the one defining the structs, I wouldn't be asking. I'd just add #[repr(C)]. The problem is that I want to use structs from libraries I don't control.

Old question

I've been using cgmath-rs, and I might switch to nalgebra if cgmath is being abandoned.

In the source of these two libraries, I don't see #[repr(C)] on the vector and matrix structs. So how are these libraries compatible with the OpenGL API? OpenGL expects a certain memory layout when you pass it pointers to vectors and matrices. It's my understanding that without #[repr(C)], the memory layout of Rust structs is undefined.

In at least one example, I've seen mem::transmute being applied to these structs just before handing them to OpenGL. I could be misunderstanding transmute, but that function would seem to preserve the memory layout of the struct. So if the layout is wrong to begin with, it's still wrong after transmute, correct?

I've also considered vecmath. But it appears that, by design, vecmath lacks helper functions to generate rotation matrices and such. True, I could implement those, but it would be nice not to have to. In any case, does vecmath's design avoid the problem of memory layout by using arrays instead of structs?

Answer 1

I've seen some libraries do this. In my experience, they use mem::transmute when it's time to pass the struct to a C function. Does transmute somehow eliminate the need for #[repr(C)]?

No.

...but it's complicated.

If you have a &Foo, and you pass it to C and do not edit it at all, then it's perfectly valid to convert your &Foo to a *const c_void and pass it to a c call, something like:

let fp = &foo as *const Foo as *const c_void

You may see people doing this as one step, using transmute;

unsafe { ffi_call(transmute(&foo), ...) }

...but it's important to understand the transmute call does not modify the memory layout; but it does consume values.

So for example, this code may result in a segfault later:

{
  let foo = Foo { ... }
  unsafe { ffi_call(&foo as _ as *const c_void); }
}

This is because the pointer &foo points to foo; but foo stops existing after the scope ends; so if it gets used later (eg. ffi call keeps a reference) a segfault will result.

You might think that boxing (ie. move to the heap) fixes this:

let foo = Box::new(Foo { ... })
unsafe { ffi_call(&*foo as *const c_void); }

...but it does not; because when the Box leaves scope it is dropped. However, because transmute moves values; this code safely moves the foo instance into the ffi call for use at any later time; note though that this is a memory leak if you don't recover the value later:

let foo = Box::new(Foo { ... })
unsafe { ffi_call(transmute(foo)); }

...but no use of transmute() will solve the absence of repr(C), and yes, you can expect rust to mess around with your struct layout; this is usually to do with drop flags, and may be resolved as part of https://github.com/rust-lang/rfcs/pull/320, since drop flags are currently the only tangible different in memory layout to structs; but since it's not explicitly covered in that RFC, I wouldn't hold my breath.

ie. tldr; If you need to pass a struct and modify it in C, it needs repr(C); if it doesn't, it won't work**.

Transmute is used for other reasons, totally unrelated to this.

** --> well, it may work, but what you're really hitting is undefined behaviour. If the libraries you're using work, it's probably because it happens to work. This is true of many things in rust; eg. mutable aliasing in certain circumstances... but it really means broken code.