Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

memcpy of overlapping buffers [duplicate]

Tags:

c++

c

memcpy

I ran into strange behavior when using the Aztec linear system solver library. Using valgrind, I found out that this library does a memcpy on overlapping buffers. Specification says that behavior of memcpy on overlapping buffers is not defined.

It turns out that memcpy on many machines has the same behavior as if you would do it with a for loop and therefore you can safely copy from a higher source to a lower destination:

for(int i = 0; i < len; i ++)
  dest[i] = source[i];

BUT on our large cluster, memcpy of overlapping buffers has a different behavior which leads to problems.

Now I wonder whether the overlapping memcpy in the library is normal or just caused by another bug in my code. Since the library is widely used I assume that the memcpy issue should have been discovered earlier. On the other hand, it's still possible that the vast majority of the memcpy implementations behave like the for loop and therefore nobody ever encountered this problem.

  • Can anyone tell me about his experiences with overlapping memcpy on various machines?
  • Which part of my computer system does actually provide memcpy?

I'd like to point out that question is about the practical experience with various implementations, not about what the specification says.

like image 654
Michael Avatar asked Sep 02 '14 18:09

Michael


People also ask

Can memcpy overlap?

The CRT function memcpy doesn't support overlapping memory.

What is memory overlapping in memcpy?

The memcpy copy function shows undefined behavior if the memory regions pointed to by the source and destination pointers overlap. The memmove function has the defined behavior in case of overlapping. So whenever in doubt, it is safer to use memmove in place of memcpy.

What are overlapping buffers?

The overlap buffer keeps the common area, and so calculates the point features in the overlapping area twice. In contrast, the dissolve buffer combines the common area into a single feature, and so calculates the point features in the overlapping area once.

Which is faster memcpy or Memmove?

"memcpy is more efficient than memmove." In your case, you most probably are not doing the exact same thing while you run the two functions. In general, USE memmove only if you have to. USE it when there is a very reasonable chance that the source and destination regions are over-lapping.


Video Answer


2 Answers

memcpy() doesn't support overlapping memory. This allows for optimizations that won't work if the buffers do overlap.

There's not much to really look into, however, because C provides an alternative that does support overlapping memory: memmove(). Its usage is identical to memcpy(). You should use it if the regions might overlap, as it accounts for that possibility.

like image 106
FatalError Avatar answered Oct 21 '22 00:10

FatalError


I've done some research on this in the past... on Linux, up until fairly recently, the implementation of memcpy() worked in a way that was similar enough to memmove() that overlapping memory wasn't an issue, and in my experience, other UNIXs were the same. This doesn't change the fact that this is undefined behavior according to the standard, and you are just lucky that on some platforms it sometimes works -- and memmove() is the standard-supported right answer.

However, in 2010, the glibc maintainers rolled out a new, optimized memcpy() that changed the behavior of memcpy() for some Intel core types where the C standard library is compiled to be faster, but no longer works like memmove() [1]. (I seem to recall also that this is new code triggered only for memory segments larger than 80 bytes). Interestingly, this caused things like the Linux version of Adobe's Flash player to break[2], as well as several other open-source packages (back in 2010 when Fedora Linux became the first to adopt the changed memcpy() in glibc).

  • [1] https://sourceware.org/bugzilla/show_bug.cgi?id=12518
  • [2] https://bugzilla.redhat.com/show_bug.cgi?id=638477
like image 31
JohnH Avatar answered Oct 21 '22 01:10

JohnH