How does free calculate used memory?

Question

How does free calculate used memory and why does it differ from what /proc reports?

# cat /proc/*/status | grep VmSize | awk '{sum += $2} END {print sum}'
281260

But free says:

# free
             total       used       free     shared    buffers     cached
Mem:        524288     326488     197800          0          0          0

Who is right? Is 281260kb memory used or 326488kb?

Answer 1

The title asks: "How does free calculate used memory?"

Answer: It asks the OS, which has to keep track of that to do its job.

More specifically, it asks the memory management subsystem. As sheepsimulator notes in the comments, the Linux kernel exposes all kinds OS maintained data in the /proc virtual filesystem, but every full service OS has to keep track of them kind of data, so it is a small matter to provide an API for free to use.

The question asks: "Why does this differ from adding up the VmSize reported for all processes?"

Answer: There are at least to thing going on here

1. Linux will promise memory to a program without actually allocating it. When you do char *p=new(1024*1024*1024*sizeof(char)); the kernel doesn't go out to get you a gigabyte right away. It just says "OK", and figures it will grab it when you start using it. Thus the need for the infamous OOM killer.
2. Dynamic libraries are shared, and a single page of real memory can be mapped into the virtual address space of more than one process.

Furthermore, your pass over the proc filesystem is not atomic.

The upshot is that the output of free more accurately reflects the use of physical memory on your machine at a given moment.

Answer 2

By 'free' I assume you mean the standard Linux version, which is usually comes from the procps suite of command line tools. Different versions of free (such as the one from busybox) report different numbers.

The procps version of 'free' obtains information about the system memory by reading /proc/meminfo. There is a syscall (sysinfo) which is also available to get memory numbers from the kernel. This can be used if a system does not have the /proc filesystem, but that is rare outside of deeply embedded systems, and procps free does not use that syscall to my knowledge.

The calculation for "used" memory is derived by taking the total memory, and subtracting free memory, cached memory, reclaimable slab memory and buffer memory. The formula, using the names from /proc/meminfo, is:

used = MemTotal - MemFree - Cached - SReclaimable - Buffers

Note that free does not reference the Vm* values for any individual processes. These are numbers for virtual memory usage, which likely do not match the physical memory usage for a process. The numbers that free reports are for physical memory.

Answer 3

The result from 'free' is more likely accurate than adding up the Virtual Memory size of each process (which is just virtual memory afterall, might even add up to more memory than is physically present!)

/proc/meminfo will give you some more of the details than 'free'.