Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What does the PCOMMIT instruction do?

In the Intel ISA extension manual the description for pcommit is a bit cryptic:

The PCOMMIT instruction causes certain store-to-memory operations to persistent memory ranges to become persistent (power failure protected). Specifically, PCOMMIT applies to those stores that have been accepted to memory.
[...]
If PCOMMIT is executed after a store to a persistent memory range is accepted to memory, the store becomes persistent when the PCOMMIT becomes globally visible.
[...]
The data in a store to persistent memory becomes persistent (durable) only after it has either been written to the targeted non-volatile device, or to some intermediate power-fail protected storage/buffer.

It names concepts like persistent memory ranges, stores accepted to memory, stores becoming persistent and non-volatile device1.

What is the exact context?


1 This cannot be the classical NV devices like the NOR Flash ROM or NVMe devices (read: new SSDs) as they are behind a variable number of bridges, including subtractive decoding ones, that the CPU has no control over.

like image 640
Margaret Bloom Avatar asked Jan 26 '17 16:01

Margaret Bloom


1 Answers

First of all pcommit has been deprecated before even shipping to an actual CPU.
Most of this answer is based on the content of the link above.


Intel, in conjunction with Micron, developed a new form of Non-volatile memory (NVM) called 3D XPoint (from its internal structure).
An actual implementation, as a disk cache, is already available and Intel started to prepare for a wider adoption of its NVM technology a while ago.

Particularly Intel imagined that some of the DIMMs could contain a portion made with 3D XPoint technology and thus constitute a non-volatile device.

This would make one or more memory ranges persistent, the collection of these persistent ranges is called the persistent domain.
One of the main features of the persistent domain is its ability to be power-fail safe.

When a store is made it goes through:

  • The store buffer.
    The store is completed/visible locally, but not globally.
    The store buffer can be flushed with different instructions (e.g. sfence).
  • The cache hierarchy.
    The store is globally visible (the cache coherence protocol ensure this).
    The cache can be flushed with different instructions (e.g. clflush, clflushopt, clwb, et al).
  • The memory controller Write Pending Queue (WPQ).
    The store is accepted to memory but it is not written to the DIMMs yet.
    The WPQ can be flushed through specific PCIe configuration registers of the memory controller or with pcommit.
  • The memory.
    The store is committed/written in the memory.

At what point of the data path above the store is in the persistent domain and thus will not be lost in case of power failure?
Some memory controller has a feature called Asynchronous DRAM Refresh that ensures that even in the case of power-loss the WPQ is flushed correctly (thanks to a battery for example).
For these platforms, the persistent domain starts at the WPQ.

Intel, however, was concerned that not all platforms would have had the ADR feature and created the pcommit instruction as a way to be sure that stores entered the persistence domain (pcommit is executable in user mode).

This is how a store was intended to be made persistent

mov [X], rax     ;Store

;Here the store has started moving to the store buffer

clwb [X]

;Here the store has moved to the cache (CLWB is ordered with previous stores) 
;and then starting moving to the memory controller WPQ 
;(the line containing X has been written back)

sfence           ;Wait for CLWB to become globally visible

;Here the store is in the WPQ

pcommit         

;The store is being committed

sfence          ;Wait for pcommit to become globally visible

;The store is committed

It turned out that every platform that is planning to support the new Intel NVM technology is also planning to support ADR, so Intel deprecated pcommit in favour of a simpler programming model:

mov [X], rax
clwb [X]
sfence         

;Here the store is in the WPQ and that's enough
like image 60
Margaret Bloom Avatar answered Nov 01 '22 04:11

Margaret Bloom