Enable/Disable Hardware Lock Elision

Question

I am using glibc 2.24 version. It has lock elision path included for pthread_mutex_lock implementation with Transactional Synchronization Extensions such as _xbegin() and _xend(). The hardware is supposed to support lock elision as hle CPU flag is for Hardware Lock Elision I think. The processor I am using is Intel(R) Xeon(R) Gold 6130 with Skylake architecture.

First I wanted to disable Lock elision but when I run the program that uses pthread_mutex_lock, with perf stat -T to monitor transactional cycles, I got 0. I assume this means pthread_mutex_lock does not use transactional path at all. After googling I found that it might be necessary to use export GLIBC_TUNABLES=glibc.elision.enable=1 first, to enable lock elision but after this step I still don't see any transactions with perf.

On the other hand when I include _xbegin(); and _xend(); directly in the process, I get some number of transactional cycles with perf stat -T, which should mean that I am looking for the right counters with perf, hopefully.

So any advice to how I can enable lock elision would be helpful. Or am I checking it incorrectly?

Update for TSX I'm using this two instructions in the main function, just like this:

_xbegin();
_xend();

I'm not sure which library it needs, I already have tens of them included. For compilation I'm using the following flags: -O3 -march=native -lpthread that are relevant for this example.

For locks I have mutex:

pthread_mutex_t * mutex;
mutex = (pthread_mutex_t *) malloc(10 * sizeof(pthread_mutex_t));
for(int k=0; k<10; k++){
    pthread_mutex_init(&mutex[k], NULL);
  }

Maybe for elision I should initialize it differently?

Answer 1

In earlier versions of glibc, before 2.27, support for TSX could only be controlled using a compile-time flag called enable-lock-elision. I don't know which versions have enable-lock-elision enabled or disabled, but that's how it used to work¹. So if you wanted to enable/dsiable TSX, you'll have to compile glibc yourself and use that flag accordingly. Starting with glibc 2.27, that compile-time option was removed and replaced by a run-time option called glibc.elision.enable. That is, glibc is always compiled with support for TSX, but TSX will only be used if the environment variable glibc.elision.enable is set to 1 before running the application (e.g., by executing export GLIBC_TUNABLES=glibc.elision.enable=1).

Before 2.27, glibc.elision.enable doesn't exist and so it has no effect. Whether TSX is used depends on the compile-time flag enable-lock-elision. You're using 2.24. So the easiest solution would be to move to 2.27 or a more recent version.

Note that all current Intel processors that support TSX appear to have the same bug that "using Intel TSX Instructions May Lead to Unpredictable System Behavior" according to the respective specification updates. For some processors, Intel has released microcode updates to actually disable TSX. However, the implementation on your processor continues to be enabled.

---

Footnotes:

(1) According to this bug report, lock elision has been disabled in glibc starting with 2.23. I think this is done by disabling enable-lock-elision when building glibc, but I've not verified this by looking at the code. But this is consistent with your observation that it is disabled in 2.24.

Answer 2

To add to the answer from Ana, I think the glibc may have custom variables on the OS intended for. At least on RHEL, "RHEL_GLIBC_TUNABLES="glibc.elision.enable=1" was already introduced in glibc 2.17. It is possible that OP's distro also has a similar environmental variable for enabling HLE at runtime.

%define glibcsrcdir glibc-2.17-c758a686
%define glibcversion 2.17
%define glibcrelease 317%{?dist}
...
# If the architecture has elision support in glibc then it should be listed
# here to enable elision for default pthread mutexes and rwlocks. The elision
# is not enabled automatically and each process has to opt-in to elision via
# the environment variable RHEL_GLIBC_TUNABLES by setting it to enabled e.g.
# RHEL_GLIBC_TUNABLES="glibc.elision.enable=1".
%define elisionarches x86_64 %{power64}
...
%ifarch %{elisionarches}
    --enable-lock-elision=yes \
%endif
...

According to glibc.spec of glibc 2.23.1 for Fedora 24, elisions has been disabled at compile-time since glibc 2.20 except on s390/s390x systems. This is reaffirmed with the glibc.spec of glibc 2.24. for Fedora 25.

# Enable lock elision support for these architectures
#
# At the moment lock elision is disabled on x86_64 until there's a CPU that
# would actually benefit from enabling it.  Intel released a microcode update
# to disable HLE and RTM at boot and the Fedora kernel now applies it early
# enough that keeping lock elision enabled should be harmless, but we have
# disabled it anyway as a conservative measure.
%define lock_elision_arches s390 s390x
...
%ifarch %{lock_elision_arches}
        --enable-lock-elision \
%endif
...
# Disable rwlock elision if --enable-lock-elision is not used.
Patch0052: glibc-disable-rwlock-elision.patch
...
* Tue Sep 30 2014 Siddhesh Poyarekar <[email protected]> - 2.20.90-6
- Sync with upstream master.
- Disable more Intel TSX usage in rwlocks (#1146967).
- Enable lock elision again on s390 and s390x.
- Enable Systemtap SDT probes for all architectures (#985109).

while it's not conclusive to say that the OP's Linux distro runs on the same glibc 2.24 as Fedora 25 does, it is likely that the generic glibc 2.24 has elision disabled as per Patch0052. That's why OP couldn't see performance gain with "export GLIBC_TUNABLES=glibc.elision.enable=1"

source:
http://rpmfind.net/linux/RPM/centos/7.9.2009/x86_64/Packages/glibc-2.17-317.el7.x86_64.html
http://rpm.pbone.net/info_idpl_48715292_distro_fedora_24_com_glibc-2.23.1-12.fc24.x86_64.rpm.html
http://rpm.pbone.net/resultsb_dist_103_size_26149846_name_glibc-2.24-10.fc25.src.rpm.html

update:
SLES12 has its own glibc versions. SP2 comes with glibc 2.22, which is different from the glibc 2.22 from Fedora 25.

SLES12 SP2	Fedora 25
\	Patch0052: glibc-disable-rwlock-elision.patch (since 2.20.90)
elision=--enable-lock-elision if [ "$dirname" = "noelision" ]; then elision=--disable-lock-elision fi	\

source:
https://www.suse.com/support/kb/doc/?id=000019071 http://rpm.pbone.net/info_idpl_55157770_distro_opensuse_com_glibc-2.24-3.29.x86_64.rpm.html