How to choose AVX compare predicate variants

Question

In the Advanced Vector Extensions (AVX) the compare instructions like _m256_cmp_ps, the last argument is a compare predicate. The choices for the predicate overwhelm me. They seem to be a tripple of type, ordering, signaling. E.g. _CMP_LE_OS is 'less than or equal, ordered, signaling.

For starters, is there a performance reason for selecting signaling or non signaling, and similarly, is ordered or unordered faster than the other?

And what does 'non signaling' even mean? I can't find this in the docs at all. Any rule of thumb on when to select what?

Here are the predicate choices from avxintrin.h:

/* Compare */ #define _CMP_EQ_OQ    0x00 /* Equal (ordered, non-signaling)  */ #define _CMP_LT_OS    0x01 /* Less-than (ordered, signaling)  */ #define _CMP_LE_OS    0x02 /* Less-than-or-equal (ordered, signaling)  */ #define _CMP_UNORD_Q  0x03 /* Unordered (non-signaling)  */ #define _CMP_NEQ_UQ   0x04 /* Not-equal (unordered, non-signaling)  */ #define _CMP_NLT_US   0x05 /* Not-less-than (unordered, signaling)  */ #define _CMP_NLE_US   0x06 /* Not-less-than-or-equal (unordered, signaling)  */ #define _CMP_ORD_Q    0x07 /* Ordered (nonsignaling)   */ #define _CMP_EQ_UQ    0x08 /* Equal (unordered, non-signaling)  */ #define _CMP_NGE_US   0x09 /* Not-greater-than-or-equal (unord, signaling)  */ #define _CMP_NGT_US   0x0a /* Not-greater-than (unordered, signaling)  */ #define _CMP_FALSE_OQ 0x0b /* False (ordered, non-signaling)  */ #define _CMP_NEQ_OQ   0x0c /* Not-equal (ordered, non-signaling)  */ #define _CMP_GE_OS    0x0d /* Greater-than-or-equal (ordered, signaling)  */ #define _CMP_GT_OS    0x0e /* Greater-than (ordered, signaling)  */ #define _CMP_TRUE_UQ  0x0f /* True (unordered, non-signaling)  */ #define _CMP_EQ_OS    0x10 /* Equal (ordered, signaling)  */ #define _CMP_LT_OQ    0x11 /* Less-than (ordered, non-signaling)  */ #define _CMP_LE_OQ    0x12 /* Less-than-or-equal (ordered, non-signaling)  */ #define _CMP_UNORD_S  0x13 /* Unordered (signaling)  */ #define _CMP_NEQ_US   0x14 /* Not-equal (unordered, signaling)  */ #define _CMP_NLT_UQ   0x15 /* Not-less-than (unordered, non-signaling)  */ #define _CMP_NLE_UQ   0x16 /* Not-less-than-or-equal (unord, non-signaling)  */ #define _CMP_ORD_S    0x17 /* Ordered (signaling)  */ #define _CMP_EQ_US    0x18 /* Equal (unordered, signaling)  */ #define _CMP_NGE_UQ   0x19 /* Not-greater-than-or-equal (unord, non-sign)  */ #define _CMP_NGT_UQ   0x1a /* Not-greater-than (unordered, non-signaling)  */ #define _CMP_FALSE_OS 0x1b /* False (ordered, signaling)  */ #define _CMP_NEQ_OS   0x1c /* Not-equal (ordered, signaling)  */ #define _CMP_GE_OQ    0x1d /* Greater-than-or-equal (ordered, non-signaling)  */ #define _CMP_GT_OQ    0x1e /* Greater-than (ordered, non-signaling)  */ #define _CMP_TRUE_US  0x1f /* True (unordered, signaling)  */ 

Answer 1

Ordered vs Unordered has to do with whether the comparison is true if one of the operands contains a NaN (see What does ordered / unordered comparison mean?). Signaling (S) vs non-signaling (Q for quiet?) will determine whether an exception is raised if an operand contains a NaN.

From a performance perspective, these should all be the same (assuming of course no exceptions are raised). If you want to be alerted when there's a NaN, then you want signaling. As for ordered vs unordered, it all depends on how you want to deal with NaNs.