Is Multi Message MSI implemented on Linux / x86?

Question

I am working on a network driver for an FPGA endpoint that supports multi-message MSI interrupts (not msix) on a PCIe bus. The host processor is an x86 Intel i7 620LM running on CentOS with a 4.2 kernel.

The FPGA endpoint correctly advertises multiple msi vectors in it's MSI capabilities register (0x101 = 32 total possible vectors).

From what I can tell, multi-message functionality was added in kernel 4.2. Unfortunately when I call pci_enable_msi_range(pdev, 1, 32); it only returns 1. When I call pci_msi_vec_count(pdev); it returns 32. I'm able to request an irq handler on the one vector and it works as expected.

Does anyone know if multi-message MSI vectors are actually supported in Linux on x86 architectures?

UPDATE: I was able to get all 32 MSI vectors enabled using a different SBC with an i7-4700EQ processor. This with a 4.4-rc1 kernel.

UPDATE: Works with 4.2 as well.

UPDATE: In this case, the problem was in coreboot. I was able to get multiple vectors working once the board vender provided an update.

Answer 1

I had problems using more than one MSI on an Intel machine independently of the Linux kernel version as well. After a lot of fiddling I realized that enabling VT-d in the bios fixed the issue (tested on Linux 4.6 kernel). I think this is because in order to support multiple MSIs on x86, the Linux kernel needs support for interrupt remapping which is one of the features provided by VT-d.

Answer 2

Of course it is. I'm not sure why exactly you get the error. but I've written few drivers that used more than 1 vector.

I just checked my PC at home, I have various drivers which use more the 1 MSI IRQ:

$ cat /proc/interrupts | grep -i msi
 33:    5683962          0          0          0          0          0          0          0   PCI-MSI 524288-edge      radeon
 34:          0          0          0          0          0          0          0          0   PCI-MSI 1572864-edge      xhci_hcd
 35:          0          0          0          0          0          0          0          0   PCI-MSI 1572865-edge      xhci_hcd
 36:          0          0          0          0          0          0          0          0   PCI-MSI 1572866-edge      xhci_hcd
 37:          0          0          0          0          0          0          0          0   PCI-MSI 1572867-edge      xhci_hcd
 38:          0          0          0          0          0          0          0          0   PCI-MSI 1572868-edge      xhci_hcd
 39:          0          0          0          0          0          0          0          0   PCI-MSI 1572869-edge      xhci_hcd
 40:          0          0          0          0          0          0          0          0   PCI-MSI 1572870-edge      xhci_hcd
 41:          0          0          0          0          0          0          0          0   PCI-MSI 1572871-edge      xhci_hcd
 42:    3807594          0          0          0          0          0          0          0   PCI-MSI 512000-edge      0000:00:1f.2
 43:          0          0          0          0          0          0          0          0   PCI-MSI 2097152-edge      xhci_hcd
 44:          0          0          0          0          0          0          0          0   PCI-MSI 2097153-edge      xhci_hcd
 45:          0          0          0          0          0          0          0          0   PCI-MSI 2097154-edge      xhci_hcd
 46:          0          0          0          0          0          0          0          0   PCI-MSI 2097155-edge      xhci_hcd
 47:          0          0          0          0          0          0          0          0   PCI-MSI 2097156-edge      xhci_hcd
 48:          0          0          0          0          0          0          0          0   PCI-MSI 2097157-edge      xhci_hcd
 49:          0          0          0          0          0          0          0          0   PCI-MSI 2097158-edge      xhci_hcd
 50:          0          0          0          0          0          0          0          0   PCI-MSI 2097159-edge      xhci_hcd
 51:     310762          0          0          0          0          0          0          0   PCI-MSI 5242880-edge      0000:0a:00.0
 52:         11          0          0          0          0          0          0          0   PCI-MSI 360448-edge      mei_me
 54:   38991293          0          0          0          0          0          0          0   PCI-MSI 7340032-edge      enp14s0
 55:          1          0          0          0          0          0          0          0   PCI-MSI 32768-edge      i915
 56:       1169          0          0          0          0          0          0          0   PCI-MSI 442368-edge      snd_hda_intel
 57:        152          0          0          0          0          0          0          0   PCI-MSI 526336-edge      snd_hda_intel

$ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    2
Core(s) per socket:    4
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 42
Model name:            Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
Stepping:              7
CPU MHz:               1634.257
CPU max MHz:           3800.0000
CPU min MHz:           1600.0000
BogoMIPS:              6825.67
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              8192K
NUMA node0 CPU(s):     0-7
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb pln pts dtherm tpr_shadow vnmi flexpriority ept vpid xsaveopt