pci_enable_device() fails after remove/rescan

Question

I have here Linux 4.4 (I used to work on an older kernel which failed in the same way) with a PCIe connected FPGA device and a driver for it which are both of my own design. These have been working well under normal conditions, but now I try to make them work in hotplug conditions. This isn't actual hardware hotplug, what I have been trying is the usual echo 1 >remove in the device's sysfs directory and echo 1 >/sys/bus/pci/rescan afterwards.

After the device reappears, my driver's initialization calls pci_enable_device() which fails while logging:

otscan 0000:02:00.0: can't enable device: BAR 0 [mem 0xf7e01000-0xf7e013ff] not claimed
otscan 0000:02:00.0: can't enable device: BAR 1 [mem 0xf7e00000-0xf7e00fff] not claimed
otscan 0000:02:00.0: can't enable device: BAR 2 [mem 0xf0200000-0xf020ffff 64bit pref] not claimed

(Normally it would stop after the first unclaimed resource, but I have modified it to go on and confirm that in fact all BARs are unclaimed.)

"Not claimed" here means that the struct resource exists but has no parent, which from what I gather is caused by request_resource() never having been called on it. I do not think this is a driver issue as the initialization routine doesn't get to do lot before it aborts due to the failure to enable the device.

This leaves the FPGA (Altera Cyclone V with hard IP PCIe core) and something I might have done wrong there, such as mishandling bus reset somehow. Other PCIe device in that computer work when replugged through sysfs.

I have been digging around for a while and still haven't figured out what why my device is treated differently by Linux. What possible property of my device could have Linux decide to not call request_resource() on my device's BARs?

Answer 1

It looks like I found the cause. I have left the class code as 0 (which is invalid) in the PCIe core configuration which worked fine when the device was present at boot. Putting in a sensible value (0x40000 for multimedia video device in my case, 0xff0000 for "unregistered device" also worked) also made it work on hotplug.

It appears Linux only partially handles devices with a 0 class code.

Answer 2

the problem seems to be incorrectly defined class in the PCIe configuration space of the FPGA's PCIe Core. make sure upper byte of CLASS REGISTER is st to something different than 0

we had a similar problem when looking through dmesg we got: "can't enable device: BAR 0 ... ... not claimed" followed by "pci_enable_device failed"