Linux device driver to allow an FPGA to DMA directly to CPU RAM

Question

I'm writing a linux device driver to allow an FPGA (currently connected to the PC via PCI express) to DMA data directly into CPU RAM. This needs to happen without any interaction and user space needs to have access to the data. Some details: - Running 64 bit Fedora 14 - System has 8GB of RAM - The FPGA (Cyclone IV) is on a PCIe card

In an attempt to accomplish this I performed the following: - Reserved the upper 2GB of RAM in grub with memmap 6GB$2GB (will not boot is I add mem=2GB). I can see that the upper 2GB of RAM is reserved in /proc/meminfo - Mapped BAR0 to allow reading and writing to FPGA registers (this works perfectly) - Implemented an mmap function in my driver with remap_pfn_range() - Use ioremap to get the virtual address of the buffer - Added ioctl calls (for testing) to write data to the buffer - Tested the mmap by making an ioctl call to write data into the buffer and verified the data was in the buffer from user space

The problem I'm facing is when the FPGA starts to DMA data to the buffer address I provide. I constantly get PTE errors (from DMAR:) or with the code below I get the following error: DMAR: [DMA Write] Request device [01:00.0] fault addr 186dc5000
DMAR: [fault reason 01] Present bit in root entry is clear DRHD: handling fault status reg 3

The address in the first line increments by 0x1000 each time based on the DMA from the FPGA

Here's my init() code:

#define IMG_BUF_OFFSET     0x180000000UL // Location in RAM (6GB)
#define IMG_BUF_SIZE       0x80000000UL  // Size of the Buffer (2GB)

#define pci_dma_h(addr) ((addr >> 16) >> 16)
#define pci_dma_l(addr) (addr & 0xffffffffUL)

if((pdev = pci_get_device(FPGA_VEN_ID, FPGA_DEV_ID, NULL)))
{
    printk("FPGA Found on the PCIe Bus\n");

    //  Enable the device 
    if(pci_enable_device(pdev))
    {
        printk("Failed to enable PCI device\n");
        return(-1);
    }
    //  Enable bus master
    pci_set_master(pdev);

    pci_read_config_word(pdev, PCI_VENDOR_ID, &id);
    printk("Vendor id: %x\n", id);
    pci_read_config_word(pdev, PCI_DEVICE_ID, &id);
    printk("Device id: %x\n", id);
    pci_read_config_word(pdev, PCI_STATUS, &id);
    printk("Device Status: %x\n", id);
    pci_read_config_dword(pdev, PCI_COMMAND, &temp);
    printk("Command Register : : %x\n", temp);
    printk("Resources Allocated :\n");
    pci_read_config_dword(pdev, PCI_BASE_ADDRESS_0, &temp);
    printk("BAR0 : %x\n", temp);

  // Get the starting address of BAR0
  bar0_ptr = (unsigned int*)pcim_iomap(pdev, 0, FPGA_CONFIG_SIZE);
  if(!bar0_ptr)
  {
     printk("Error mapping Bar0\n");
     return -1;
  }
  printk("Remapped BAR0\n");

  // Set DMA Masking
  if(!pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) 
  {
     pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
     printk("Device setup for 64bit DMA\n");
  }
  else if(!pci_set_dma_mask(pdev, DMA_BIT_MASK(32)))
  {
     pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
     printk("Device setup for 32bit DMA\n");
  }
  else
  {
     printk(KERN_WARNING"No suitable DMA available.\n");
     return -1;
  }

  // Get a pointer to reserved lower RAM in kernel address space (virtual address)
  virt_addr = ioremap(IMG_BUF_OFFSET, IMG_BUF_SIZE);
  kernel_image_buffer_ptr = (unsigned char*)virt_addr;
  memset(kernel_image_buffer_ptr, 0, IMG_BUF_SIZE);
  printk("Remapped image buffer: 0x%p\n", (void*)virt_addr);

}

Here's my mmap code:

unsigned long image_buffer;
unsigned int  low;
unsigned int  high;

if(remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
                   vma->vm_end - vma->vm_start,
                   vma->vm_page_prot))
{
   return(-EAGAIN);
}

image_buffer = (vma->vm_pgoff << PAGE_SHIFT);

if(0 > check_mem_region(IMG_BUF_OFFSET, IMG_BUF_SIZE))
{
   printk("Failed to check region...memory in use\n");
   return -1;
}

request_mem_region(IMG_BUF_OFFSET, IMG_BUF_SIZE, DRV_NAME);

// Get the bus address from the virtual address above
//dma_page   = virt_to_page(addr);
//dma_offset = ((unsigned long)addr & ~PAGE_MASK);
//dma_addr   = pci_map_page(pdev, dma_page, dma_offset, IMG_BUF_SIZE, PCI_DMA_FROMDEVICE);    
//dma_addr = pci_map_single(pdev, image_buffer, IMG_BUF_SIZE, PCI_DMA_FROMDEVICE);   
//dma_addr = IMG_BUF_OFFSET;
//printk("DMA Address: 0x%p\n", (void*)dma_addr);

// Write start or image buffer address to the FPGA
low  = pci_dma_l(image_buffer);
low &= 0xfffffffc;
high = pci_dma_h(image_buffer);
if(high != 0)
   low |= 0x00000001;

*(bar0_ptr + (17024/4)) = 0;

//printk("DMA Address LOW : 0x%x\n", cpu_to_le32(low));
//printk("DMA Address HIGH: 0x%x\n", cpu_to_le32(high));
*(bar0_ptr + (4096/4))  = cpu_to_le32(low); //2147483649; 
*(bar0_ptr + (4100/4))  = cpu_to_le32(high); 
*(bar0_ptr + (17052/4)) = cpu_to_le32(low & 0xfffffffe);//2147483648;

printk("Process Read Command: Addr:0x%x Ret:0x%x\n", 4096, *(bar0_ptr + (4096/4)));
printk("Process Read Command: Addr:0x%x Ret:0x%x\n", 4100, *(bar0_ptr + (4100/4)));
printk("Process Read Command: Addr:0x%x Ret:0x%x\n", 17052, *(bar0_ptr + (17052/4)));
return(0);

Thank you for any help you can provide.

Answer 1

Do you control the RTL code that writes the TLP packets yourself, or can you name the DMA engine and PCIe BFM (bus functional model) you are using? What do your packets look like in the simulator? Most decent BFM should trap this rather than let you find it post-deploy with a PCIe hardware capture system.

To target the upper 2GB of RAM you will need to be sending 2DW (64-bit) addresses from the device. Are the bits in your Fmt/Type set to do this? The faulting address looks like a masked 32-bit bus address, so something at this level is likely incorrect. Also bear in mind that because PCIe is big-endian take care when writing the target addresses to the PCIe device endpoint. You might have the lower bytes of the target address dropping into the payload if Fmt is incorrect - again a decent BFM should spot the resulting packet length mismatch.

If you have a recent motherboard/modern CPU, the PCIe endpoint should do PCIe AER (advanced error reporting), so if running a recent Centos/RHEL 6.3 you should get a dmesg report of endpoint faults. This is very useful as the report capture the first handful of DW's of the packet to special capture registers, so you can review the TLP as received.

In your kernel driver, I see you setup the DMA mask, that is not sufficient as you have not programmed the mmu to allow writes to the pages from the device. Look at the implementation of pci_alloc_consistent() to see what else you should be calling to achieve this.

Answer 2

If you are still looking for a reason, then it goes like this: Your kernel has DMA_REMAPPING flags enabled by default, thus IOMMU is throwing the above error, as IOMMU context/domain entries are not programmed for your device.

You can try using intel_iommu=off in the kernel command line or putting IOMMU in bypass mode for your device. Regards, Samir