[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-dev] Re: [PATCH V2 2/2] virtio-pci: add PASID configuration extended capability
On Wed, Jan 19, 2022 at 06:53:17PM -0500, Michael S. Tsirkin wrote: > > I think we should be explicit about this particular case because someone > > implementing this extension might get it wrong. MSIs should not have a > > PASID because IOMMUs don't support it: > > > > * VT-d treats Requests-with-PASID to the special range 0xfeexxxxx as > > normal DMA (3.14 Handling Requests to Interrupt Address Range) > > * AMD IOMMU reports an error for MSIs with PASID (2.2.7.7 PCIe TLP PASID > > Prefix). > > Ouch. I didn't realize. Really weird, of the "what were they thinking" > variety. The natural thing would be to have PASID==VM and scope > both DMA and MSI within PASID. I guess that is precluded on these > platforms then. > I think the problem is mainly that MSIs appear as normal memory writes requests. If PCI had given them a special type, IOMMU/IRQ remapping hardware could route them more easily, but as is they only have the transaction's address to work with. So vendors made different choices for supporting MSIs with their IRQ remapping hardware. Intel and AMD define reserved address ranges 0xfeexxxxx, where any memory transaction is treated as MSI. This is fine when the kernel is in charge of allocating the I/O address space for DMA, it just needs to make sure IOVAs are not allocated within the address region reserved for MSIs. But it's more complicated with SVA. One of the use of PASID is assigning a partition of device (here a group of virtqueues) to a process. With this approach (SVA) the PASID accesses the process' page tables, so the virtual address space is shared between CPU and IOMMU, IOVA==VA. How would we deal with MSIs in this case, if they had a PASID? On x86, if the IOMMU performed IRQ remapping instead of address translation, on a memory write with PASID to address 0xfeexxxxx, the process couldn't use any address in that range for DMA. The process would need to filter addresses returned by malloc() and treat some of them as non-DMA'able, which defeats the purpose of SVA. To avoid having to do this Intel treats all DMA with PASID as normal memory access, and the process can't accidentally trigger MSIs by using the wrong address. I think AMD behaves the same way but am not entirely sure - the wording in one part of the spec (2.2.7.7) seems to imply that PASID access to the MSI range would trigger an IOMMU error report, in which case the range would need carving out anyway. > > * Arm requires creating mappings to the MSI controller in the SMMU. This > > has implications for SVA where the PASID accesses the whole process > > address space: if MSI transactions have a PASID prefix, that requires > > mapping the MSI controller into the process address space on Arm, which > > isn't convenient. > > I'd like to understand this part a bit better. Can you explain? > We are talking about the IOMMU address space right? > What is wrong with mapping the MSI controller there? > Isn't convenient in what sense? On Arm the IRQ remapping device is separate from the IOMMU, so there is no special address range as far as the IOMMU is concerned. An MSI goes through IOMMU address translation like any other memory write, the IOVA gets translated into a PA that corresponds to the doorbell (MMIO) address of the IRQ remapping device. The IOVA of the MSI is chosen by the OS and can have any value. Now if MSIs had a PASID, then the IOMMU would translate MSI addresses with the page tables corresponding to that PASID, which with SVA correspond to the process page tables. So the kernel would need to create a mapping inside the process' address space, to the PA of the IRQ remapping device. That's the part that is inconvenient (would not be a security issue because accesses to the doorbell from CPUs are ignored according to Arm's platform spec, but it would certainly be awkward to set up). Thanks, Jean
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]