[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-dev] Re: [PATCH V2 2/2] virtio-pci: add PASID configuration extended capability
On Mon, Jan 24, 2022 at 03:26:35PM +0000, Jean-Philippe Brucker wrote: > On Wed, Jan 19, 2022 at 06:53:17PM -0500, Michael S. Tsirkin wrote: > > > I think we should be explicit about this particular case because someone > > > implementing this extension might get it wrong. MSIs should not have a > > > PASID because IOMMUs don't support it: > > > > > > * VT-d treats Requests-with-PASID to the special range 0xfeexxxxx as > > > normal DMA (3.14 Handling Requests to Interrupt Address Range) > > > * AMD IOMMU reports an error for MSIs with PASID (2.2.7.7 PCIe® TLP PASID > > > Prefix). > > > > Ouch. I didn't realize. Really weird, of the "what were they thinking" > > variety. The natural thing would be to have PASID==VM and scope > > both DMA and MSI within PASID. I guess that is precluded on these > > platforms then. > > > > I think the problem is mainly that MSIs appear as normal memory writes > requests. If PCI had given them a special type, IOMMU/IRQ remapping > hardware could route them more easily, but as is they only have the > transaction's address to work with. So vendors made different choices for > supporting MSIs with their IRQ remapping hardware. Intel and AMD define > reserved address ranges 0xfeexxxxx, where any memory transaction is > treated as MSI. > > This is fine when the kernel is in charge of allocating the I/O address > space for DMA, it just needs to make sure IOVAs are not allocated within > the address region reserved for MSIs. But it's more complicated with SVA. > > One of the use of PASID is assigning a partition of device (here a group > of virtqueues) to a process. With this approach (SVA) the PASID accesses > the process' page tables, so the virtual address space is shared between > CPU and IOMMU, IOVA==VA. How would we deal with MSIs in this case, if they > had a PASID? Basically what I would do is what ARM has done ;) > On x86, if the IOMMU performed IRQ remapping instead of > address translation, on a memory write with PASID to address 0xfeexxxxx, > the process couldn't use any address in that range for DMA. The process > would need to filter addresses returned by malloc() and treat some of them > as non-DMA'able, which defeats the purpose of SVA. To avoid having to do > this Intel treats all DMA with PASID as normal memory access, and the > process can't accidentally trigger MSIs by using the wrong address. I > think AMD behaves the same way but am not entirely sure - the wording in > one part of the spec (2.2.7.7) seems to imply that PASID access to the MSI > range would trigger an IOMMU error report, in which case the range would > need carving out anyway. > > > > * Arm requires creating mappings to the MSI controller in the SMMU. This > > > has implications for SVA where the PASID accesses the whole process > > > address space: if MSI transactions have a PASID prefix, that requires > > > mapping the MSI controller into the process address space on Arm, which > > > isn't convenient. > > > > I'd like to understand this part a bit better. Can you explain? > > We are talking about the IOMMU address space right? > > What is wrong with mapping the MSI controller there? > > Isn't convenient in what sense? > > On Arm the IRQ remapping device is separate from the IOMMU, so there is no > special address range as far as the IOMMU is concerned. An MSI goes > through IOMMU address translation like any other memory write, the IOVA > gets translated into a PA that corresponds to the doorbell (MMIO) address > of the IRQ remapping device. The IOVA of the MSI is chosen by the OS and > can have any value. Now if MSIs had a PASID, then the IOMMU would > translate MSI addresses with the page tables corresponding to that PASID, > which with SVA correspond to the process page tables. So the kernel would > need to create a mapping inside the process' address space, to the PA of > the IRQ remapping device. That's the part that is inconvenient (would not > be a security issue because accesses to the doorbell from CPUs are ignored > according to Arm's platform spec, but it would certainly be awkward to set > up). > > Thanks, > Jean The way I'd do it, is probably by an mmap call. Nothing too tricky about that. Gives you a VA or it can get a VA with MMAP_FIXED if process wants to allocate a range. And for a VM, we just naturally use the regular doorbell address. In fact, as you describe it it seems that we do have a similar problem on AMD, we need to preclude VAs in the MSI range somehow. so I guess 2 vendors out of 3 need to carve up the process address range. -- MST
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]