AMD IOMMU [PSARC/2008/561]

[Garrett D'Amore]

Vikram Hegde wrote:
> Garrett D'Amore wrote:
>> It would be cooler if this could be dealt with using ordinary DMA 
>> attributes.  Won't DDI_DMA_FORCE_PHYSICAL (see the dma_attr_flags for 
>> ddi_dma_attr(9s)) work for this?
>>
> Yes, that would work as well and is the preferred and established 
> solution. This flag will be respected by the IOMMU. However, it seems 
> that an alternative mechanism that involves no touching of the driver 
> code is desirable as well. Do you agree ?

Yes.  I'd prefer not to "document" the alternative mechanism, since I 
view that as a hack/workaround rather than the correct behavior of using 
the DMA attributes.

    -- Garrett
>
> Thanks,
> Vikram
>>    - Garrett
>>
>>
>> Vikram Hegde wrote:
>>> Hi Kais,
>>>
>>> Below is the text I am planning to add to the PSARC case. And yes, 
>>> the disable properties will be a part of this PSARC case.
>>>
>>> Vikram
>>>
>>> =====================================================================
>>> Enabling/Disabling the IOMMU
>>> =============================
>>> It is recognized that driver or platform developers may not in 
>>> certain cases desire the
>>> IOMMU to be enabled. To accomodate such needs we propose the 
>>> following global properties
>>> in the iommu driver.conf file
>>>
>>> "global-disable" - can be set to 1 to disable IOMMU platform wide.
>>> "exclude-list"   - list of driver names that want the IOMMU disabled.
>>> ======================================================================
>>>
>>> Vikram
>>>
>>> Kais Belgaied wrote:
>>>> Vikram, Jerry,
>>>>
>>>> So, is the ability to disable the IOMMU on a per driver basis in 
>>>> this case's scope?
>>>> what interface is being proposed for doing this?
>>>>
>>>> Jerry, assuming the answer  the to first question above is 'yes', 
>>>> the case cannot be in
>>>> a "closed approved automatic" state while waiting for some of its 
>>>> spec.
>>>>
>>>>    Kais.
>>>>
>>>> On 09/03/08 23:16, Matthew Jacob wrote:
>>>>> Garrett D'Amore wrote:
>>>>>> Vikram Hegde wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>>> Abilities to disable the IOMMU as a choice? Performance impact 
>>>>>>>> on various systems and interactions with other projects that 
>>>>>>>> have gone to some length to work within S/G and its performance 
>>>>>>>> characteristics. How will this affect them? Cache effects? 
>>>>>>>> Interactions with some VM work currently being undertaken?
>>>>>>>>
>>>>>>>> I can't imagine such a major change being 
>>>>>>>> closed/approved/automatic. 
>>>>>>> Performance is driver dependant. There is no way a single 
>>>>>>> project team can test the gamut of drivers out there and certify 
>>>>>>> that the IOMMU will  give acceptable performance for all needs 
>>>>>>> (which is itself a subjective opinion). We have always planned 
>>>>>>> to provide the ability to disable the IOMMU on a per driver 
>>>>>>> basis. Does that address your performance concerns ?
>>>>>>
>>>>>> Actually, for many cases, IOMMU may *improve* performance.  This 
>>>>>> is because it will make it easier to satisfy device constraints 
>>>>>> for DMA resources without requiring the use of a bounce buffer or 
>>>>>> dma windows.
>>>>>
>>>>> That is almost always the exception now. The other issue IIRC is 
>>>>> the interaction between the IOMMU and memory caches. In many data 
>>>>> movement appliance contexts you use scatter-gather (scatter-gather 
>>>>> in name only- contiguous memory is good) to physical pages which 
>>>>> are never mapped in.
>>>>>
>>>>> The main point here is to avoid a situation that we're currently 
>>>>> in now with respect MSI-X and Interrupt/CPU binding and make sure 
>>>>> that there are sufficient control surfaces to allow for a broad 
>>>>> range of possible implementation choices.
>>>>>
>>>>>>
>>>>>> That said, drivers which assume that setup and tear down of DMA 
>>>>>> is "cheap" may take a perf. hit when using an IOMMU.  This is 
>>>>>> already the case for SPARC systems, btw.
>>>>>
>>>>> Except for those (ancient) ones that used 36-bit MBus physical 
>>>>> addresses and bypassed the IOMMU.
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>