AMD IOMMU [PSARC/2008/561]

[Vikram Hegde]

Hi all,

ok, what I am hearing is that we keep these properties Sun internal and 
maybe use for debugging and testing. So I will keep this in the code but 
will not document it. Is this acceptable to all ?

Vikram

Garrett D'Amore wrote:
> Furthermore, drivers that don't want to re-spin for this should 
> continue to function normally.  The only reason to avoid use of the 
> IOMMU is either performance optimization or to workaround incorrect 
> driver implementation.
>
> As indicated, I don't mind have have some properties "undocumented" 
> usage, but we really should be encouraging folks that care to avoid 
> the IOMMU to use the documented API.
>
> (And, IMO, there really should be no reason to avoid the IOMMU if 
> you've written your code to perform well under regular DDI compliant 
> DMA.  The IOMMU itself affords some significant benefits, making it 
> much easier to get single contiguous DMA segments, reducing the need 
> to deal with multiple cookies -- scatter gather -- or DMA windows.  
> Never mind the huge benefits it affords for virtualization and the 
> fault tolerance benefits that come for free as well.)
>
>    -- Garrett
>
> Wesley Shao wrote:
>> FORCE_PHYSICAL allows finer granularity and the proposed properties
>> make the drivers non-portable to SPARC platforms. They should really be
>> used for internal debug.
>>
>> Besides, they make the drivers hard to run on virtualized platforms.
>>
>> Wes
>>
>> Vikram Hedgde wrote:
>>> Hi,
>>>
>>>> Yes.  I'd prefer not to "document" the alternative mechanism, since 
>>>> I view that as a hack/workaround rather than the correct behavior 
>>>> of using the DMA attributes.
>>> How about 3rd party drivers that driver vendors don't want to rev 
>>> and re-release solely for this. Shouldn't there be a switch that 
>>> they can suggest to customers to use with their existing driver 
>>> binaries ?
>>>
>>> Vikram
>>>
>>>
>>> Garrett D'Amore wrote:
>>> Vikram Hegde wrote:
>>>> Garrett D'Amore wrote:
>>>>> It would be cooler if this could be dealt with using ordinary DMA 
>>>>> attributes.  Won't DDI_DMA_FORCE_PHYSICAL (see the dma_attr_flags 
>>>>> for ddi_dma_attr(9s)) work for this?
>>>>>
>>>> Yes, that would work as well and is the preferred and established 
>>>> solution. This flag will be respected by the IOMMU. However, it 
>>>> seems that an alternative mechanism that involves no touching of 
>>>> the driver code is desirable as well. Do you agree ?
>>>
>>> Yes.  I'd prefer not to "document" the alternative mechanism, since 
>>> I view that as a hack/workaround rather than the correct behavior of 
>>> using the DMA attributes.
>>>
>>>    -- Garrett
>>>>
>>>> Thanks,
>>>> Vikram
>>>>>    - Garrett
>>>>>
>>>>>
>>>>> Vikram Hegde wrote:
>>>>>> Hi Kais,
>>>>>>
>>>>>> Below is the text I am planning to add to the PSARC case. And 
>>>>>> yes, the disable properties will be a part of this PSARC case.
>>>>>>
>>>>>> Vikram
>>>>>>
>>>>>> ===================================================================== 
>>>>>>
>>>>>> Enabling/Disabling the IOMMU
>>>>>> =============================
>>>>>> It is recognized that driver or platform developers may not in 
>>>>>> certain cases desire the
>>>>>> IOMMU to be enabled. To accomodate such needs we propose the 
>>>>>> following global properties
>>>>>> in the iommu driver.conf file
>>>>>>
>>>>>> "global-disable" - can be set to 1 to disable IOMMU platform wide.
>>>>>> "exclude-list"   - list of driver names that want the IOMMU 
>>>>>> disabled.
>>>>>> ====================================================================== 
>>>>>>
>>>>>>
>>>>>> Vikram
>>>>>>
>>>>>> Kais Belgaied wrote:
>>>>>>> Vikram, Jerry,
>>>>>>>
>>>>>>> So, is the ability to disable the IOMMU on a per driver basis in 
>>>>>>> this case's scope?
>>>>>>> what interface is being proposed for doing this?
>>>>>>>
>>>>>>> Jerry, assuming the answer  the to first question above is 
>>>>>>> 'yes', the case cannot be in
>>>>>>> a "closed approved automatic" state while waiting for some of 
>>>>>>> its spec.
>>>>>>>
>>>>>>>    Kais.
>>>>>>>
>>>>>>> On 09/03/08 23:16, Matthew Jacob wrote:
>>>>>>>> Garrett D'Amore wrote:
>>>>>>>>> Vikram Hegde wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>>> Abilities to disable the IOMMU as a choice? Performance 
>>>>>>>>>>> impact on various systems and interactions with other 
>>>>>>>>>>> projects that have gone to some length to work within S/G 
>>>>>>>>>>> and its performance characteristics. How will this affect 
>>>>>>>>>>> them? Cache effects? Interactions with some VM work 
>>>>>>>>>>> currently being undertaken?
>>>>>>>>>>>
>>>>>>>>>>> I can't imagine such a major change being 
>>>>>>>>>>> closed/approved/automatic. 
>>>>>>>>>> Performance is driver dependant. There is no way a single 
>>>>>>>>>> project team can test the gamut of drivers out there and 
>>>>>>>>>> certify that the IOMMU will  give acceptable performance for 
>>>>>>>>>> all needs (which is itself a subjective opinion). We have 
>>>>>>>>>> always planned to provide the ability to disable the IOMMU on 
>>>>>>>>>> a per driver basis. Does that address your performance 
>>>>>>>>>> concerns ?
>>>>>>>>>
>>>>>>>>> Actually, for many cases, IOMMU may *improve* performance.  
>>>>>>>>> This is because it will make it easier to satisfy device 
>>>>>>>>> constraints for DMA resources without requiring the use of a 
>>>>>>>>> bounce buffer or dma windows.
>>>>>>>>
>>>>>>>> That is almost always the exception now. The other issue IIRC 
>>>>>>>> is the interaction between the IOMMU and memory caches. In many 
>>>>>>>> data movement appliance contexts you use scatter-gather 
>>>>>>>> (scatter-gather in name only- contiguous memory is good) to 
>>>>>>>> physical pages which are never mapped in.
>>>>>>>>
>>>>>>>> The main point here is to avoid a situation that we're 
>>>>>>>> currently in now with respect MSI-X and Interrupt/CPU binding 
>>>>>>>> and make sure that there are sufficient control surfaces to 
>>>>>>>> allow for a broad range of possible implementation choices.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> That said, drivers which assume that setup and tear down of 
>>>>>>>>> DMA is "cheap" may take a perf. hit when using an IOMMU.  This 
>>>>>>>>> is already the case for SPARC systems, btw.
>>>>>>>>
>>>>>>>> Except for those (ancient) ones that used 36-bit MBus physical 
>>>>>>>> addresses and bypassed the IOMMU.
>>
>>
>


-- 



===============================================================================

thread(n):projecting helical rib by which parts can be screwed together

===============================================================================