AMD IOMMU [PSARC/2008/561]

[Matthew Jacob]

Vikram Hegde wrote:
> Hi Matthew,
>> No, sorry.
> Any reason why FORCE_PHYSICAL is not acceptable ? It has been around 
> for the SPARC side for some time now and seems to meet the needs of 
> everyone on that side. It provides exactly the same functionality as 
> you want.
>
Because it changes the effective semantics of what it was previously. 
Previously it was a performance hack to get 36 bit MBus addresses and 
get around the IOMMU limitations (at least IIRC) and as such it 
encapsulated a semantic that was specific to SPARC. So much so that the 
man page even mentions it.

S/G DMA on the other hand, has been specific more to non-SPARC and has 
been the default there. Turning around and now making it /not/ the 
default on the /hope/ that those who don't want that change will get 
around to recompiling and changing their drivers ignores the intent of 
what the DDI is about.

In short, similarity in names of objects does not mean similarity in 
effective and assumed semantics.

In general I think it's bad policy for Sun to create interfaces that 
consumers of those interfaces cannot disable if there are workable 
alternates. It's even more problematic when the default underlying 
implementation just changes. Furthermore, even if you classified such 
interfaces as evolving, things that have been long standing have at 
least some expectation of remaining the same from (minor) release to 
release.

The /whole /point of the DDI is to create a framework that is friendly 
and usable and reliable for third parties to use. It does /not/ exist to 
have private side interfaces for Sun internal use only, although nearly 
all of the DDI has ended up that way (which was /not/ the intent of some 
of us when we ent through the effort of starting this).

You have in some senses two goals here now, though. The first is the DDI 
idea. The second is to improve the underlying framework in a way that is 
believable and measurable and helps support broadest framework within 
Sun itself. As such, the IOMMU addition is probably a good idea- 
certainly the people proposing it believe so or they wouldn't be doing it.

However, if you're not making this a major release type of change (in 
which case, IMHO, such as it is, lots of things should be thrown out and 
redone from scratch), then the addition of the IOMMU as the underlying 
transport instead of S/G DMA should be done in a less invasive way so 
that previous binaries don't actually do anything different (as a 
default). If it turns out that they are /provably/ correct in 
environments where the IOMMU is used instead, then the P-teams and the 
third parties could then either recompile or otherwise turn on that 
usage. That is prudent.

So if you're going to make such a big underlying change, I would suggest 
instead that until it's a major release, you could add a FORCE_IOMMU 
flag. This would have the effect of taking driver binaries to work 
exactly as they have done before and allow drivers written by both Sun 
and third parties to /try out/ the IOMMU environment as needed or desired.

The above isn't probably reasoned out as well as it should be, but it 
does seem to me that you should either dump the DDI and it's 
overcautious "let's not change things (for decades)" approach (and take 
an approach more like (the good parts) of Linux), or if you're going to 
treat the DDI in its multiple roles, err very much on the side of caution.

-matt