Interrupt affinity interfaces and PCITool enhancements [PSARC/2009/340 FastTrack timeout 06/17/2009]

[Kais Belgaied]

On 06/15/09 20:38, Evan Yan wrote:
> Hi Kais,
>
> Thanks for the comments.
>
> Pcitool and the interrupt affinity interfaces use the same under-layer
> implementation to re-target interrupts to some cpu. Whatever read
> operation will reflect the current binding status and whatever write
> operation will override the former settings.
>   

so, back the example,  let's say you you use pcitool to bind the 
interrupts from a physical NIC nxge0 to
cpu1, the usecrossbow's dladm  to  set-linprop cpus=2 vnic1 and cpus=3 
vnic2 (where vnic1 and vnic2
are built over nxge0) will pcitool show that nxge0's interrupts are 
bound to cpus 1, 2 and 3?


    Kais

> Thanks,
> -Evan
>
> Kais Belgaied wrote:
>   
>>>     This case also includes the contract for Crossbow framework to use these
>>>     interrupt affinity interfaces in place of existing PCITool ioctl interfaces.
>>>   
>>>     
>>>       
>> If I look at the this case in isolation from its expected consumers, and 
>> with pcitool as the
>> only consumer of the CPU affinity APIs, I have no trouble sending a +1.
>> However, when considering the overall architecture  that includes both 
>> this case's deliverables as well as the changes
>> expected imminently  from its external consumers,  I am unclear on how 
>> the system will behave when
>> we  use both pcitool and those consumers' interrupt settings.
>>
>> I'll use the interaction with Crossbow as an example. The point is 
>> similar for the interaction with other tools
>> (intd(1m), etc). Say the system has a physical NIC nxge0, whose 
>> interrupts are bound the cpu's 1,2,3,4
>> using pcitool modified by this case. Later one creates vnic1 over nxge0 
>> which
>> used a couple of hardware rings out of nxge0's, and uses dladm 
>> set-linkprop cpus=5,6 vnic1.
>> With the changes imported from this case, the implementation of that 
>> call of dladm will attempt
>> to have the MSI/X interrupts  assigned to  the rings (thus the vnic1)  
>> bound to CPUs 5 and 6.
>> Will such setting of vnic1's  interrupts, fail because of a conflict 
>> with a previous binding by pcitool?
>> will it succeed silently? what will the call to pcitool querying about 
>> the interrupt binding for
>> nxge0 return then? CPUs 1,2,3,4 only, as set ? or will it surprisingly 
>> show 1,2,3,4,5,6?
>> how about the other way around? will dladm  get-linkprop vnic1 see the 
>> settings that were previously done
>> by pcitool?
>>
>>
>>     Kais.
>>   
>>     
>
>