[intel-platform-dev] IO Virtual Address management questions.

[Raj, Ashok]

Hi Allen

I think mark has this request from us from a while back and is working
on an interface not necessarily the same steps as you indicate. 

- allocate 4k aligned page that is locked
- get a virtual and physical addr as output.

We would also need to easily translate between virtual/physical and vice
versa. Since when we walk the pagetables we would need a way to convert
the physical addr to virtual instead of keeping a separate lookup table.


Cheers,
ashok raj
- Open Source Technology Center
   


>-----Original Message-----
>From: Baolu Lu [mailto:lubaolu at gmail.com]
>Sent: Saturday, December 01, 2007 10:37 PM
>To: Raj, Ashok
>Cc: Mark.Johnson at sun.com; Lu, Baolu; intel-platform-dev at opensolaris.org
>Subject: Re: [intel-platform-dev] IO Virtual Address management
questions.
>
>Hi Mark,
>
>In addition to these questions, can I add another one?
>
>We need function interfaces to allocate/free the iommu pages. The page
>size is 4k. Can the following steps work? If so, what are the
>functions in each step?
>
>1. allocate a physical page
>2. lock this page
>3. map this page to virtual address
>
>Can you give me some reference code in the kernel source tree?
>
>Thanks
>
>On Nov 14, 2007 8:44 AM, Raj, Ashok <ashok.raj at intel.com> wrote:
>> Hi Mark
>>
>> Here goes the second question about iova management. Basically iam
>> looking for 2 things
>>
>> 1. A way to carve out ranges that cannot be allocated.
>> 2. A way to clone the iova areana for each new domain I allocate.
>>
>> Seems like I can use the same for other domain allocation code, looks
>> like I can use it generically even as a bitvec allocator. Say I have
256
>> domains, I could just use this with granularity of 1, just like how
the
>> dtrace code is using it?
>>
>> More details below.
>>
>> IO Virtual Address space management.
>> ------------------------------------
>> 1. We need a method to allocate io virtual address (IOVA). Solaris
>> already has
>>    support for this via vmem_create(), vmem_alloc() and vmem_free().
>>
>>    We need to create holes in the address space for the following
items.
>>
>>    - Remove APIC address range 0xfee00000 - 0xfeefffff
>>    - Remove certain 1-1 ranges, such as RMRR's from an address map
>>    - Remove any mem ranges that exist under p2p side to avoid peer to
>> peer
>>    pci side effects. It would be simple to just remove them from the
>> address
>>    range.
>>
>>    What is the best way to punch holes in the vmem arena. I see two
ways
>>    but not sure if there is a better way.
>>
>>    If there is a new api like vmem_del(vmem_t *, void *addr, size_t
>> size)
>>
>>    And is there a way to clone the areana created? we would need to
do
>> this
>>    each time we create a iova for each domain to manage. It might be
>> easy if
>>    i had an iova areana with holes, and just clone them for each new
>> domain
>>    i create.
>>
>>    it would be great if vmem could do the splicing. Without something
>> like this
>>
>>    - We could add and create the areana leaving holes with multiple
>> vmem_add()
>>    calls.
>>
>>
>>    - Just create a large address space, and use vmem_xalloc() to
>> allocate and
>>    remove these holes. Performance is not a problem since this is
>> startup time
>>    not speed path.
>>
>>    say
>>
>>         since we will only map at page granurality its enough if we
>>         account just page numbers.
>>
>>         iova_area = vmem_create(name, 1, BTOP(UINT64_MAX), 1, NULL,
>> NULL, NULL,
>>                     0, VM_SLEEP | VMC_IDENTIFIER);
>>
>>         but to cut out the un-allocatable holes say for e.g
>>
>>         vmem_xalloc(iova_area, 0xff, 1, 0, 0xfee00, 0xfeeff,
VM_SLEEP);
>>
>> Cheers,
>> ashok raj
>> - Open Source Technology Center
>>
>> _______________________________________________
>> intel-platform-dev mailing list
>> intel-platform-dev at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/intel-platform-dev
>>