[intel-platform-dev] IO Virtual Address management questions.

[Raj, Ashok]

Hi Mark

Here goes the second question about iova management. Basically iam
looking for 2 things

1. A way to carve out ranges that cannot be allocated. 
2. A way to clone the iova areana for each new domain I allocate.

Seems like I can use the same for other domain allocation code, looks
like I can use it generically even as a bitvec allocator. Say I have 256
domains, I could just use this with granularity of 1, just like how the
dtrace code is using it? 

More details below.

IO Virtual Address space management.
------------------------------------
1. We need a method to allocate io virtual address (IOVA). Solaris
already has
   support for this via vmem_create(), vmem_alloc() and vmem_free().

   We need to create holes in the address space for the following items.

   - Remove APIC address range 0xfee00000 - 0xfeefffff
   - Remove certain 1-1 ranges, such as RMRR's from an address map
   - Remove any mem ranges that exist under p2p side to avoid peer to
peer
   pci side effects. It would be simple to just remove them from the
address 
   range.
   
   What is the best way to punch holes in the vmem arena. I see two ways
   but not sure if there is a better way.

   If there is a new api like vmem_del(vmem_t *, void *addr, size_t
size)

   And is there a way to clone the areana created? we would need to do
this 
   each time we create a iova for each domain to manage. It might be
easy if
   i had an iova areana with holes, and just clone them for each new
domain
   i create.

   it would be great if vmem could do the splicing. Without something
like this

   - We could add and create the areana leaving holes with multiple
vmem_add()
   calls.


   - Just create a large address space, and use vmem_xalloc() to
allocate and
   remove these holes. Performance is not a problem since this is
startup time
   not speed path.

   say

	since we will only map at page granurality its enough if we 
	account just page numbers.

	iova_area = vmem_create(name, 1, BTOP(UINT64_MAX), 1, NULL,
NULL, NULL,
		    0, VM_SLEEP | VMC_IDENTIFIER);
   
   	but to cut out the un-allocatable holes say for e.g 

	vmem_xalloc(iova_area, 0xff, 1, 0, 0xfee00, 0xfeeff, VM_SLEEP);

Cheers,
ashok raj
- Open Source Technology Center