[driver-discuss] ddi_dma_attr_t, dma_attr_maxxfer quesiton.
Garrett D'Amore
garrett at damore.org
Tue Dec 4 23:09:37 PST 2007
Carlos Cumming wrote:
> Mark Johnson wrote:
>
>> You should support partial mappings. You can see 1M transfers come
>> down the scsi stack. You would
>> want that broken into multiple 128K DMA "windows".
>>
> My understanding was that the call to scsi_hba_attach_setup(dev,
> &hba_dma_attr, softs->tran, 0) would limit requests I see to the
> attributes defined by hba_dma_attr. It might even be more efficient for
> my caller to be handling it than the hba driver.
>
Yes. There is nice and efficient support for DMA partial mappings in
the SCSA framework. To do this you might want to look at tran_setup_pkt
and tran_destroy_pkt. Then the SCSA framework will give you DMA
chunks that are already broken up (actually using partial mappings) by
stashing DMA cookies in the SCSI pkt as pkt->pkt_cookies and
pkt->pkt_ncookie or somesuch.
Have a look at my blk2scsa layer generic layer to see an example.
http://cr.opensolaris.org/~gdamore/blk2scsa/
(Note that blk2scsa is a bit more complicated because it also supports
"generic" layers that might not even support DMA. But it assumes DMA by
default and is optimized for the DMA case.)
-- Garrett
> Thanks, Carlos
>
>> e.g. I believe /dev/rdsk/c* is case where this
>> will happen.
>>
>>
>>
>>>> e.g. if a dma engine has a 16 bit size for the DMA engine, the max
>>>> size it can DMA is 64K - 1 byte. (0xffff). A few DMA
>>>> engines will use 0 to signify the 0x10000 value, but that means
>>>> adding another bit to the counter or adding some more
>>>> complicated logic.
>>>>
>>> [the above] is how I understood it.... But if you want to get optimal
>>> performance out of your hw, then you'd want to do a full 64k DMA
>>> (somehow). Xfering 65535 (64k - 1) bytes is a really oddball value.
>>> Specially if you have a block (512 byte) device like I have.
>>>
>>> In my case I'm actually, "logically" limited. I.e., the firmware in
>>> the device is limiting me to 128k, the actual counter goes to 2^32 -
>>> 1 (to use your parlance). So I take it in my case using a full value
>>> of 128k (i.e., 0x20000) in dma_attr_maxxfer is correct.
>>>
>> yes.
>>
>>
>>
>> MRJ
>>
>>
>>
>>
>>
>>> Thanks!!!
>>>
>>> Carlos
>>>
>>>> Carlos Cumming wrote:
>>>>
>>>>> I'm writing a SCSI driver. The board I'm talking to can do a max
>>>>> 128k (256 blocks) DMA. I am using the following two dma_attr_t's:
>>>>>
>>>>> static ddi_dma_attr_t hba_dma_attr = {
>>>>> DMA_ATTR_V0, // dma_attr_version
>>>>> Version of this structure
>>>>> (u64)0, // dma_attr_addr_lo
>>>>> Lowest usable address
>>>>> (u64)0xffffffffffffffff, // dma_attr_addr_hi
>>>>> Highest usable address
>>>>> (u64)(256 * 512) - 1, // dma_attr_count_max
>>>>> Maximum DMAable byte count (1 sge * maxxfer)
>>>>> 512, // dma_attr_align
>>>>> Minimum alignment (This value correct?)
>>>>> 0x7f, // dma_attr_burstsizes
>>>>> Burst sizes (??? big values don't work)
>>>>> (u64)4, // dma_attr_minxfer
>>>>> Minimum transfer
>>>>> (u64)(256 * 512) - 1, // dma_attr_maxxfer
>>>>> Maximum transfer count for entire I/O.
>>>>> (u64)0xffffffffffffffff, // dma_attr_seg
>>>>> Maximum segment (boundry DMA engine cannot cross).
>>>>> 512, // dma_attr_sgllen
>>>>> Maximum scatter gather list entries
>>>>> 512, // dma_attr_granular
>>>>> Granularity
>>>>> 0, // dma_attr_flags
>>>>> Flags (reserved)
>>>>> };
>>>>>
>>>>> static ddi_dma_attr_t cmd_dma_attr = {
>>>>> DMA_ATTR_V0, // dma_attr_version
>>>>> Version of this structure
>>>>> (u64)0, // dma_attr_addr_lo
>>>>> Lowest usable address
>>>>> (u64)0xffffffffffffffff, // dma_attr_addr_hi
>>>>> Highest usable address
>>>>> (u64)(256 * 512) - 1, // dma_attr_count_max
>>>>> Maximum DMAable byte count (1 sge * maxxfer)
>>>>> 512, // dma_attr_align
>>>>> Minimum alignment (This value correct?)
>>>>> 0x7f, // dma_attr_burstsizes
>>>>> Burst sizes (??? big values don't work)
>>>>> (u64)512, // dma_attr_minxfer
>>>>> Minimum transfer
>>>>> (u64)(256 * 512) - 1, // dma_attr_maxxfer
>>>>> Maximum transfer count for entire I/O.
>>>>> (u64)0xffffffffffffffff, // dma_attr_seg
>>>>> Maximum segment (boundry DMA engine cannot cross).
>>>>> 512, // dma_attr_sgllen
>>>>> Maximum scatter gather list entries
>>>>> 512, // dma_attr_granular
>>>>> Granularity
>>>>> 0, // dma_attr_flags
>>>>> Flags (reserved)
>>>>> };
>>>>>
>>>>>
>>>>>
>>>>> Some time later, I use the first one:
>>>>>
>>>>> if (scsi_hba_attach_setup(dev, &hba_dma_attr, softs->tran, 0) !=
>>>>> DDI_SUCCESS) ..
>>>>>
>>>>>
>>>>>
>>>>> Then even later (but still within attach if that matters) I use
>>>>> cmd_dma_attr...
>>>>>
>>>>> if (ddi_dma_alloc_handle(softs->dev, &cmd_dma_attr,
>>>>> DDI_DMA_SLEEP, NULL, &softs->dma_hndl) != DDI_SUCCESS) {
>>>>> cmn_err(CE_WARN, "tw%d: tw_create_reqs: Cannot alloc dma
>>>>> handle for command blocks.", inst);
>>>>> return(1);
>>>>> }
>>>>>
>>>>> #define MEMSIZE (256 * 512)
>>>>>
>>>>> if (ddi_dma_mem_alloc(softs->dma_hndl, MEMSIZE, &accattr,
>>>>> MEMTYPE, DDI_DMA_SLEEP, NULL, &mem, &rlen, &softs->acc_hndl)
>>>>> != DDI_SUCCESS) {
>>>>> cmn_err(CE_WARN, "tw%d: tw_create_reqs: Cannot alloc dma
>>>>> memory for command blocks.", inst);
>>>>> goto free_hndl;
>>>>> }
>>>>>
>>>>> {
>>>>> int a;
>>>>>
>>>>> if ((a = ddi_dma_addr_bind_handle(softs->dma_hndl, NULL, mem,
>>>>> rlen, MEMTYPE, DDI_DMA_SLEEP, NULL, &dmacookie, &num_dmacookies))
>>>>> != DDI_DMA_MAPPED) {
>>>>> cmn_err(CE_WARN, "tw%d: tw_create_reqs: Cannot bind dma
>>>>> memory to handle for command blocks. %d", inst, a);
>>>>> goto free_mem;
>>>>> }
>>>>> }
>>>>>
>>>>>
>>>>> The above ddi_dma_addr_bind_handle(.. ) _FAILS. It prints "Cannot
>>>>> bind dma memory to handle for command blocks. 1". The trailing ,"1"
>>>>> is DDI_ENOMEM if I understand it correctly.
>>>>>
>>>> What is MEMTYPE set to. It looks like you have set
>>>> DDI_DMA_PARTIAL which is correct.
>>>>
>>>> You are not getting a failure returned. You are getting
>>>> DDI_DMA_PARTIAL_MAP which means you have multiple DMA
>>>> windows.
>>>>
>>>> #define DDI_DMA_PARTIAL_MAP 1
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Now, here's where I get lost. If you look at
>>>>> cmd_dma_attr.dma_attr_maxxfer above, it's set to (256 * 512) - 1.
>>>>> I'm trying to get 256 * 512 of memory. So, it makes sense that it
>>>>> fails ENOMEM, because I'm requesting one more byte than
>>>>> dma_attr_maxxfer.
>>>>>
>>>>> Or does it make sense...
>>>>>
>>>>> _Every_ example that I look at shows dma_attr_maxxfer set to a
>>>>> power of 2 - 1. I.e., the way everyone else uses dma_attr_maxxfer
>>>>> is as if it's the max value of a dma counter, just like
>>>>> dma_attr_count_max.
>>>>>
>>>>> So I'm feeling just a bit squeamish using a nice even power of two
>>>>> value for dma_attr_maxxfer when nobody else is using it that way. I
>>>>> figure they know something I don't.
>>>>>
>>>>> Can someone give me a clue???
>>>>>
>>>> The reason folks use power of 2's - 1 for things like maxxfer, seg,
>>>> dma count is because counters are usually powers of two :-)
>>>>
>>>> e.g. if a dma engine has a 16 bit size for the DMA engine,
>>>> the max size it can DMA is 64K - 1 byte. (0xffff). A few DMA
>>>> engines will use 0 to signify the 0x10000 value, but that
>>>> means adding another bit to the counter or adding some more
>>>> complicated logic.
>>>>
>>>>
>>>> MRJ
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> driver-discuss mailing list
>>> driver-discuss at opensolaris.org
>>> http://mail.opensolaris.org/mailman/listinfo/driver-discuss
>>>
>
> _______________________________________________
> driver-discuss mailing list
> driver-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/driver-discuss
>
>
More information about the driver-discuss
mailing list