SCSI_HBA_ADDR_COMPLEX SCSA data structure linkage [PSARC/2008/675 FastTrack timeout 11/12/2008]
Garrett D'Amore
gdamore at sun.com
Sun Nov 2 21:30:05 PST 2008
markcarl wrote:
+1 from me as well.
-- Garrett
> +1
>
> -- mark
>
> On Nov 2, 2008, at 7:24 PM, Christopher Horne <cth at sac.sfbay.sun.com>
> wrote:
>
>> I am sponsoring the following fasttrack for myself. Micro/patch
>> binding is requested. The timer is set to expire on Nov. 12 2008.
>>
>> -Chris
>>
>> 1. Introduction
>> 1.1. Project/Component Working Name:
>> SCSI_HBA_ADDR_COMPLEX SCSA data structure linkage
>> 1.2. Name of Document Author/Supplier:
>> Author: Chris Horne
>> 1.3. Date of This Document:
>> Thu Oct 23 2008
>>
>> 4. Technical Description
>>
>> 4.1 Background:
>>
>> Initial Sun Common SCSI Architecture (SCSA) data structures were
>> designed to support copper scsi addressing, where both the target
>> and lun numbers were small integers. In this model, an HBA driver
>> was expected to maintain per-scsi_device(9S) HBA private data inside
>> an array embedded in the HBA's initiator port soft state. This
>> array, indexed by the scsi_address(9S) a_target and a_lun(9S)
>> fields, was dimensioned by the maximum target and lun numbers
>> supported by the HBA driver.
>>
>> The limitation of this approach was exposed by transports such as
>> fiber-channel, and overcoming the limitation was complicated by the
>> fact that a scsi_address(9S) is embedded at the base of two
>> well-know SCSA data structures: both the scsi_device(9S) structure
>> and the scsi_pkt(9S) structure.
>>
>> To solve this problem, interfaces were needed to allow an HBA driver
>> to maintain and access per-scsi_device(9S) HBA private data, given
>> only the scsi_address(9S) information of an allocated scsi_pkt(9S)
>> (as established by a 'pkt->pkt_address = *ap;' structure assignment
>> off the scsi_init_pkt(9F)/tran_init_pkt(9E) implementation).
>>
>> Given the SCSA environment and development constraints at the time,
>> 'cloning' the scsi_hba_tran(9S) structure was an effective data
>> structure implementation approach (an expedient kludge that
>> minimized breakage, but wasted space).
>>
>> Unfortunately, the 'clone' data structure implementation details
>> ended up being leaked as interfaces:
>>
>> o scsi_hba_attach(9E) SCSI_HBA_TRAN_CLONE flag.
>>
>> o scsi_hba_tran(9S) 'tran_sd' field, pointing to
>> scsi_device(9S).
>>
>> o scsi_hba_tran(9S) 'tran_tgt_private' field, used to implement
>> per-scsi_device(9S) HBA private data (inside the 'cloned'
>> scsi_hba_tran(9S) structure).
>>
>> The first ARC references to SCSI_HBA_TRAN_CLONE is in [1], from
>> 1993. The oldest CR is 1156862 "scsi_hba: if cloning, passes wrong
>> transport structure to hba in tran_tgt_free", from the same
>> timeframe. There is no ARC record introducing SCSI_HBA_TRAN_CLONE
>> as an interface, public or private: records go back to 1991. It
>> seems to have just leaked into the DDI based on need.
>>
>> 4.2 Proposal:
>>
>> This case proposes solving the same basic problem with interfaces
>> that minimize exposure of implementation artifacts. The proposed
>> interfaces are consolidation private, but future promotion is
>> expected (with future deprecation of the "clone" 'interfaces').
>>
>> The new proposed interfaces are:
>>
>> o SCSI_HBA_ADDR_COMPLEX:
>>
>> A scsi_hba_attach_setup(9E) caller can indicate that an
>> initiator port requires a 'complex' target/lun address space
>> representation the new SCSI_HBA_ADDR_COMPLEX flag. The
>> properties associated with a scsi_device(9S) target/lun
>> unit-address are available to the HBA at tran_tgt_init(9E)
>> time. The HBA driver can translate unit-address property values
>> into an appropriate private form, and store that information in
>> its per-scsi_device(9S) HBA private data.
>>
>> o scsi_device_hba_private_set(9F),
>> scsi_device_hba_private_set(9F):
>>
>> A SCSI_HBA_ADDR_COMPLEX HBA can maintain/obtain
>> per-scsi_device(9S) HBA private data using
>> scsi_device_hba_private_set(9F) and
>> scsi_device_hba_private_get(9F).
>>
>> o scsi_address_device(9F):
>>
>> A SCSI_HBA_ADDR_COMPLEX HBA can call scsi_address_device(9F) to
>> obtain a pointer to the scsi_device(9S) given a pointer to a
>> scsi_address(9S).
>>
>> A SCSI_HBA_ADDR_COMPLEX HBA is expected to establish
>> per-scsi_device(9S) HBA private data in its tran_tgt_init(9E)
>> implementation, and free that HBA private data in its
>> tran_tgt_fini(9E) implementation.
>>
>> In its tran_start(9E) implementation, SCSI_HBA_ADDR_COMPLEX HBA can
>> obtain its scsi_device(9S) pointer from the scsi_address(9S) by
>> calling scsi_address_device(9F). Once the HBA driver obtains the
>> scsi_device(9S) pointer, the per-scsi_device(9S) HBA private data
>> can be obtained using scsi_device_hba_private_get(9F).
>>
>> At this point in time, implementing the proposed interfaces no
>> longer requires 'cloning': the CRs that required 'cloning' [3] have
>> now been resolved. The new implementation will be placing the SCSI
>> parallel interconnect (SPI) scsi_address(9S) fields (a_target,
>> a_lun) in a union, and in the SCSI_HBA_ADDR_COMPLEX case the union
>> will be a pointer to the scsi_device(9S) structure, with
>> per-scsi_device(9S) HBA private data maintained in the
>> scsi_device(9S) itself (sd_hba_private), instead of in a cloned
>> scsi_hba_tran(9S) structure. Pictures (.gif) of the old and new
>> data structure linkage are provided in the materials directory.
>>
>> While it would be possible to have the new (9F) interfaces function
>> correctly for the SCSI_HBA_TRAN_CLONE case too, that is not
>> planned.
>>
>> Having a single scsi_hba_tran(9S) structure per SCSI initiator_port
>> will be important for futute SCSA work, where the SCSA framework
>> will need to be enhanced to maintain new per-target_port
>> information.
>>
>>
>> 4.3 Proposed Interfaces
>>
>> ------------------------------------------------------------------
>> Interface Name Comm.Level Comments
>> ------------------------------------------------------------------
>>
>> SCSI_HBA_ADDR_COMPLEX Private scsi_hba_attach_setup(9F)
>> flag indicating HBA
>> transport has 'complex'
>> address space.
>>
>> SCSI_HBA_ADDR_SPI Private default: HBA expects
>> scsi_address(9S) in
>> SCSI Parallel Interconnect
>> form (a_target, a_lun).
>>
>> SCSI_HBA_TRAN_CLONE no-change use discouraged.
>>
>> scsi_address_device(9F) Private Given a pointer to a
>> scsi_address(9S),
>> return pointer to
>> scsi_device(9S).
>>
>>
>> scsi_device_hba_private_get(9F) Private per-scsi_device(9S)
>> scsi_device_hba_private_set(9F) HBA private data
>> interfaces.
>>
>>
>> Prototypes for new (9F) functions: common/sys/scsi/conf/device.h:
>>
>> struct scsi_device *scsi_address_device(struct scsi_address *sa);
>> void scsi_device_hba_private_set(struct scsi_device *sd, void
>> *data);
>> void *scsi_device_hba_private_get(struct scsi_device *sd);
>>
>> 4.4 Man Pages
>>
>> See Appendix A
>>
>> 4.4 Release Binding
>>
>> Micro/patch binding is requested.
>>
>> 4.5 References
>>
>>
>> [1] PSARC/1993/259 SCSI HBA interface [Jerry Gilliam]
>> http://sac.sfbay/PSARC/1993/259
>> http://www.opensolaris.org/os/community/arc/caselog/PSARC/1993/259
>>
>> [2] PSARC/1996/113 SCSA SCSI-3 enhancements [Sudershan Goyal]
>> http://sac.sfbay/PSARC/1996/113
>> http://www.opensolaris.org/os/community/arc/caselog/PSARC/1996/113
>>
>> [3] CRs associated with scsi_address/scsi_device/scsi_pkt cleanup
>> 6269673 scsi target drivers should not reference scsi_address(4s)...
>> 6657256 SCSA should detect scsi_pkt allocation violations
>> 6695222 ata has dependency on scsi_device(9S) size
>> 6755515 preparation for scsi_address being unionized
>>
>> http://monaco.sfbay.sun.com/detail.jsf?cr=6269673,6657256,6695222,6755515
>>
>>
>>
>>
>> Appendix A:
>>
>> A.1: Changes to scsi_hba_attach_setup(9F)
>> =========================================
>> :r!diff -U5 -w scsi_hba_attach_setup.9f.orig scsi_hba_attach_setup.9f
>>
>> --- scsi_hba_attach_setup.9f.orig Tue Oct 21 19:12:37 2008
>> +++ scsi_hba_attach_setup.9f Tue Oct 28 08:57:27 2008
>> @@ -25,12 +25,14 @@
>>
>> hba_lim A pointer to a ddi_dma_lim(9S) structure.
>>
>> hba_tran A pointer to a scsi_hba_tran(9S) structure.
>>
>> - hba_flags Flag modifiers. The only defined flag value
>> - is SCSI_HBA_TRAN_CLONE.
>> + hba_flags Flag modifiers. The only defined flag values are
>> + SCSI_HBA_ADDR_SPI, SCSI_HBA_ADDR_COMPLEX, and
>> + SCSI_HBA_TRAN_CLONE. SCSI_HBA_TRAN_CLONE is
>> + obsolete, use SCSI_HBA_ADDR_COMPLEX instead.
>>
>> hba_options Optional features provided by the HBA driver
>> for future extensions; must be NULL.
>>
>> hba_dma_attr A pointer to a ddi_dma_attr(9S) structure.
>> @@ -55,10 +57,24 @@
>> scsi_hba_attach() and scsi_hba_attach_setup() use the
>> dev_bus_ops field in the dev_ops(9S) structure. The HBA
>> driver should initialize this field to NULL before calling
>> scsi_hba_attach() or scsi_hba_attach_setup().
>>
>> + A scsi_address(9S) unit-address form for a scsi_device(9S) is
>> + chosen based on whether SCSI_HBA_ADDR_SPI or SCSI_HBA_ADDR_COMPLEX
>> + is requested in hba_flags. Only one flag can be specified; if
>> + neither flag is specified SCSI_HBA_ADDR_SPI is assumed. If
>> + SCSI_HBA_ADDR_SPI is selected then the a_target and a_lun fields
>> + of a scsi_address structure are established by the SCSA library
>> + code prior to tran_tgt_init(9E). If SCSI_HBA_ADDR_COMPLEX is
>> + selected then the host adapter driver is expected to establish an
>> + opaque private representation of device unit-address, based
>> + on property values, during tran_tgt_init(9E). See
>> scsi_address(9S)
>> + for more detail.
>> +
>> + The SCSI_HBA_TRAN_CLONE flag is also supported, but use
>> + of SCSI_HBA_ADDR_COMPLEX is prefered.
>> If SCSI_HBA_TRAN_CLONE is requested in hba_flags, the
>> hba_tran structure will be cloned once for each target
>> attached to the HBA. The cloning of the structure will occur
>> before the tran_tgt_init(9E) entry point is called to ini-
>> tialize a target. At all subsequent HBA entry points,
>> @@ -189,5 +205,9 @@
>> get device driver after scsi_hba_detach() is called.
>>
>> The scsi_hba_attach() function is obsolete and will be dis-
>> continued in a future release. This function is replaced by
>> scsi_hba_attach_setup().
>> +
>> + The use of SCSI_HBA_TRAN_CLONE is obsolete and will be
>> + discontinued in a future release, use SCSI_HBA_ADDR_COMPLEX
>> + instead.
>>
>>
>> =========================================
>> A.2: Changes to scsi_address(9S)
>> :r!diff -U5 -w scsi_address.9s.orig scsi_address.9s
>>
>> --- scsi_address.9s.orig Tue Oct 21 19:12:37 2008
>> +++ scsi_address.9s Mon Oct 27 20:34:36 2008
>> @@ -8,45 +8,107 @@
>>
>> INTERFACE LEVEL
>> Solaris architecture specific (Solaris DDI)
>>
>> DESCRIPTION
>> - A scsi_address structure defines the addressing components
>> - for a SCSI target device. The address of the target device
>> - is separated into two components: target number and logical
>> - unit number. The two addressing components are used to
>> - uniquely identify any type of SCSI device; however, most
>> - devices can be addressed with the target component of the
>> - address.
>> -
>> - In the case where only the target component is used to
>> - address the device, the logical unit should be set to 0. If
>> - the SCSI target device supports logical units, then the HBA
>> - must interpret the logical units field of the data struc-
>> - ture.
>> -
>> - The pkt_address member of a scsi_pkt(9S) is initialized by
>> - scsi_init_pkt(9F).
>> + A scsi_address structure stores the host routing and device
>> + unit-address information necessary to reference a specific SCSI
>> + target device logical unit function.
>> +
>> + Host routing information is used by the system to identify the
>> + HBA and initiator port.
>> +
>> + Device unit-address information is SCSA's representation of the
>> + "@unit-address" portion of a SCSI target driver scsi_device node
>> + in the the /devices tree. Separate portions of the device
>> + unit-address information define the target port address, the
>> + logical unit address on the target, and (optionally) a
>> + subfunction implemented by a logical unit. Device scsi_device
>> + unit-address information is used exclusively by the host adapter
>> + driver (with the legacy exception of target drivers communicating
>> + with SCSI Parallel Interconnect (SPI) SCSI-1 devices that embed
>> + SCSI logical unit addressing in the CDB).
>> +
>> + Two forms of scsi_device unit-address representation are
>> + supported within the scsi_address structure: a host adapter
>> + driver chooses the form by its hba_flags argument to
>> + scsi_hba_attach_setup(9F): SCSI_HBA_ADDR_SPI or
>> + SCSI_HBA_ADDR_COMPLEX. For SCSI interconnects that have a complex
>> + scsi_device unit-address space, like fibre-channel,
>> + SCSI_HBA_ADDR_COMPLEX should be selected. To support legacy host
>> + adapter drivers that don't explicitly choose a form,
>> + SCSI_HBA_ADDR_SPI is assumed.
>> +
>> + For host adapter drivers that choose SCSI_HBA_ADDR_SPI, two
>> + scsi_device unit-address portions, target number (a_target) and
>> + logical unit number (a_lun), are directly represented within the
>> + scsi_address structure.
>> +
>> + For host adapter drivers that choose SCSI_HBA_ADDR_COMPLEX, the
>> + tran_tgt_init(9E) implementation is expected to store a
>> + per-scsi_device(9S) HBA private data pointer, based on its
>> + interpretation of the scsi(9P) device unit-address properties,
>> + using scsi_device_hba_private_set(9F). This approach allows SCSI
>> + interconnects to have their own private scsi_device unit-address
>> + representation, which is particularly important for the target
>> + portion of the unit-address, which is transport-specific. During
>> + tran_start(9E), an HBA driver can locate the scsi_device(9S)
>> + structure using scsi_address_device(9F), and obtain the
>> + per-scsi_device(9S) HBA private data pointer using
>> + scsi_device_hba_private_get(9F). The HBA private data should be
>> + freed by tran_tgt_free(9E).
>> +
>> + The pkt_address member of a scsi_pkt(9S) is initialized after
>> + scsi_init_pkt(9F) as a result of the host adapter driver
>> + performing a structure assignment in its tran_init_pkt(9E)
>> + implementation.
>>
>> STRUCTURE MEMBERS
>> - scsi_hba_tran_t *a_hba_tran; /* Transport vectors for the
>> SCSI bus */
>> - ushort_t a_target; /* SCSI target id */
>> - uchar_t a_lun; /* SCSI logical unit */
>> -
>> - a_hba_tran is a pointer to the controlling HBA's transport
>> - vector structure. The SCSA interface uses this field to pass
>> - any transport requests from the SCSI target device drivers
>> - to the HBA driver.
>> -
>> - a_target is the target component of the SCSI address.
>> -
>> - a_lun is the logical unit component of the SCSI address. The
>> - logical unit is used to further distinguish a SCSI target
>> - device that supports multiple logical units from one that
>> - does not. The makecom(9F) family of functions use the a_lun
>> - field to set the logical unit field in the SCSI CDB, for
>> - compatibility with SCSI-1.
>> +
>> + scsi_hba_tran_t *a_hba_tran; /* Transport vector */
>> + ushort_t a_target; /* SPI: SCSI target id */
>> + uchar_t a_lun; /* SPI: SCSI logical unit */
>> + scsi_device *a_sd; /* COMPLEX: pointer to
>> scsi_device */
>> +
>> + a_hba_tran Pointer to the controlling host adapter driver
>> + transport vector structure. The SCSA interface
>> + uses this field to pass any transport requests
>> + from the SCSI target device drivers to the host
>> + adapter driver.
>> +
>> + For SCSI_HBA_ADDR_COMPLEX:
>> +
>> + a_sd A pointer to the scsi_device(9S) structure.
>> + This field is set by SCSA library code. An HBA
>> + driver should not access this field directly -
>> + use scsi_address_device(9F) instead.
>> +
>> + For SCSI_HBA_ADDR_SPI:
>> +
>> + a_target target component of the device unit-address,
>> + limited to the range 0-65535).
>> +
>> + a_lun logical unit co
More information about the opensolaris-arc
mailing list