X11 MIT-SHM Extension across zones [PSARC 2007/500, fast-track timeout 9/13/07]

[Alan Coopersmith]

I am sponsoring this fasttrack for myself, and have set the timeout to
Thursday, Sept. 13.   It requests a patch release binding.

This project enhances the X server to allow the MIT-SHM extension to operate
between clients in non-global zones and the X server in the global zone.

The MIT-SHM extension uses SysV shared memory to transport pixmaps between X
clients and X servers without having to write them to the socket or pipe and
read them back out.   The current Solaris implementation contains two
impediments blocking use of this extension between different zones on the same
system.

1) access control check in X servers

   Since the X client tells the X server what shared memory segment it
   has created for the pixmap and the X server attachs to it with its
   privilege level/uid (which is usually higher than the client's and
   often at least different from the client), the X server first
   verifies the client isn't asking it to connect to a segment the client
   shouldn't have access to.  It uses getpeerucred() to get the client
   credentials, and shmctl() to get the ownership and permissions of
   the shared memory segment and compares to make sure the credentials
   returned by shmctl should be able to perform the requested access
   (depending on the operation, either read-only or read-write), If
   getpeerucred() fails, or if the zoneid returned by getpeerucred
   does not match the zone returned by getzoneid(), access will only
   be allowed if the shared memory segment may be read (and if needed,
   written) by "other".

   This project modifies the check so that instead of requiring the client
   zoneid to match the X server's, it uses the shmctl64() function to get the
   zoneid that created the shared memory segment and compares the client's
   zone to that zone.   If that matches, then the clients uid & gid may be
   used to verify access, otherwise the client will still be restricted to
   shared memory segments using the "other" access permissions.

   Since shmctl64() is a Project Private function in libc, this project
   includes a contract between the X team and the Zones team allowing
   cross-consolidation use of this interface.

2) XShmQueryExtension() in libXext

   To avoid telling clients they can use MIT-SHM when the connection to the
   Xserver is remote, XShmQueryExtension() in libXext returns False if it
   believes the X server is on another machine.   The current algorithm
   used is:
        if stat(display_connection) returns a mode of S_IFIFO
           connection is local
        else
           getpeername(display_connection)
           if socket_family is AF_UNIX
              connection is local
           else
              for each local interface returned by SIOCGLIFCONF ioctl
                  if address == peer socket address
                     connection is local
   and if none of those find the connection is local, the function reports
   that MIT-SHM is not connected.

   This project greatly simplifies the above algorithm to a single check:

        if getpeerucred(display_connection) == 0 && ucred_getzoneid() != -1
           connection is local
        else
           connection is remote


Interfaces Imported:
====================

Interface:              Stability:                      Definition/Comments:
------------------------------------------------------------------------------
shmctl64()              Contracted Project Private      [1], [2]
getpeerucred()          Evolving                        PSARC 2003/197,
                                                          PSARC 2004/336

Interfaces Exported:
====================

no new interfaces exported

References:
[1] shmctl64.txt in case materials directory
[2] contract-01.txt in case materials directory

-- 
	-Alan Coopersmith-           alan.coopersmith at sun.com
	 Sun Microsystems, Inc. - X Window System Engineering