2008/688 Sun Cluster TCP/IP Hooks Update

[James Carlson]

I'm sponsoring this fast-track for Huafeng Lu and the Sun Cluster
team.  The timer is set to 11/17/2008, and a draft of the contract is
available in the case directory as "contract-01".

Note: the fast-track materials were presented to me as "open," though
many of the cases related to these hooks are currently closed.  The
change in the system that is the subject of this case, though, should
be reviewable independently, so this shouldn't cause problems.

Introduction

  "TCP/IP hooks for clustering" (PSARC 1997/314) introduced several
  TCP/IP hooks for the Sun Cluster product, and the contract is
  1997/314:contract-1.new.  These hooks are called by the Solaris
  TCP/IP code at certain places.  Now, to support client-side shared
  address (RFE 6717519), we need to modify one of these hooks,
  cl_inet_connect().

Release Binding

  A patch/micro release binding is asserted.  This will require the
  release of coordinated patches for Solaris and Sun Cluster.

Interfaces

    Exported Interface	       Stability               Comments
    ================== 	       =========               ========
    cl_inet_connect	       Removed (was Contracted) PSARC 1997/314
    cl_inet_connect2	       Contr. Project Private  function pointer
    cluster_version_supported  Contr. Project Private  cluster version
    						       supported by Solaris
    udp_t.udp_cl_v6lastdst     Project Private         most recent dest addr
    udp_t.udp_cl_lastdstport   Project Private         most recent dest port

Detailed Description

  Currently, a shared address is used only at the server side.  Server
  applications are started on Cluster nodes, which reply to requests
  coming from external clients.  This proposal (RFE 6717519) aims to
  allow Cluster nodes to use shared addresses as client addresses to
  talk to external servers.  Note: this proposal only handles TCP and
  UDP; SCTP is beyond its scope.

  For TCP, several Cluster nodes bind to the same shared address, then
  connect to external applications. The cl_inet_connect() hook is
  called at connect time (for both outgoing and incoming connection
  requests) to check if the 5-tuple is unique. The hook is added with
  a return value to indicate whether the check is passed, and an input
  parameter to indicate whether the connection is outgoing or
  incoming.

  The situation for UDP is a bit different.  Since there is no real
  connection in UDP and the same socket can be used to send to
  multiple destinations, the major task is to register the five-tuple
  at the cluster level.  Such registration is performed at connect(),
  sendto() and sendmsg(), which call the cl_inet_connect() hook at
  proper time.  For performance reasons, sendto()/sendmsg() calls the
  hook only when the destination is new.  To facilitate this, two new
  fields are added to the udp_t struct to store the most recent
  destination address and port: in6_addr_t udp_cl_v6lastdst; in_port_t
  udp_cl_lastdstport; In a sendto() or sendmsg(), the
  cl_inet_connect() hook is called only when the destination is
  different from most recent one.

Versioning

  In addition to the hook signature changes listed above, the hook
  name will also be changed for versioning reasons.  Thus, overall the
  new signature of the hook is:

	int (*cl_inet_connect2)(uint8_t protocol,
	    boolean_t is_outgoing, sa_family_t addr_family,
	    uint8_t *laddrp, in_port_t lport,
	    uint8_t *faddrp, in_port_t fport);

  Here's the old signature for comparison.  Note: after renaming, the
  old name cl_inet_connect is obsolete and will no longer be called
  from Solaris.

	void (*cl_inet_connect)(uint8_t protocol,
	    sa_family_t addr_family,
	    uint8_t *laddrp, in_port_t lport,
	    uint8_t *faddrp, in_port_t fport);

  With changes at both Solaris and Cluster sides, there is a "version
  matching" issue.  There is no mechanism to check if the Cluster
  version matches the Solaris release; the Cluster simply associate
  the hook with an implementation, if the hook is defined in Solaris.
  Thus, if the Cluster product is not running with the correct Solaris
  release, the hook will be associated with the wrong implementation,
  and may cause serious problems or even panic.

  To solve this problem, the function pointer is renamed as above.  To
  help with future versioning, a new version string is introduced in
  Solaris code:

	const char *cluster_version_supported = "1.1";

  The new Cluster code checks which Cluster version Solaris supports,
  and associates the corresponding hook implementation with the hook.

  This is a generic mechanism, but doesn't work for the currently
  released Cluster code, which doesn't check the version.  As a
  workaround, we rename the hook in Solaris so that the current
  Cluster version won't be loaded after upgrading ON.  This prevents
  the user from damaging his system, and release notes will indicate
  what the errors mean.

Out of Scope

  Note the actual implementation of these hooks are in the Cluster
  code, not in the Solaris code.  Solaris TCP/IP code only declares
  and calls them.  For this project, the Cluster team is responsible
  for changing the implementation of the cl_inet_connect2() hook in
  the Sun Cluster code.

-- 
James Carlson, Solaris Networking              <james.d.carlson at sun.com>
Sun Microsystems / 35 Network Drive        71.232W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677