SAM-QFS Tar Format Update [PSARC/2008/267 FastTrack timeout 05/07/2008]

[Don Cragun]

>Date: Tue, 06 May 2008 15:38:56 -0500
>From: Dean Roehrich <Dean.Roehrich at sun.com>
>
>On Tue, May 06, 2008 at 10:32:28AM -0700, Don Cragun wrote:
>> >Detail:
 ... ... ...
>> >	The default behavior will call for 'sam-arcopy' to continue writing
>> >	the old-style tar format.  The new format can be selected by setting a
>> >	new 'tar_format = posix' option in the existing SAM-QFS config file,
>> >	/etc/opt/SUNWsamfs/defaults.conf.
>> 
>> Why isn't this 'tar_format = pax'?  The POSIX standard specifies three
>
>The project team agreed to this change.

OK.

>
 ... ... ...
>> 
>> In the standard, ustar and pax archive formats have magic as a 6 octet
>> field.  Am I correct in assuming that you mean magic will be set to
>> "ustar" (with a trailing null byte) and version will be set to "00"
>> (with no trailing null) as specified for the ustar and pax archive
>> formats?
>
>The functional spec has been clarified to say:
>
>	The pax format requires that the magic is set to "ustar[null]" and the
>	version to "00" (which is not null terminated).
>
>
>I've updated the case with the new spec.

OK.

>
>
>> >	If an old version of SAM-QFS attempts to read a new pax format tar
>> >	header it will declare the file 'damaged', which is an existing
>> >	SAM-QFS concept.  No legacy patch is planned, and this incompatibility
>> >	will be documented for customers.
>> >
>> >	The SAM-QFS 'sls -D' command and option, a SAM-QFS-aware version of
>> >	ls(1), will be modified to report the tape format used for the given
>> >	file, if that file is currently on tape.  It will make this decision
>> >	based on an available bit in the on-disk inode, which is currently
>> >	zeroed on existing SAM-QFS filesystems.
>> >
>> >	A new versioned library, libpax_xhdr.so, will be added to SAM-QFS.
>> >	This library is created by abstracting the tar-specific bits out of
>> >	'stager' and 'arcopy'.
>> 
>> Rather than factor tar, pax, stager, and arcopy internals into a new
>> library, why not just invoke the pax utility to read and write pax
>> archives?
>
>The sam-arcopy and sam-stagerd daemons are able to use filesystem-specific I/O
>operations to read and write the files to/from the filesystem.  This means
>files "invisibly" move between the disk and tape, which is a necessary feature
>of HSM products such as SAM-QFS to make the disk appear "bottomless".
>
>The sam-arcopy and sam-stagerd daemons are highly threaded to optimize their
>particular activity profile, and they use direct I/O.  That activity profile
>also makes it expensive to fork+exec a tar command for every file request,
>because they jump around a tape rather than reading/writing it from start to
>finish.

Tapes much have changed a lot since I stopped using them regularly.  I
don't remember any drives (other than DECtapes) where you could skip
around on the tape while writing files to it.  You could do it on
reads, but it was highly inefficient.

>
>In summary: the current design is that these daemons have the bare-bones bits
>necessary to read and write the archives.  The project team doesn't want to
>redesign these daemons, they just want to add the new header format.

OK.  Thanks for the info.

 - Don

>
>Dean