[nfs-discuss] Re: [zfs-discuss] Re: NFS Performance and Tar

[Spencer Shepler]

On Tue, Roch wrote:
> 
> Joerg Schilling writes:
>  > Roch <Roch.Bourbonnais at Sun.COM> wrote:
>  > 
>  > > I would   add that  this  is not   a  bug or  deficientcy in
>  > > implementation. Any NFS implementation tweak to make 'tar x'
>  > > go  as fast as   direct attached will  lead  to silent  data
>  > > corruption (tar x   succeeds but the files  don't  checksum
>  > > ok). 
>  > >
>  > > Interestingly the quality of service of 'tar x' is higher
>  > > over NFS than direct attach since, with direct attach,
>  > > there is no guarantee that files are set to storage whereas
>  > > there is one with NFS.
>  > 
>  > Why do you beieve this?
>  > 
>  > Neither Sun tar nor GNU tar call fsync which is the only way to 
>  > enforce data integrity over NFS.
> 
> I tend to agree with this although I'd say that in practice,
> from performance perspective,  calling fsync should  be more
> relevant for direct attach.   For NFS, doesn't close-to-open
> and other  aspect of the protocol need  to enforce much more
> synchronous  operations ?  For tar  x over  nfs I'd bet  the
> fsync will be an over-the-wire ops (say  0.5ms) but will not
> add an additional I/O latency (5ms) on each file extract.

The close-to-open behavior of NFS clients is what ensures that the
file data is on stable storage when close() returns.
The meta-data requirements of NFS is what ensures that file creation,
removal, renames, etc. are on stable storage when the server
sends a response.

So, unless the NFS server is behaving badly, the NFS client has
a synchronous behavior and for some that means more "safe" but
usually means that it is also slower that local access.

> My target for a single threaded 'tar x' of small files is to
> be able to run over NFS at 1 file per I/O latency, no matter
> what the backend FS is. I  guess that 'star -yes-fsync' over
> direct attach  should   behave the same  ?  Or  do you  have
> concurrency in there...see below.
> 
>  > 
>  > If you like to test this, use star. Star by default calls 
>  > fsync before it closes a written file in x mode. To switch this
>  > off, use star -no-fsync.
>  > 
>  > > Net Net, for single threaded 'tar x', data integrity
>  > > consideration forces NFS to provide a high quality slow
>  > > service. For direct attach, we don't have those data
>  > > integrity issues, and the community has managed to get by the 
>  > > lower quality higher speed service.
>  > 
>  > What dou you have in mind?
>  > 
>  > A tar that calls fsync in detached threads?
>  > 
> 
> You tell me ? We have 2 issues
> 
> 	can we make 'tar x' over direct attach, safe (fsync)
> 	and posix compliant while staying close to current
> 	performance characteristics ? In other words do we
> 	have the posix leeway to extract files in parallel ?
> 
> 	For NFS, can we make 'tar x' fast and reliable while
> 	keeping a principle of  least surprise for users  on
> 	this non-posix FS.

Having tar create/write/close files concurrently would be a 
big win over NFS mounts on almost any system.

Spencer