[sam-qfs-discuss] multiple initiators - concurrent block rw access - Is it true ?

[Henry Newman]

I am not sure that striping will matter as the limitation will be the IP
network more than likely than the RAID LUNs.  Round-robin will be much
faster for multiple streams of I/O given storage contention. That was the
original reason round-robin was developed.  

Regards,

Henry

> -----Original Message-----
> From: sam-qfs-discuss-bounces at opensolaris.org [mailto:sam-qfs-discuss-
> bounces at opensolaris.org] On Behalf Of Peter Englmaier
> Sent: Thursday, July 24, 2008 4:05 AM
> To: Wilkinson, Alex
> Cc: sam-qfs-discuss at opensolaris.org
> Subject: Re: [sam-qfs-discuss] multiple initiators - concurrent block
> rw access - Is it true ?
> 
> >
> Hi Alex:
> 
> > In a nutshell, what we want is multiple intiators to concurrently
> have
> > read()/write() access to a single large 30TB LUN via fibre channel
> i.e
> > sharing a single LUN so that one initiator write()'s a file whilst
> > another
> > initiator read()'s a file.
> 
> So you want a shared filesystem. QFS can be used for that. However,
> qfs will be faster if you
> divide the LUN in several and make QFS stripe the data over them.
> 
> > I didn't think this was possible and that we would
> > have to use CIFS or NFS to do something like this. The problem is
> that
> > CIFS and NFS overheads slow IO down significantly (from what I have
> > read).
> 
> CIFS I don't know, but NFS can be almost wire speed. This is a matter
> of using large enough window sizes. QFS will be faster, because it has
> direct access to disk, so it can be as fast as you're disk raids are.
> We have for example ~ 400 MB/s for large files meassured with dd. With
> NFS we have ~85 MB/s over one 1 Gbits link. You can trunk multiple
> links to have higher speed (if clients are on the same subnet,
> otherwise the router might be a problem). Rumors say, that some recent
> 10 GBit/s NICs can reach 100% wire speed.
> 
> > At the block level what I dont understand is why multiple initiators
> > cannot do a
> > read() on a block concurrently without casuing corruption ? Does
> > that mean at
> > this level there is no file-locking (I suppose its not a file yet?)
> > and thats
> > why we need to use NFS or CIFS ? But surely UFS etc provides file
> > locking ?
> 
> QFS has the mh_write option:
> 
>       mh_write  Enables simultaneous reads and writes to the same
>                 file from multiple hosts.  If mh_write is used,
>                 the Sun StorEdge QFS shared file system switches
>                 all hosts into directio. The application must use
>                 page aligned memory buffers and well formed sector
>                 I/O (512 bytes).  Caution, if the application does
>                 not adhere to these alignment rules, data
>                 correctness is not guaranteed.
> 
> 
> If you enable this, you can write (at your own risk) from two servers
> accessing the same SAN.
> This can only cause file corruption, not filesystem corruption. If you
> write to the same block concurrently, then
> the last write wins. Maybe that's what you want?
> 
> 
> Best, Peter
> 
> _______________________________________________
> sam-qfs-discuss mailing list
> sam-qfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/sam-qfs-discuss