[sam-qfs-discuss] multiple initiators - concurrent block rw access - Is it true ?

[Peter Englmaier]

>
Hi Alex:

> In a nutshell, what we want is multiple intiators to concurrently have
> read()/write() access to a single large 30TB LUN via fibre channel i.e
> sharing a single LUN so that one initiator write()'s a file whilst  
> another
> initiator read()'s a file.

So you want a shared filesystem. QFS can be used for that. However,  
qfs will be faster if you
divide the LUN in several and make QFS stripe the data over them.

> I didn't think this was possible and that we would
> have to use CIFS or NFS to do something like this. The problem is that
> CIFS and NFS overheads slow IO down significantly (from what I have  
> read).

CIFS I don't know, but NFS can be almost wire speed. This is a matter  
of using large enough window sizes. QFS will be faster, because it has  
direct access to disk, so it can be as fast as you're disk raids are.  
We have for example ~ 400 MB/s for large files meassured with dd. With  
NFS we have ~85 MB/s over one 1 Gbits link. You can trunk multiple  
links to have higher speed (if clients are on the same subnet,  
otherwise the router might be a problem). Rumors say, that some recent  
10 GBit/s NICs can reach 100% wire speed.

> At the block level what I dont understand is why multiple initiators  
> cannot do a
> read() on a block concurrently without casuing corruption ? Does  
> that mean at
> this level there is no file-locking (I suppose its not a file yet?)  
> and thats
> why we need to use NFS or CIFS ? But surely UFS etc provides file  
> locking ?

QFS has the mh_write option:

      mh_write  Enables simultaneous reads and writes to the same
                file from multiple hosts.  If mh_write is used,
                the Sun StorEdge QFS shared file system switches
                all hosts into directio. The application must use
                page aligned memory buffers and well formed sector
                I/O (512 bytes).  Caution, if the application does
                not adhere to these alignment rules, data
                correctness is not guaranteed.


If you enable this, you can write (at your own risk) from two servers  
accessing the same SAN.
This can only cause file corruption, not filesystem corruption. If you  
write to the same block concurrently, then
the last write wins. Maybe that's what you want?


Best, Peter