[cifs-discuss] [zfs-discuss] [nfs-discuss] Why can't we write to files created in multi-protocol se

[John Keiffer]

Mark,

Does it matter that the share IS mounted nfsv4?

>From the client:

bash-3.1$ mount
<ip>:/volumes/ha2/f1 on /mnt/leo1/nfsv3 type nfs4 (rw,addr=<ip>)

bash-3.1$ pwd
/mnt/leo1/nfsv3

bash-3.1$ nfs4_getfacl .
A::smb at matrix:rwaDxTnNCo
A::nobody:rwaDxTnNCo
D::OWNER@:
A::OWNER@:rwaxTNCo
D:g:GROUP@:wa
A:g:GROUP@:rx
D::EVERYONE@:waTC
A::EVERYONE@:rxtncy

bash-3.1$ nfs4_getfacl cifs.txt
D::OWNER@:rwax
A::OWNER@:TNCo
D:g:GROUP@:rwax
A:g:GROUP@:
D::EVERYONE@:rwaxTC
A::EVERYONE@:tncy

This system can't do 'ls -V', so I'm having to use nfs4_getfacl instead (not as convienient).

Thanks,
John

-----Original Message-----
From: Mark.Shellenbaum at Sun.COM [mailto:Mark.Shellenbaum at Sun.COM] 
Sent: Thursday, July 02, 2009 6:16 AM
To: Afshin Salek
Cc: John Keiffer; zfs-discuss at opensolaris.org; cifs-discuss at opensolaris.org
Subject: Re: [zfs-discuss] [cifs-discuss] [nfs-discuss] Why can't we write to files created in multi-protocol se

Afshin Salek wrote:
> I can't really explain the changes that happen to the file's
> ACL using vi over NFS. I'm CC'ing zfs-discuss maybe someone
> there can help out.
> 
> Afshin
> 

This is caused by vim trying to preserve ACLs and the NFSv3 server 
making some bad assumptions.

What is happening is that vim tries to find out what if any POSIX draft 
ACLs exist on the file.  POSIX draft ACLs aren't supported by ZFS and 
the file system returns ENOSYS.  The NFS server sees this error and is 
afraid that returning that error will cause problems for the client so 
it fabricates an ACL based on the mode bits of the file.  This causes 
"vim" to see an ACL that equates to a mode of 000.  Then after writing 
the data "vim" restores an ACL that equates to the mode.  The NFS server 
actually translates the POSIX draft ACL into a NFSv4 ACL and that is the 
ACL you actual see on the file after the exiting "vim".

Here is the snipit of code from NFS that actually causes the problem
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/nfs/nfs_acl_srv.c#98

The assumption made in the comment really aren't accurate anymore. 
Solaris can generally deal with an error from VOP_GETSECATTR() now and 
probably predates ZFS being integrated into ON.

There are a couple of ways to work around the problem.

- Recompile "vim" without ACL support.
- Use NFSv4 instead of NFSv3

This really should be a bug that needs to be investigated by the NFS 
team to determine if/when they really need to fabricate an ACL.  I would 
guess they probably don't need to do that anymore.

   -Mark


> John Keiffer wrote:
>> Looks like this:
>>
>> nmc at leo-ha2:/$ ls -Vd ha2/f1/
>> drwxr-xr-x+  3 enguser  root           4 Jul  1 14:51 ha2/f1/
>>                user:smb:rwxp-D-ARW-Co-:-------:allow
>>                user:nfs:rwxp-D-ARW-Co-:-------:allow
>>                  owner@:--------------:-------:deny
>>                  owner@:rwxp---A-W-Co-:-------:allow
>>                  group@:-w-p----------:-------:deny
>>                  group@:r-x-----------:-------:allow
>>               everyone@:-w-p---A-W-Co-:-------:deny
>>               everyone@:r-x---a-R-c--s:-------:allow
>>
>> Thanks,
>> John
>>
>> -----Original Message-----
>> From: Afshin.Ardakani at Sun.COM [mailto:Afshin.Ardakani at Sun.COM] Sent: 
>> Wednesday, July 01, 2009 6:17 PM
>> To: John Keiffer
>> Cc: cifs-discuss at opensolaris.org
>> Subject: Re: [cifs-discuss] [nfs-discuss] Why can't we write to files 
>> created in multi-protocol se
>>
>> How does the ACL for 'f1' look like?
>>
>> Afshin
>>
>> John Keiffer wrote:
>>> Well... I may have had an idamp problem before, which I believe I've 
>>> now corrected. This is my current idamp config:
>>>
>>> add     "wingroup:Domain Users at matrix.lab"      unixgroup:group2
>>> add     winuser:enguser at matrix.lab      unixuser:enguser
>>> wingroup:Domain Admins at matrix.lab       ==      gid:2147483650
>>> wingroup:Authenticated Users    ==      gid:2147483651
>>> wingroup:Network        ==      gid:2147483652
>>> wingroup:Administrators at BUILTIN ==      gid:2147483653
>>>
>>>
>>> I still have some questions regarding access from both CIFS and NFS:
>>>
>>> After steping on the file from Linux and vi with the ! I believe it 
>>> reordered the ACL's like this:
>>>
>>> nmc at leo-ha2:/$ ls -V ha2/f1/
>>> total 2
>>> ----------+  1 enguser  group2         6 Jul  1 14:32 cifs.txt
>>>            group:group2:rwxp----------:-------:deny
>>>               everyone@:r-x--------Co-:-------:deny
>>>            group:group2:-------------s:-------:allow
>>>            user:enguser:rwxpdDaARWcCos:fd-----:allow
>>>               everyone@:------a-R-c--s:-------:allow
>>>
>>> Which means that when I try and access it from Windows I can't, 
>>> because group2 has write deny (among other things). If I remove the 
>>> user ACL and insert it at the beginning, I can write again from 
>>> Windows...
>>>
>>> nmc at leo-ha2:/$ chmod A3- ha2/f1/cifs.txt
>>>
>>> nmc at leo-ha2:/$ chmod A0+user:enguser:rwxpdDaARWcCos:fd-----:allow 
>>> ha2/f1/cifs.txt
>>> nmc at leo-ha2:/$ ls -V ha2/f1/
>>> total 2
>>> ----------+  1 enguser  group2         6 Jul  1 14:32 cifs.txt
>>>            user:enguser:rwxpdDaARWcCos:fd-----:allow
>>>            group:group2:rwxp----------:-------:deny
>>>               everyone@:r-x--------Co-:-------:deny
>>>            group:group2:-------------s:-------:allow
>>>               everyone@:------a-R-c--s:-------:allow
>>>
>>> Until I ! save it again from Linux, because then the ACLs are changed 
>>> (such that nobody can do much of anything because of the deny lines):
>>>
>>> nmc at leo-ha2:/$ ls -V ha2/f1/cifs.txt
>>> ----------   1 enguser  group2        27 Jul  1 14:48 ha2/f1/cifs.txt
>>>                  owner@:rwxp----------:-------:deny
>>>                  owner@:-------A-W-Co-:-------:allow
>>>                  group@:rwxp----------:-------:deny
>>>                  group@:--------------:-------:allow
>>>               everyone@:rwxp---A-W-Co-:-------:deny
>>>               everyone@:------a-R-c--s:-------:allow
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss