irods / irods_client_nfsrods Goto Github PK

An nfs4j Virtual File System implementation supporting the iRODS Data Grid

License: BSD 3-Clause "New" or "Revised" License

Java 94.71% Shell 4.77% Dockerfile 0.52%

irods_client_nfsrods's Introduction

The Integrated Rule-Oriented Data System (iRODS) is open source data management software used by research, commercial, and governmental organizations worldwide.

iRODS is released as a production-level distribution aimed at deployment in mission critical environments. It virtualizes data storage resources, so users can take control of their data, regardless of where and on what device the data is stored.

The development infrastructure supports exhaustive testing on supported platforms; plugin support for microservices, storage resources, authentication mechanisms, network protocols, rule engines, new API endpoints, and databases; and extensive documentation, training, and support services.

iRODS: https://irods.org
iRODS Consortium: https://irods.org/consortium
RENCI: https://renci.org

Core Competencies

iRODS implements data virtualization, allowing access to distributed storage assets under a unified namespace, and freeing organizations from getting locked in to single-vendor storage solutions.
iRODS enables data discovery using a metadata catalog that describes every data object, collection, and every storage resource in the iRODS Zone.
iRODS automates data workflows, with a rule engine framework that permits any action to be initiated by any trigger on any server or client in the Zone.
iRODS enables secure collaboration, so users only need to log in to their home Zone to access data hosted on a remote Zone.

History

iRODS has a 25+ year history of funded projects.

Funders have included DARPA, NSF, DOD, DOE, LC, NARA, NASA, NOAA, USPTO, and LLNL.

https://irods.org/history

License

iRODS is released under a 3-clause BSD License.

Reporting Security Vulnerabilities

See SECURITY.md for details.

Links to elsewhere...

irods_client_nfsrods's People

Contributors

Stargazers

Watchers

Forkers

korydraughn trel ekoi mpuskaric rudibroekhuizen mherkazandjian jbeal-work

irods_client_nfsrods's Issues

Setting permissions for unknown users breaks NFSRODS ACL support

If nfs4_setfacl is used to modify permissions for a user that does not exist in the OS and iRODS, NFSRODS will delete all permissions and will not be able to recover.

Fixing the permissions from the client-side does not appear to be possible due to the fact that NFSRODS caches ACL information.

Fixing the permissions from the server-side is possible, but NFSRODS will not detect the changes.

The only known solution is to restart the NFSRODS server and remount it.

duplicate directory entries in nfsrods mount

Our nfsrods mount uses lookupcache=none

[root@irods1 ~]# mount -t nfs4
localhost:/ on /mnt/nfsrods type nfs4 (rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp6,timeo=600,retrans=2,sec=sys,clientaddr=::1,lookupcache=none,local_lock=none,addr=::1)

But today I noticed that it was still returning duplicate directory entries.

[root@irods1 ~]# ls /mnt/nfsrods/home
frahm   holtat    jobl6604  monaghaa  pabi5658  public  shouse
frahm   joan5896  jobl6604  nfsrods   pabi5658  rods    shouse
holtat  joan5896  monaghaa  nfsrods   public    rods

re-mounting cleared this up; but it's troubling nonetheless.

[root@irods1 ~]# umount /mnt/nfsrods
[root@irods1 ~]# mount -v -t nfs4 -o sec=sys,port=2049,lookupcache=none localhost:/ /mnt/nfsrods
mount.nfs4: timeout set for Wed Sep 18 12:46:53 2019
mount.nfs4: trying text-based options 'sec=sys,port=2049,lookupcache=none,vers=4.1,addr=::1,clientaddr=::1'
[root@irods1 ~]# ls /mnt/nfsrods/home
frahm  holtat  joan5896  jobl6604  monaghaa  nfsrods  pabi5658  public  rods  shouse

until chmod is implemented, return a better error code

unix chmod currently returns Remote I/O Error as it is not implemented by the NFSRODS server.

Remove support for anything other than NFSv4

NFSRODS requires NFSv4 ACLs in order to handle iRODS permissions properly.

Properly handle groups in iRODS

i/o error and traceback when accessing NFSRODS as root

This TB should be handled, and the error code should probably be "permission denied."

[root@irods1 ~]# sha1sum /mnt/home/joan5896/gpfs-expels.csv 
sha1sum: /mnt/home/joan5896/gpfs-expels.csv: Input/output error

traceback.txt

i/o error when a single replica is missing

Recently I got an i/o error from NFSRODS when trying to access a file.

[janderson@fox1 ~]$ sha1sum /mnt/nfsrods/home/janderson/spore.bb
sha1sum: /mnt/nfsrods/home/janderson/spore.bb: Input/output error

This file has three replicas under a replResc.

[janderson@fox1 ~]$ ilsresc
rootResc:passthru
└── replResc:replication
    ├── fox1Resc:unixfilesystem
    ├── mybook:unixfilesystem
    └── rsync_net:unixfilesystem
www:passthru
└── ln1:unixfilesystem

[janderson@fox1 ~]$ ils -AL spore.bb
  janderson         0 rootResc;replResc;fox1Resc        11101 2018-12-03.12:32 & spore.bb
    sha2:ZVhrwYvtAvQDdhspTCxz1z8XO9u6YI90bxrkZOWYLHI=    generic    /srv/civilfritz/irods/Vault/home/janderson/spore.bb
        ACL - janderson#civilfritz.net:own   
  janderson         1 rootResc;replResc;mybook        11101 2018-12-12.15:08 & spore.bb
    sha2:ZVhrwYvtAvQDdhspTCxz1z8XO9u6YI90bxrkZOWYLHI=    generic    /media/mybook/Vault/home/janderson/spore.bb
        ACL - janderson#civilfritz.net:own   
  janderson         2 rootResc;replResc;rsync_net        11101 2019-03-25.21:13 & spore.bb
    sha2:ZVhrwYvtAvQDdhspTCxz1z8XO9u6YI90bxrkZOWYLHI=    generic    /media/rsync.net/Vault/home/janderson/spore.bb
        ACL - janderson#civilfritz.net:own

and one of these resources was unmounted. After mounting, it works.

[janderson@fox1 ~]$ sudo -u irods sshfs -o idmap=user [email protected]: /media/rsync.net
[janderson@fox1 ~]$ sha1sum /mnt/nfsrods/home/janderson/spore.bb
93b18b58ba1aa3cccdd0be0dfde67ecd73290e58  /mnt/nfsrods/home/janderson/spore.bb

I understand that this is ultimately a limitation in irods itself; but if there are replicas available, they should all be consulted before returning an i/o error.

public access collections not showing up during ls

I ran the following series of ichmod to grant public read access to some top-level directories

ichmod -M read public /
ichmod -M read public /curcZone
ichmod -M read public /curcZone/pl
ichmod -M read public /curcZone/pl/archive

This collection contains other sub-collections

[root@irods1 ~]# ils /curcZone/pl/archive | sed 's,archive/.*,archive/[redacted],'
/curcZone/pl/archive:
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]
  C- /curcZone/pl/archive/[redacted]

but when I ls this with NFSRODS there are no directories shown.

[root@irods1 ~]# sudo -u joan5896 ls /mnt/nfsrods/pl/archive
[root@irods1 ~]#

The behavior does not change if I re-mount NFSRODS.

effective uid not honored or not set correctly

I have a local account, joan5896 with uid 416810:

# sudo -u joan5896 id --user
416810
# sudo -u joan5896 id --user --name
joan5896

This account is mapped into NFSRODS.

docker run -d --name nfsrods \
             -p 2049:2049 \
             -v /etc/irods/nfsrods:/nfsrods_config:ro \
             -v /etc/irods/nfsrods-passwd:/etc/passwd:ro \
             -v /etc/irods/nfsrods-shadow:/etc/shadow:ro \
             nfsrods

[root@irods1 ~]# grep joan5896 /etc/irods/nfsrods-passwd 
joan5896:*:416810:416810:Jonathon Anderson,,,:/home/joan5896:/bin/bash

My home directory is owned by the irods user with this same name.

# sudo -u irods ils -AL /curcZone/home/joan5896
/curcZone/home/joan5896:
        ACL - joan5896#curcZone:own   
        Inheritance - Disabled
  joan5896          0 demoResc          342 2019-08-14.22:12 & gpfs-expels.csv
        generic    /var/lib/irods/Vault/home/joan5896/gpfs-expels.csv
        ACL - joan5896#curcZone:own   
  joan5896          1 strongbox1_01          342 2019-08-14.22:14 & gpfs-expels.csv
        generic    /mnt/strongbox1_01/home/joan5896/gpfs-expels.csv
        ACL - joan5896#curcZone:own   
  joan5896          2 strongbox2_01          342 2019-08-15.15:10 & gpfs-expels.csv
        generic    /mnt/strongbox2_01/home/joan5896/gpfs-expels.csv
        ACL - joan5896#curcZone:own

But an attempt to read this directory as this user returns permission denied.

[root@irods1 ~]# sudo -u joan5896 ls /mnt/nfsrods/home/joan5896
ls: cannot access /mnt/nfsrods/home/joan5896: Permission denied
[root@irods1 ~]# su joan5896 -c "ls /mnt/nfsrods/home/joan5896"
ls: cannot access /mnt/nfsrods/home/joan5896: Permission denied

irods-permission-denied.txt

add test runner for posix test suite

Handle overflow for inode numbers

Replace principal parsing code with more weapons-grade solution

Failed to register at portmap: portmap service not available

Sep 30 22:28:43 irods1.rc.int.colorado.edu docker[12731]: 2019-10-01 04:28:43.973 WARN  Thread-1 [OncRpcSvc] - Failed to register at portmap: portmap service not available

If NFSRODS is not meant to be able to register at portmap in the container, then it should be configured to not attempt to register at portmap. Otherwise, this WARN is noise that might lead an admin to believe something is wrong.

NFSRODS should support user and group auto-creation

When using nfs4_setfacl to set permissions for a user and/or group that exists locally, but not in iRODS, NFSRODS should support the ability to automatically create these users and groups in iRODS to properly reflect the intended permissions as mapped from upstream.

Remove CMake packaging files and instructions

NFSRODS is meant to be deployed as a docker container. CMake is not needed for this.

Properly map permissions from iRODS

Upgrade Log4j version from 1.x to 2.x

warn on deprecated configuration settings

log a WARNING for formerly used configuration settings

including, but not limited to...

user_information_refresh_time_in_minutes
irods_proxy_admin_account
irods_server

Add test cases

survive underlying irods restart

NFSRODS should either be able to survive an irods restart, or automatically reconnect.

[root@irods1 _build]# systemctl restart irods
[root@irods1 _build]# ls /mnt/nfsrods 
ls: cannot access /mnt/nfsrods: Remote I/O error

add negotiation_policy to irods_client configuration

an enum with possible values:
CS_NEG_REQUIRE,
CS_NEG_DONT_CARE.
CS_NEG_REFUSE

Default should be set to CS_NEG_REFUSE.

Most environments with direct network connections will not need to use SSL. This default setting will allow Zones configured to normally use PAM auth (and SSL) for their users to have non-SSL mount point into the Zone via NFSRODS.

Move hard-coded values into configuration file

change "user_information_refresh_time_in_minutes" to "user_information_refresh_time_in_milliseconds"

to keep consistent with the other milliseconds settings

Create CMakefile for packaging

deb
rpm
packaged /etc/nfsrods/README

restarting NFSRODS without re-mounting at the client produces incorrect behavior

If I start NFSRODS and mount, I can see the VFS correctly with ls.

[root@irods1 ~]# docker start nfsrods
nfsrods
[root@irods1 ~]# docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS                    NAMES
879249206d41        nfsrods             "./start.sh"        3 days ago          Up 20 seconds       0.0.0.0:2049->2049/tcp   nfsrods
[root@irods1 ~]# mount -o sec=sys,port=2049 localhost:/ /mnt/nfsrods
[root@irods1 ~]# ls /mnt/nfsrods/home/joan5896
check_x509.py  gpfs-expels.csv  quota-test  x-get-selection-owner.c  zeroes

But if I restart NFSRODS without re-mounting, I often get incorrect and inconsistent behavior.

[root@irods1 ~]# docker restart nfsrods
nfsrods
[root@irods1 ~]# ls /mnt/nfsrods/home/joan5896/
ls: cannot access /mnt/nfsrods/home/joan5896/: No such file or directory
[root@irods1 ~]# ls /mnt/nfsrods/
ls: cannot access /mnt/nfsrods/home: No such file or directory
ls: cannot access /mnt/nfsrods/pl: No such file or directory
ls: cannot access /mnt/nfsrods/trash: No such file or directory
home  pl  trash

Note in the last example that the directories were returned and reported "No such file or directory."

This behavior seems to resolve itself after a short time.

Investigate caching constructed objects

this may go well with the proposed use of REDIS for the inode to iRODS path
mapping

Update Collection MTime before listing collection contents

This would solve the issue of possibly leaking information between users.

file with only read access not readable

Trying to read/get a file with only read access is triggering CAT_NO_ACCESS_PERMISSION in the rodsLog.

Setting read/write permission on the file allows the read to complete successfully.

nfsrods should log a WARNING message if an export does not have acls enabled

nfsrods depends on acl support being enabled at the export for proper behavior; but if it's not (particularly, as was our case, if your configuration predates d1159b4) it becomes difficult to track down the permission denied errors.

NFSRODS should log during startup if a defined export does not have acls enabled. This should presumably go here:

https://github.com/irods/irods_client_nfsrods/blob/master/irods-vfs-impl/src/main/java/org/irods/nfsrods/vfs/ServerMain.java#L88

Inaccessible objects should not be listed by "ls"

Users should only be able to list objects they have access to.

Create Dockerfile for building

Handle case where the uid for "nobody"/"nfsnobody" user/group is not defined in the /etc/passwd file

"nobody" vs "nfsnobody" appears to be OS-dependent.

change internal configuration location

For readability, consider changing internal configuration location from
/nfsrods_ext
to
/nfsrods_config

Multi-owned collections and data objects are not accessible

Given two users, alice and bob.

alice gives own permissions on a collection to bob.

ichmod own bob /tempZone/home/alice/shared

Now bob can use iput to put files into the collection shared by alice. However, if bob tries to put a file, via an NFSRODS mount, into the collection shared by alice, then NFSRODS stops it and causes a permission error to be printed to the console.

Due to the linux permissions model, sharing collections and data objects is not possible without allowing all users some access to all collections in the path.

move "irods_proxy_admin_account" configs to "irods_client" section

move and rename

    "irods_proxy_admin_account": {
        "username": "rods",
        "password": "rods"
    }

    "irods_client":
        "proxy_rodsadmin_username": "rods",
        "proxy_rodsadmin_password": "rods"
    }

unhandled exception when logical quotas policy denies access

I'm testing the new logical quotas policy plugin.

https://github.com/korydraughn/irods_rule_engine_plugin_logical_quotas

When it denies write due to policy enforcement it generates an io error in NFSRODS due to an unhandled traceback.

[root@irods1 _build]# sudo -u joan5896 cp /etc/hosts /mnt/nfsrods/home/joan5896/hosts.7
cp: cannot create regular file ‘/mnt/nfsrods/home/joan5896/hosts.7’: Remote I/O error

Best case would be for this to generate a quota exceeded error. Barring that, there's probably something better than "Remote I/O error" from an unhandled exception.

in rodslog:

Oct 29 19:36:18 pid:5396 remote addresses: 10.225.128.219, 172.17.0.2 ERROR: Policy Violation: Adding object exceeds maximum number of objects limit

in nfsrods:

2019-10-30 01:36:18.153 DEBUG Thread-16 [IRODSVirtualFileSystem] - vfs::checkAcl
2019-10-30 01:36:18.153 DEBUG Thread-16 [IRODSIdMapper] - resolveUser - _userID = 416810
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _subject uid         = 416810
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _subject primary gid = 416810
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _inode path          = /curcZone/home/joan5896
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask          = 128
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - username             = joan5896
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_READ_DATA         = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_LIST_DIRECTORY    = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_WRITE_DATA        = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_ADD_FILE          = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_APPEND_DATA       = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_ADD_SUBDIRECTORY  = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_READ_NAMED_ATTRS  = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_WRITE_NAMED_ATTRS = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_EXECUTE           = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_DELETE_CHILD      = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_READ_ATTRIBUTES   = 128
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_WRITE_ATTRIBUTES  = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_DELETE            = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_READ_ACL          = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_WRITE_ACL         = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_WRITE_OWNER       = 0
2019-10-30 01:36:18.154 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_SYNCHRONIZE       = 0
2019-10-30 01:36:18.165 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - Object is a collection, access allowed.
2019-10-30 01:36:18.165 DEBUG Thread-16 [IRODSIdMapper] - resolveUser - _userID = 416810
2019-10-30 01:36:18.165 WARN  Thread-16 [IRODSSession] - closing a connection that is not held, silently ignore
2019-10-30 01:36:18.165 DEBUG Thread-16 [IRODSVirtualFileSystem] - vfs::getattr
2019-10-30 01:36:18.165 DEBUG Thread-16 [IRODSVirtualFileSystem] - statPath - _inodeNumber          = 3
2019-10-30 01:36:18.165 DEBUG Thread-16 [IRODSVirtualFileSystem] - statPath - _path                 = /curcZone/home/joan5896
2019-10-30 01:36:18.165 DEBUG Thread-16 [IRODSIdMapper] - resolveUser - _userID = 416810
2019-10-30 01:36:18.171 DEBUG Thread-16 [IRODSVirtualFileSystem] - statPath - iRODS stat info   = ObjStat [absolutePath=/curcZone/home/joan5896, objectPath=, objectType=COLLECTION, dataId=2832164, checksum=, ownerName=joan5896, ownerZone=curcZone, objSize=0, createdAt=Thu Aug 15 03:40:38 UTC 2019, modifiedAt=Wed Oct 30 00:16:40 UTC 2019, specColType=NORMAL, collectionPath=, cacheDir=, cacheDirty=false, replNumber=0, standInGeneratedObjStat=false]
2019-10-30 01:36:18.171 DEBUG Thread-16 [IRODSVirtualFileSystem] - statPath - Secret owner name = joan5896
2019-10-30 01:36:18.171 DEBUG Thread-16 [IRODSVirtualFileSystem] - getObjectType - Returning cached object type for [/curcZone/home/joan5896] ...
2019-10-30 01:36:18.204 DEBUG Thread-16 [IRODSVirtualFileSystem] - setStatMode - _path = /curcZone/home/joan5896
2019-10-30 01:36:18.235 DEBUG Thread-16 [IRODSVirtualFileSystem] - calcMode - permission =
UserFilePermission
    userName:joan5896
    userId:
    filePermissionEnum:OWN
   userType:RODS_USER
   userZone:curcZone
2019-10-30 01:36:18.235 DEBUG Thread-16 [IRODSVirtualFileSystem] - statPath - User ID           = 416810
2019-10-30 01:36:18.235 DEBUG Thread-16 [IRODSVirtualFileSystem] - statPath - Group ID          = 65534
2019-10-30 01:36:18.235 DEBUG Thread-16 [IRODSVirtualFileSystem] - statPath - Permissions       = drwx------
2019-10-30 01:36:18.235 DEBUG Thread-16 [IRODSVirtualFileSystem] - statPath - Stat              = drwx------    1 416810 65534    0 Oct 30 00:16
2019-10-30 01:36:18.236 DEBUG Thread-16 [IRODSIdMapper] - resolveUser - _userID = 416810
2019-10-30 01:36:18.236 WARN  Thread-16 [IRODSSession] - closing a connection that is not held, silently ignore
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - vfs::checkAcl
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSIdMapper] - resolveUser - _userID = 416810
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _subject uid         = 416810
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _subject primary gid = 416810
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _inode path          = /curcZone/home/joan5896
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask          = 2
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - username             = joan5896
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_READ_DATA         = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_LIST_DIRECTORY    = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_WRITE_DATA        = 2
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_ADD_FILE          = 2
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_APPEND_DATA       = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_ADD_SUBDIRECTORY  = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_READ_NAMED_ATTRS  = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_WRITE_NAMED_ATTRS = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_EXECUTE           = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_DELETE_CHILD      = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_READ_ATTRIBUTES   = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_WRITE_ATTRIBUTES  = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_DELETE            = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_READ_ACL          = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_WRITE_ACL         = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_WRITE_OWNER       = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - _accessMask & ACE4_SYNCHRONIZE       = 0
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - No attribute/ACL operations requested.
2019-10-30 01:36:18.237 DEBUG Thread-16 [IRODSVirtualFileSystem] - getObjectType - Returning cached object type for [/curcZone/home/joan5896] ...
2019-10-30 01:36:18.273 DEBUG Thread-16 [IRODSVirtualFileSystem] - checkAcl - User is an owner, access allowed.
2019-10-30 01:36:18.274 DEBUG Thread-16 [IRODSIdMapper] - resolveUser - _userID = 416810
2019-10-30 01:36:18.274 WARN  Thread-16 [IRODSSession] - closing a connection that is not held, silently ignore
2019-10-30 01:36:18.274 DEBUG Thread-16 [IRODSVirtualFileSystem] - vfs::create
2019-10-30 01:36:18.274 DEBUG Thread-16 [IRODSVirtualFileSystem] - create - _parent      = /curcZone/home/joan5896
2019-10-30 01:36:18.274 DEBUG Thread-16 [IRODSVirtualFileSystem] - create - _type        = REGULAR
2019-10-30 01:36:18.274 DEBUG Thread-16 [IRODSVirtualFileSystem] - create - _name        = hosts.7
2019-10-30 01:36:18.277 DEBUG Thread-16 [IRODSVirtualFileSystem] - create - _subject     = Subject:
        Principal: UidPrincipal[416810]
        Principal: GidPrincipal[416810,primary]
        Principal: GidPrincipal[416810]
        Principal: GidPrincipal[1000000]
        Principal: GidPrincipal[1000505]
        Principal: GidPrincipal[1000509]
        Principal: GidPrincipal[1100098]
        Principal: GidPrincipal[1101822]
        Principal: GidPrincipal[2001163]
        Principal: GidPrincipal[2002144]
        Principal: GidPrincipal[2002838]

2019-10-30 01:36:18.277 DEBUG Thread-16 [IRODSVirtualFileSystem] - create - _subject uid = 416810
2019-10-30 01:36:18.277 DEBUG Thread-16 [IRODSVirtualFileSystem] - create - _subject gid = 416810
2019-10-30 01:36:18.277 DEBUG Thread-16 [IRODSVirtualFileSystem] - create - _mode        = -rw-r--r--
2019-10-30 01:36:18.277 DEBUG Thread-16 [IRODSIdMapper] - resolveUser - _userID = 416810
2019-10-30 01:36:18.394 DEBUG Thread-16 [IRODSVirtualFileSystem] - create - Creating new file [irods://[email protected]:1247/curcZone/home/joan5896/hosts.7] ...
2019-10-30 01:36:18.412 ERROR Thread-16 [IRODSMidLevelProtocol] - IRODS error occured msg : -130000
2019-10-30 01:36:18.412 ERROR Thread-16 [IRODSFileImpl] - JargonException caught and rethrown as IOException:Invalid input parameter
org.irods.jargon.core.exception.InvalidInputParameterException: Invalid input parameter
        at org.irods.jargon.core.connection.IRODSErrorScanner.checkSpecificCodesAndThrowIfExceptionLocated(IRODSErrorScanner.java:189) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.connection.IRODSErrorScanner.inspectAndThrowIfNeeded(IRODSErrorScanner.java:115) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.connection.IRODSMidLevelProtocol.processMessageInfoLessThanZero(IRODSMidLevelProtocol.java:1399) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.connection.IRODSMidLevelProtocol.readMessage(IRODSMidLevelProtocol.java:903) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.connection.IRODSMidLevelProtocol.readMessage(IRODSMidLevelProtocol.java:871) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.connection.IRODSMidLevelProtocol.irodsFunction(IRODSMidLevelProtocol.java:284) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.connection.IRODSMidLevelProtocol.irodsFunction(IRODSMidLevelProtocol.java:410) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.pub.IRODSFileSystemAOImpl.createFileInResource(IRODSFileSystemAOImpl.java:880) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.pub.IRODSFileSystemAOImpl.createFile(IRODSFileSystemAOImpl.java:782) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.pub.io.IRODSFileImpl.createNewFile(IRODSFileImpl.java:347) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.nfsrods.vfs.IRODSVirtualFileSystem.create(IRODSVirtualFileSystem.java:247) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.dcache.nfs.vfs.PseudoFs.create(PseudoFs.java:156) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.dcache.nfs.v4.OperationOPEN.process(OperationOPEN.java:153) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.dcache.nfs.v4.NFSServerV41.NFSPROC4_COMPOUND_4(NFSServerV41.java:204) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.dcache.nfs.v4.xdr.nfs4_prot_NFS4_PROGRAM_ServerStub.dispatchOncRpcCall(nfs4_prot_NFS4_PROGRAM_ServerStub.java:48) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.dcache.oncrpc4j.rpc.RpcDispatcher$1.lambda$run$0(RpcDispatcher.java:100) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
        at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_222]
        at org.dcache.oncrpc4j.rpc.RpcDispatcher$1.run(RpcDispatcher.java:99) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:593) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:573) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
2019-10-30 01:36:18.412 DEBUG Thread-16 [IRODSIdMapper] - resolveUser - _userID = 416810
2019-10-30 01:36:18.412 ERROR Thread-16 [NFSServerV41] - Unhandled exception:
java.io.IOException: org.irods.jargon.core.exception.InvalidInputParameterException: Invalid input parameter
        at org.irods.jargon.core.pub.io.IRODSFileImpl.createNewFile(IRODSFileImpl.java:358) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.nfsrods.vfs.IRODSVirtualFileSystem.create(IRODSVirtualFileSystem.java:247) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.dcache.nfs.vfs.PseudoFs.create(PseudoFs.java:156) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.dcache.nfs.v4.OperationOPEN.process(OperationOPEN.java:153) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.dcache.nfs.v4.NFSServerV41.NFSPROC4_COMPOUND_4(NFSServerV41.java:204) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.dcache.nfs.v4.xdr.nfs4_prot_NFS4_PROGRAM_ServerStub.dispatchOncRpcCall(nfs4_prot_NFS4_PROGRAM_ServerStub.java:48) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.dcache.oncrpc4j.rpc.RpcDispatcher$1.lambda$run$0(RpcDispatcher.java:100) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_222]
        at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_222]
        at org.dcache.oncrpc4j.rpc.RpcDispatcher$1.run(RpcDispatcher.java:99) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:593) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:573) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]
Caused by: org.irods.jargon.core.exception.InvalidInputParameterException: Invalid input parameter
        at org.irods.jargon.core.connection.IRODSErrorScanner.checkSpecificCodesAndThrowIfExceptionLocated(IRODSErrorScanner.java:189) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.connection.IRODSErrorScanner.inspectAndThrowIfNeeded(IRODSErrorScanner.java:115) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.connection.IRODSMidLevelProtocol.processMessageInfoLessThanZero(IRODSMidLevelProtocol.java:1399) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.connection.IRODSMidLevelProtocol.readMessage(IRODSMidLevelProtocol.java:903) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.connection.IRODSMidLevelProtocol.readMessage(IRODSMidLevelProtocol.java:871) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.connection.IRODSMidLevelProtocol.irodsFunction(IRODSMidLevelProtocol.java:284) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.connection.IRODSMidLevelProtocol.irodsFunction(IRODSMidLevelProtocol.java:410) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.pub.IRODSFileSystemAOImpl.createFileInResource(IRODSFileSystemAOImpl.java:880) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.pub.IRODSFileSystemAOImpl.createFile(IRODSFileSystemAOImpl.java:782) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        at org.irods.jargon.core.pub.io.IRODSFileImpl.createNewFile(IRODSFileImpl.java:347) ~[nfsrods-1.0.0-SNAPSHOT-jar-with-dependencies.jar:?]
        ... 12 more

null pointer exception when nfsnobody (uid/gid 65534) not in passwd

If nfsnobody (or a user with uid/gid 65534) isn't in the id map, nfsrods throws an unhandled null pointer exception.

# docker logs -f nfsrods
2019-09-19 20:38:52.138 DEBUG Thread-1 [ServerMain] - main - Server config ==> { }
2019-09-19 20:38:52.162 DEBUG Thread-1 [ServerMain] - configureClientServerNegotiationPolicy - Policy = CS_NEG_REFUSE
Exception in thread "main" java.lang.ExceptionInInitializerError
       at org.irods.nfsrods.vfs.ServerMain.main(ServerMain.java:74)
Caused by: java.lang.NullPointerException
       at org.irods.nfsrods.vfs.IRODSIdMapper.<clinit>(IRODSIdMapper.java:42)
       ... 1 more

This happened because I didn't have nfs-utils installed, which defined the nfsnobody user on CentOS.

This exception should be handled and a meaningful error message should be written to the log.

Investigate PAM authentication

Either document why NFSRODS needs a shadow file or remove the dependency

I can think of no reason why NFSRODS should need a shadow file. Its purpose should either be clearly documented, or it should be stubbed out so it isn't required.

Investigate hasIOLayout member function

implement SSL to iRODS

add log_level configuration

default should be INFO for now, but probably WARNING for 1.0.0
allow override with environment variable (NFSRODS_LOG_LEVEL?)

introduce whitelist capability for nfs4_setfacl

This feature would introduce a whitelist defined via metadata on users and groups.

If a user is in the whitelist or in a group in the whitelist, they would be able to run nfs4_setfacl on the specified logical path or any collection or object 'below' it, regardless of their own permissions on that collection or object.

Use Case

Data within a double-blind study should not be visible by the curators of the system. However, in order to allow others to see the data, the curators need to be able to set permissions. This set of curators would be defined by a search in the catalog of users or groups with the following attached AVU:

a - irods::nfsrods::grant_nfs4_setfacl
v - <logical_path_prefix>
u -

Note

If this is implemented by calling 'change permissions' as the NFSRODS proxy_admin_account, then we need to confirm whether the curator's username appears in the server (and therefore in any audit logging).

Iterating over synchronized collection needs to be wrapped in a synchronized block

The following lines should be wrapped in a synchronized block to prevent unexpected results.

irods_client_nfsrods/irods-vfs-impl/src/main/java/org/irods/nfsrods/vfs/InodeToPathMapper.java

Lines 73 to 76 in bcfab2f

 Iterator<Long> it = availableInodeNumbers_.iterator(); 

 Long inodeNumber = it.next(); 

 it.remove(); 

 return inodeNumber;

Pull out well named constants

need to implement link(), used by WinSCP

When putting a file or overwriting a file via WinSCP...

From NFSRODS log:

2019-09-16 15:18:58.917 ERROR [NFSServerV41] - Unhandled exception:
java.lang.UnsupportedOperationException: Not supported
	at org.irods.nfsrods.vfs.IRODSVirtualFileSystem.link(IRODSVirtualFileSystem.java:789)
	at org.dcache.nfs.vfs.PseudoFs.link(PseudoFs.java:202)
	at org.dcache.nfs.v4.OperationLINK.process(OperationLINK.java:68)
	at org.dcache.nfs.v4.NFSServerV41.NFSPROC4_COMPOUND_4(NFSServerV41.java:204)

Might be possible via GenQuery to ask how many logical paths are pointing to the same physical path on the same resource... If > 1, then do something smart...

Swapping files corrupts multi-owner permissions silently

Known applications that do this are:

Vim
Nano

	Iterator<Long> it = availableInodeNumbers_.iterator();
	Long inodeNumber = it.next();
	it.remove();
	return inodeNumber;