wfraser / fuse-mt

FUSE-MT: a multithreaded wrapper on top of FUSE for Rust

FUSE-MT

Documentation

This code is a wrapper on top of the Rust FUSE crate with the following additions:

• Dispatch system calls on multiple threads, so that e.g. I/O doesn't block directory listing.
• Translate inodes into paths, to simplify filesystem implementation.

The fuser crate provides a minimal, low-level access to the FUSE kernel API, whereas this crate is more high-level, like the FUSE C API.

It includes a sample filesystem that uses the crate to pass all system calls through to another filesystem at any arbitrary path.

This is a work-in-progress. Bug reports, pull requests, and other feedback are welcome!

Some random notes on the implementation:

• The trait that filesystems will implement is called FilesystemMT, and instead of the FUSE crate's convention of having methods return void and including a "reply" parameter, the methods return their values. This feels more idiomatic to me. They also take &Path arguments instead of inode numbers.
• Currently, only the following calls are dispatched to other threads:
  • read
  • write
  • flush
  • fsync
• Other calls run synchronously on the main thread because either it is expected that they will complete quickly and/or they require mutating internal state of the InodeTranslator and I want to avoid needing locking in there.
• The inode/path translation is always done on the main thread.
• It might be a good idea to limit the number of concurrent read and write operations in flight. I'm not sure yet how many outstanding read/write requests FUSE will issue though, so it might be a non-issue.

fuse-mt's Issues

Feature: Use `OpenOptions` type for `open`

While idiomitazing things up, I thought maybe some of the argument types should change as well. Every flags-argument which comes as an int would look nicer as a struct (or a bitfield, when that enters the language).

For some of them, this means adding dedicated types, but specifically for open we have OpenOptions which is used by the stdlib for passing flags to open.

The implementation is of course straight-forward, I wanted to ask what you think about this beforehand.

Make a new release

The last release 0.5.1 was in 16/08/2020. Since then there've been a few commits that are pretty useful (e.g. switching to fuser and updating time format). Also I'm working on str4d/rage#324 and it requires a new release of fuse-mt.

Occasional "forget entry" messages on unmount

My FUSE program occasionally prints messages to the terminal upon being unmounted. Here's an example:

forget entry InodeTableEntry { path: Some("/LambdaComplex/out"), lookups: 2, generation: 0 }

Based on a quick search of this repo for "forget entry", I assume that this is the code that's causing it. Perhaps that println! should be a debug!?

Filesystem mutability

The fuse::Filesystem trait methods all take &mut self, but the fuse_mt::FilesystemMT traits take &self. What's the reasoning behind this? This makes it rather difficult to store a filesystem in memory, which is what I'm trying to do.

Sockets now implemented in fuse-rust

Greetings, regarding your comment in the source example/src/passthrough.rs:

This is committed to rust-fuse but not yet released. See: zargony/fuse-rs#79

The mentioned PR was already merged on 2017-01-09 and the following release 0.3.0 was made on 2017-01-25 and thus should include socket support.

utimens_macos sets values for which there is no way to query

FilesystemMTincludes utimens_macos which sets chgtime and bkuptime but since those are not included in FileAttr how can this API ever work? There doesn't seem to be any way to get to those attributes.

Remove ino field from FileAttr struct?

Several functions (at least getattr and lookup) require returning a FileAttr. Part of this FileAttr is the inode number as a u64, and therefore it seems as if I (as the crate's user) am supposed to keep track of the inode numbers. However, the number I provide in this reply is not what is actually shown by stat(1); I assume that this crate's built-in path -> inode mapping is overriding the inode numbers that I supply.

Because of this, I propose that the ino field be removed from the fuse_mt::FileAttr struct, as it apparently serves no purpose.

Please tell me if I got any of this wrong!

Feature: merge getattr and lookup

Under the standard fuse implementation, getattr and lookup have two different roles - getattr is for an a file for which we already have a handle (inode); whereas lookup is for looking up by name a file, for which we don't have a handle, but resides inside a directory for which we do.
Under the path abstraction in fuse-mt, the distinction no longer exists - both methods look up a file by name, except one does so with the full path, the other with the full path of the parent dir and the name of the file.
Therefore, I suggest these methods be merged, probably to getattr, and that internally (i.e. when implementing Filesystem) both getattr and lookup will use the target fs getattr.
I can draft a pull-request.

Feature: store handler objects instead of fh

I think a pretty common pattern for both the open and the opendir methods would be to store some handler object, returning an fh as a key to access it later.

Assuming this, the library could request the higher-level FS to return an object from open and opendir, handler the storing of it themselves, handing to all calls which currently get fh (read, readir, etc.), and disposing of it on release and releasedir (without exposing them to the higher-level FS; if the implementer of that wants, they can add logic in drop).

I'm not sure if this belongs here; while this is not limiting in any sense (one can always choose the types of these objects to be u64 to retain current behaviour or () if they're implementing stateless access; this does require some thought about release, though), it does incur some inevitable overhead (at least until Rust lets one write monomorphisations specializations, where a special implementation could be written for u64 and (). Edit: rust-lang/rust#31844). I'm currently writing this logic as a separate crate, over fuse-mt - not yet in cargo, but you can see the code here: https://github.com/amosonn/rust_fuse_fl.

However, adding this to the lower level library will definitely have some benefits. Aside from removing another level of indirection for the unaffected methods (not all that important, I assume the compiler can optimize it away anyway), currently I have to use a RwLock over the entire HashMap to have interior mutability, and release calls actually block until all reads, etc. have completed (which is even worse, given that currently release isn't run in a separate thread). If this were part of fuse-mt, we could use a HashMap over Arcs instead, and release will simply remove the copy residing in the HashMap, so the drop will be called in the last thread using that file.

What do you think?

Generation numbers should ensure inode uniqueness across remounts

The title says it all. As per FUSE docs, (generation, inode) combination must not reoccur even after application restart, because there are things, like NFS, that rely on it being unique. Hence initializing generation numbers to zero like it happens now is bad. I suggest using something like a nanosecond-precision timestamp of current time at the moment of InodeTable creation, or a random value.

Feature: Support iterators in readdir

Given the usage of the current Vec<DirectoryEntry> the user provides, one could go with the more general Box<IntoIterator<DirectoryEntry>>, or perhaps a generic for the whole FilesystemMT trait, which is a bit uglier but probably better for runtime. This won't hurt users who use the current API; in fact, if FilesystemMT becomes generic, the default type could be Vec<DirectoryEntry>, and in the other case, the users only need to wrap their vector up; but it also allows lazy checking of the directory contents, as is enabled by the low-level API, without the hassel of tracking the position.

Why does the API have file handles and Paths in several calls?

I've been trialing a fuse filesystem implementation to see if it's workable to have a local filesystem that overflows into a NAS so you can have your multi-TB photo/video collection always available on your laptop. So far I've just implemented a basic naive in-RAM filesystem to try out the API:

https://github.com/pedrocr/syncer/blob/1110097ecac53dca8133b0fd8d883c1ea597dba4/src/main.rs

This seems to work fine but I have just ignored the file handles in the API completely and gone with the Paths at all time. Is this way of working completely broken? Having file handles is problematic for my design as I'm planning on having the backend be content addressable instead of having some kind of sequential inode number.

FilesystemMT::init default implementation

Default implementation of fuse::Filesystem::init method in fuse-rs lib does nothing. But FilesystemMT::init by default cases error.

https://github.com/zargony/fuse-rs/blob/39fde4a5c47ce370d228ac190f950bd835db7f47/src/lib.rs#L93-L95

fuse-mt/src/types.rs

Lines 137 to 139 in 55cda3c

	fn init(&self, _req: RequestInfo) -> ResultEmpty {
	Err(0)
	}

fuse-mt/src/fusemt.rs

Lines 97 to 100 in 55cda3c

	fn init(&mut self, req: &fuse::Request<'_>) -> Result<(), libc::c_int> {
	debug!("init");
	self.target.init(req.info())
	}

Why it may be important? Because if you miss the implementation of this method the fuse-rs will not work properly. And it's hard to find the issue because it doesn't panic and seemingly works. It just always provides a error ls: cannot access '': Connection refused.

If it's considered that method is important it could not have a default implementation at all so it would be required to implement it.

Type discrepancy for permissions

The chmod function passes the mode as a u32, but the perm field of FileAttr is a u16

Consider providing inode numbers or inode-associated user data to handlers

It is often necessary to associate some data with files. Currently, data association is only possible via some HashMap<PathBuf, UserData> field in FS struct. Long strings with long common prefixes, which pathnames usually are, aren't the best thing for quick lookups, though. I see the following two alternatives (which would be most useful provided existence of lookup/forget methods from #20, which then also should accept some of the parameters described below).

For one, handlers could receive inode numbers for parameters in addition to Paths. Then user can at least employ HashMap<Inode, UserData>, which would have faster lookup. If user further relies on the current inode reuse behavior of fuse-mt, they might even go for Vec<UserData> and index into it with inode numbers, and it also would be more parallelizable than HashMap.

For another alternative, user may be spared the effort to keep inode-associated data altogether. InodeTable might be made generic on and keep inside a user-defined value for every inode, which it would provide to handlers for use/modification. Current InodeTable would be equivalent to InodeTable<()>, which can be made the default with something like FuseMT<T, U=()> { inodes: InodeTable<U> }.

It would be most flexible, if those alternatives were combined, and instead of user data parameters, handlers were provided with inode numbers and direct access to InodeTable for lookup/modification of user data by inode number. Then user could freely access the data for all inodes, and not just for inode that a handler was called with.

In fact, if InodeTable were exposed to user, then it would also be possible to omit Path parameters to handlers altogether and provide user with just the inode numbers and facility to lookup paths by inode. It would cut some overhead for when user doesn't even need a Path to do some action. A fully-sugared backward-compatible interface with Path parameters probably could be provided on top of that.

It would be nice if serde support was added

I'm trying to use serde to make it easier to serialize a file metadata struct to disk. It would be nice if the FileType enum implemented Serialize/Deserialize. This can probably be done behind a feature flag so that not everyone needs to drag in serde even if they're not using it.

Requiring 'static for FilesystemMT seems too strict

I'm trying to keep a reference to part of my FilesystemMT implementation to be able to have concurrent threads managing state (e.g., sync to disk every X seconds). The signature of FuseMT::new() requires 'static which makes this harder to do. Would it make sense to use crossbeam::scope or something similar to allow having both the fuse_mt and implementation threads run properly scoped?

Consider returning Future instead of Result from Filesystem methods

Currently all methods in the FilesystemMT return a Result indicating the response to the method call.

While it is true that all requests are currently processed concurrently using threading, it may be more ergonomic to make everything run on a reactor (from tokio-rs core crate) and change all methods to return Futures instead of Results.

In my case, I am trying to use async IO to perform network requests (I'm writing a network filesystem) and threading on the library level is not useful. Utilising futures-rs means that implementors can use whichever mechanism is more appropriate to them (whether that be disk IO on a thread pool or network requests on a reactor).

I'd be happy to prototype this in a fork and send a PR if you are open to it!

read() seems to have multithreading issues

I'm getting inconsistent results from read() if I enable multithreading (doesn't happen with just a single thread) and if I do a read() implementation that returns at most 4096 bytes (my block size) even if the program requests more. All my code is safe rust and so far all my filesystem is in memory so there are no external side effects to cause this. Reading read(2) it seems it should be legal to return less bytes than requested and it does indeed work fine without threading.

fio fails to benchmark [passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0xe75a000: Invalid argument (os error 22)]

The fio (used in most popular disk benchmarker like KDiskMark) fails with the following errors:

  fuse-mt/example on  master [?] via 🦀 v1.64.0-nightly >>> fio --loops=2 --size=512m --filename=./mountpoint/fiotest.tmp --stonewall --ioengine=libaio --
direct=1 \
                                                                    --name=Seqread --bs=1m --rw=read \
                                                                    --name=4kQD32read --bs=4k --iodepth=32 --rw=randread
Seqread: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=1
4kQD32read: (g=1): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
fio-3.31
Starting 2 processes
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=0, buflen=1048576
fio: pid=24904, err=22/file:io_u.c:1846, func=io_u error, error=Invalid argument
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=136785920, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=97517568, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=322498560, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=21110784, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=243191808, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=331292672, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=373399552, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=152776704, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=154963968, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=45359104, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=207089664, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=33755136, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=36495360, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=197152768, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=524095488, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=105418752, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=216604672, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=123449344, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=353476608, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=156409856, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=100343808, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=429608960, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=298127360, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=95707136, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=233742336, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=287895552, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=5505024, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=82231296, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=453476352, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=242589696, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=406925312, buflen=4096
fio: io_u error on file ./mountpoint/fiotest.tmp: Invalid argument: read offset=306835456, buflen=4096
fio: pid=24905, err=22/file:io_u.c:1846, func=io_u error, error=Invalid argument

Seqread: (groupid=0, jobs=1): err=22 (file:io_u.c:1846, func=io_u error, error=Invalid argument): pid=24904: Sat Sep  3 16:12:56 2022
  cpu          : usr=0.00%, sys=0.00%, ctx=2, majf=0, minf=47
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=50.0%, 4=50.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=1,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1
4kQD32read: (groupid=1, jobs=1): err=22 (file:io_u.c:1846, func=io_u error, error=Invalid argument): pid=24905: Sat Sep  3 16:12:56 2022
  cpu          : usr=0.00%, sys=0.00%, ctx=33, majf=0, minf=46
  IO depths    : 1=3.1%, 2=6.2%, 4=12.5%, 8=25.0%, 16=50.0%, 32=3.1%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=66.7%, 8=0.0%, 16=0.0%, 32=33.3%, 64=0.0%, >=64=0.0%
     issued rwts: total=32,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):

Run status group 1 (all jobs):

Logs generated from the example passthrough fs:

  fuse-mt/example on  master [?] via 🦀 v1.64.0-nightly took 3m57s >>> cargo run -- mounted mountpoint
# ...
fuser::request: DEBUG: FUSE( 48) ino 0x0000000000000001 STATFS
fuse_mt::fusemt: DEBUG: statfs: "/"
passthrufs::passthrough: DEBUG: statfs: "/"
fuser::request: DEBUG: FUSE( 50) ino 0x0000000000000001 LOOKUP name "fiotest.tmp"
fuse_mt::fusemt: DEBUG: lookup: "/", "fiotest.tmp"
passthrufs::passthrough: DEBUG: getattr: "/fiotest.tmp"
passthrufs::passthrough: DEBUG: stat_real: "mounted/fiotest.tmp"
fuse_mt::inode_table: DEBUG: adding 2 -> "/fiotest.tmp" with 0 lookups
fuse_mt::inode_table: DEBUG: lookups on 2 -> Some("/fiotest.tmp") now 1
fuser::request: DEBUG: FUSE( 52) ino 0x0000000000000001 GETATTR
fuse_mt::fusemt: DEBUG: getattr: "/"
passthrufs::passthrough: DEBUG: getattr: "/"
passthrufs::passthrough: DEBUG: stat_real: "mounted/"
fuser::request: DEBUG: FUSE( 54) ino 0x0000000000000002 OPEN flags 0xc000
fuse_mt::fusemt: DEBUG: open: "/fiotest.tmp"
passthrufs::passthrough: DEBUG: open: "/fiotest.tmp" flags=0xc000
fuser::request: DEBUG: FUSE( 56) ino 0x0000000000000002 READ fh FileHandle(5), offset 0, size 131072
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x20000 @ 0x0
fuse_mt::fusemt: DEBUG: initializing threadpool with 1 threads
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x20000 @ 0x0
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x20000 @ 0x0: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 58) ino 0x0000000000000002 FLUSH fh FileHandle(5), lock owner LockOwner(15463802717853242365)
fuse_mt::fusemt: DEBUG: flush: "/fiotest.tmp"
passthrufs::passthrough: DEBUG: flush: "/fiotest.tmp"
fuser::request: DEBUG: FUSE( 60) ino 0x0000000000000002 RELEASE fh FileHandle(5), flags 0xc000, flush false, lock owner Some(LockOwner(0))
fuse_mt::fusemt: DEBUG: release: "/fiotest.tmp"
passthrufs::passthrough: DEBUG: release: "/fiotest.tmp"
fuser::request: DEBUG: FUSE( 62) ino 0x0000000000000002 GETATTR
fuse_mt::fusemt: DEBUG: getattr: "/fiotest.tmp"
passthrufs::passthrough: DEBUG: getattr: "/fiotest.tmp"
passthrufs::passthrough: DEBUG: stat_real: "mounted/fiotest.tmp"
fuser::request: DEBUG: FUSE( 64) ino 0x0000000000000002 OPEN flags 0xc000
fuse_mt::fusemt: DEBUG: open: "/fiotest.tmp"
passthrufs::passthrough: DEBUG: open: "/fiotest.tmp" flags=0xc000
fuser::request: DEBUG: FUSE( 66) ino 0x0000000000000002 READ fh FileHandle(5), offset 136785920, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x8273000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x8273000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x8273000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 68) ino 0x0000000000000002 READ fh FileHandle(5), offset 97517568, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x5d00000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x5d00000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x5d00000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 70) ino 0x0000000000000002 READ fh FileHandle(5), offset 322498560, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1338f000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1338f000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x1338f000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 72) ino 0x0000000000000002 READ fh FileHandle(5), offset 21110784, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1422000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1422000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x1422000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 74) ino 0x0000000000000002 READ fh FileHandle(5), offset 243191808, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0xe7ed000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0xe7ed000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0xe7ed000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 76) ino 0x0000000000000002 READ fh FileHandle(5), offset 331292672, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x13bf2000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x13bf2000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x13bf2000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 78) ino 0x0000000000000002 READ fh FileHandle(5), offset 373399552, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1641a000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1641a000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x1641a000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 80) ino 0x0000000000000002 READ fh FileHandle(5), offset 152776704, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x91b3000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x91b3000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x91b3000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 82) ino 0x0000000000000002 READ fh FileHandle(5), offset 154963968, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x93c9000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x93c9000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x93c9000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 84) ino 0x0000000000000002 READ fh FileHandle(5), offset 45359104, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x2b42000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x2b42000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x2b42000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 86) ino 0x0000000000000002 READ fh FileHandle(5), offset 207089664, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0xc57f000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0xc57f000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0xc57f000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 88) ino 0x0000000000000002 READ fh FileHandle(5), offset 33755136, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x2031000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x2031000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x2031000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 90) ino 0x0000000000000002 READ fh FileHandle(5), offset 36495360, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x22ce000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x22ce000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x22ce000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 92) ino 0x0000000000000002 READ fh FileHandle(5), offset 197152768, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0xbc05000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0xbc05000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0xbc05000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 94) ino 0x0000000000000002 READ fh FileHandle(5), offset 524095488, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1f3d1000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1f3d1000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x1f3d1000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 96) ino 0x0000000000000002 READ fh FileHandle(5), offset 105418752, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x6489000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x6489000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x6489000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE( 98) ino 0x0000000000000002 READ fh FileHandle(5), offset 216604672, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0xce92000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0xce92000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0xce92000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(100) ino 0x0000000000000002 READ fh FileHandle(5), offset 123449344, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x75bb000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x75bb000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x75bb000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(102) ino 0x0000000000000002 READ fh FileHandle(5), offset 353476608, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1511a000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1511a000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x1511a000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(104) ino 0x0000000000000002 READ fh FileHandle(5), offset 156409856, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x952a000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x952a000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x952a000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(106) ino 0x0000000000000002 READ fh FileHandle(5), offset 100343808, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x5fb2000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x5fb2000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x5fb2000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(108) ino 0x0000000000000002 READ fh FileHandle(5), offset 429608960, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x199b5000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x199b5000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x199b5000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(110) ino 0x0000000000000002 READ fh FileHandle(5), offset 298127360, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x11c51000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x11c51000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x11c51000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(112) ino 0x0000000000000002 READ fh FileHandle(5), offset 95707136, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x5b46000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x5b46000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x5b46000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(114) ino 0x0000000000000002 READ fh FileHandle(5), offset 233742336, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0xdeea000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0xdeea000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0xdeea000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(116) ino 0x0000000000000002 READ fh FileHandle(5), offset 287895552, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1128f000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1128f000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x1128f000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(118) ino 0x0000000000000002 READ fh FileHandle(5), offset 5505024, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x540000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x540000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x540000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(120) ino 0x0000000000000002 READ fh FileHandle(5), offset 82231296, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x4e6c000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x4e6c000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x4e6c000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(122) ino 0x0000000000000002 READ fh FileHandle(5), offset 453476352, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1b078000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1b078000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x1b078000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(124) ino 0x0000000000000002 READ fh FileHandle(5), offset 242589696, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0xe75a000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0xe75a000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0xe75a000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(126) ino 0x0000000000000002 READ fh FileHandle(5), offset 406925312, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x18413000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x18413000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x18413000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(128) ino 0x0000000000000002 READ fh FileHandle(5), offset 306835456, size 4096
fuse_mt::fusemt: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1249f000
passthrufs::passthrough: DEBUG: read: "/fiotest.tmp" 0x1000 @ 0x1249f000
passthrufs::passthrough: ERROR: read "/fiotest.tmp", 0x1000 @ 0x1249f000: Invalid argument (os error 22)
fuser::request: DEBUG: FUSE(130) ino 0x0000000000000002 FLUSH fh FileHandle(5), lock owner LockOwner(6080363487507376540)
fuse_mt::fusemt: DEBUG: flush: "/fiotest.tmp"
passthrufs::passthrough: DEBUG: flush: "/fiotest.tmp"
fuser::request: DEBUG: FUSE(132) ino 0x0000000000000002 RELEASE fh FileHandle(5), flags 0xc000, flush false, lock owner Some(LockOwner(0))
fuse_mt::fusemt: DEBUG: release: "/fiotest.tmp"
passthrufs::passthrough: DEBUG: release: "/fiotest.tmp"
fuser::request: DEBUG: FUSE(134) ino 0x0000000000000001 LOOKUP name ".git"
fuse_mt::fusemt: DEBUG: lookup: "/", ".git"
passthrufs::passthrough: DEBUG: getattr: "/.git"
passthrufs::passthrough: DEBUG: stat_real: "mounted/.git"
passthrufs::passthrough: ERROR: lstat("/.git"): No such file or directory (os error 2)
fuser::request: DEBUG: FUSE(136) ino 0x0000000000000001 LOOKUP name ".git"
fuse_mt::fusemt: DEBUG: lookup: "/", ".git"
passthrufs::passthrough: DEBUG: getattr: "/.git"
passthrufs::passthrough: DEBUG: stat_real: "mounted/.git"
passthrufs::passthrough: ERROR: lstat("/.git"): No such file or directory (os error 2)
fuser::request: DEBUG: FUSE(138) ino 0x0000000000000001 OPENDIR flags 0x18800
fuse_mt::fusemt: DEBUG: opendir: "/"
passthrufs::passthrough: DEBUG: opendir: "mounted/" (flags = 0o304000)
fuser::request: DEBUG: FUSE(140) ino 0x0000000000000001 READDIR fh FileHandle(2), offset 0, size 4096
fuse_mt::fusemt: DEBUG: readdir: "/" @ 0
fuse_mt::fusemt: DEBUG: entries not yet fetched; requesting with fh 94456831506736
passthrufs::passthrough: DEBUG: readdir: "/"
fuse_mt::fusemt: DEBUG: directory has 4 entries
fuse_mt::fusemt: DEBUG: readdir: adding entry #0, "hi"
fuse_mt::fusemt: DEBUG: readdir: adding entry #1, "."
fuse_mt::fusemt: DEBUG: readdir: adding entry #2, ".."
fuse_mt::fusemt: DEBUG: readdir: adding entry #3, "fiotest.tmp"
fuser::request: DEBUG: FUSE(142) ino 0x0000000000000001 LOOKUP name ".gitignore"
fuse_mt::fusemt: DEBUG: lookup: "/", ".gitignore"
passthrufs::passthrough: DEBUG: getattr: "/.gitignore"
passthrufs::passthrough: DEBUG: stat_real: "mounted/.gitignore"
passthrufs::passthrough: ERROR: lstat("/.gitignore"): No such file or directory (os error 2)
fuser::request: DEBUG: FUSE(144) ino 0x0000000000000001 RELEASEDIR fh FileHandle(2), flags 0x18800, flush false, lock owner Some(LockOwner(0))
fuse_mt::fusemt: DEBUG: releasedir: "/"
passthrufs::passthrough: DEBUG: releasedir: "/"
fuser::request: DEBUG: FUSE(146) ino 0x0000000000000001 STATFS
fuse_mt::fusemt: DEBUG: statfs: "/"
passthrufs::passthrough: DEBUG: statfs: "/"
fuser::request: DEBUG: FUSE(148) ino 0x0000000000000001 STATFS
fuse_mt::fusemt: DEBUG: statfs: "/"
passthrufs::passthrough: DEBUG: statfs: "/"
fuser::request: DEBUG: FUSE(150) ino 0x0000000000000001 STATFS
fuse_mt::fusemt: DEBUG: statfs: "/"
passthrufs::passthrough: DEBUG: statfs: "/"
fuser::request: DEBUG: FUSE(152) ino 0x0000000000000001 STATFS
fuse_mt::fusemt: DEBUG: statfs: "/"
passthrufs::passthrough: DEBUG: statfs: "/"

Not too sure what is missing, haven't take a good look at the codebase. Although stuffs like bindfs based on C High-Level API works as expected.

Cloning might be bad

A few of FuseMT's methods (like read and write) call clone() on the underlying filesystem. Doesn't this result in making another copy of all of the filesystem's data? That seems pretty wasteful for an in-memory filesystem. Is there any way around having to do this?

fuse_mt::FuseMT is underdocumented.

fuse_mt::mount requries Filesystem trait, but docs suggest implementing FilesystemMT trait.

FuseMT seems to be the missing link, but it is not documented whatsoever.

I also expected fuse_mt to have a wrapper around mount and spawn_mount that does fuse_mt::FuseMT::new automatically.

Completeness

How close is fuse-mt to being ready for prime-time? I see that there's already a working example, and the API seems like a straightforward layer over the rust-fuse crate. Is the path translation part already stable enough to be used over rust-fuse?

Hardlinks seem to be broken

I'm trying to add hardlinks by implementing link() but that doesn't even ever get called. strace tells me linkat() is getting called and receiving ENOENT but I don't see that reaching my code at all. All I see are two calls to getattr() for the destination of the link which doesn't yet exist because link() hasn't yet been called. Symlinks work fine though.

Feature request: Simple-threaded mode.

Maybe one can opt-out threading, but gain ability to have !Send + !Sync filesystems?

It can be a Cargo feature or some...

Workaround: set threads to 1 and unsafe impl Send MyFs {} unsafe impl Sync for MyFs.

Missing derives

Many of the structs defined in FuseMT are missing basic derives, such as Debug and Clone (and when applicable, Copy). I added them, see PR #9.

Allow access to the inode/parent id

I am planning to implement a filesystem, that has a tag based folder structure.

So e.g. a folder has the id 001 and another folder has the id 002.
In folder 001 is a file 011 and 021 etc.

If I want to resolve all folders/files in folder 001 I need to be able to execute a request like:
SELECT * FROM objects WHERE parent_id = '001'

So I need the information of the parent id / object in the readdir method. Would it be possible to give access to the parents inode or store some custom id's, so that I know what the requested object actually is?

regards,
Dominik

Support for Fsnotify

See this.

Reintroduce lookup/forget calls as notifications

It would be nice to have lookup(&Path)/forget(&Path) methods getting called when an inode is looked up for the first time/is removed after it's forgotten for the last time. The methods are not intended to do or return anything that Fuse would rely on, like actual lookup. The default implementations would be empty and hence optimized out.

It is useful to keep track of what kernel actually needs at the moment. For a FS that I'm writing I have to associate some resources with some files and directories, and it would lend itself well to allocate/free these resources on actual demand.

Move dispatch of requests to seperate thread

I am having some performance problems that appear to be caused by my slowness to handle getattr() requests. I am reformatting data on the fly, so determining the size of a reformatted file is a relatively expensive operation. (NFS seems to be aggressive with sending getattr() also )

But what also seems to be happening in my scenario is that the request dispatch stalls when I handling getattr() - so I get starvation when handling read() requests. ( I.e. no request will be handed to the thread pool while the main thread is busy processing its own requests )

I am think that a better solution would be to have a dedicated dispatch thread, 1 main thread for handling requests that should be handled synchronously, and use the theeadpool for requests that can be handled asynchronously. This way the dispatch won't block,