zen-fs / dom

DOM backends for ZenFS

ZenFS DOM Backends

ZenFS backends for DOM APIs. DOM APIs are only available natively in browsers.

Important

Please read the ZenFS core documentation!

Backends

• WebStorage: Stores files in a Storage object, like localStorage and sessionStorage.
• IndexedDB: Stores files into an IndexedDB object database.
• WebAccess: Store files using the File System Access API.

For more information, see the API documentation.

Usage

Note

The examples are written in ESM. If you are using CJS, you can require the package. If running in a browser you can add a script tag to your HTML pointing to the browser.min.js and use ZenFS DOM via the global ZenFS_DOM object.

import { configure, fs } from '@zenfs/core';
import { WebStorage } from '@zenfs/dom';

await configure({ backend: WebStorage, storage: localStorage });

if (!fs.existsSync('/test.txt')) {
	fs.writeFileSync('/test.txt', 'This will persist across reloads!');
}

const contents = fs.readFileSync('/test.txt', 'utf-8');
console.log(contents);

dom's Issues

In the `browser.min.js` version, an error occurs.

Using this for init.

window.ZenFS.configure({
      backend: window.ZenFS_DOM.IndexedDB,
      storeName: "VirtualFS",
  }).then(() => console.log(window.ZenFS));

Getting this error:

Uncaught (in promise) TypeError: (destructured parameter) is undefined
    _initialize AsyncStore.ts:162
    ready filesystem.ts:341
    ready AsyncStore.ts:141
    xs config.ts:61
    Pg config.ts:96
    run test.js:9
    async* test.js:14
AsyncStore.ts:162:29

From my investigation, _initialize seems to run twice and on the second run, it fails.

XML backend: Store Filesystem inside the DOM

Imagine if we have something similar [to] Storage but instead of storing it on Local Storage we would store the filesystem inside the DOM.
This would make it possible for a "quine", that is, a program that produces itself as output.

This way we would be able to save the .html file with ctrl+s since the filesystem is inside the DOM and not in the browser(the .html file has the filesystem).

I can do a Pull Request but is this project still going on? Is there an active fork of it?

~ @ilse-langnar

jvilk/BrowserFS#331

Can't read root directory with `StorageStore`

With localStorage Storage Backend,
build on top of SyncStore,
we can't read root dir.

This because unsymetric serialization/deserialization
of Uint8Array data into string.

• serialisation SyncStore.js:L40: data.toString() // "1,2,3,4..."
• deserialisation SyncStore.js:L31: encode(data); // TextEncoder asciiToUint8

stack:

    h ApiError.js:99
    FileError ApiError.js:65
    ENOTDIR ApiError.js:80
    getDirListing SyncStore.js:364
    readdirSync SyncStore.js:251
    BrowserFS filesystem.js:503
    BrowserFS filesystem.js:9
    E filesystem.js:5
    readdir filesystem.js:502

Note: strings are stored as utf16, that's about 2 bytes per chars

Simple fix: (storage size = data length * 7) (about 2.57+1 ascii char by data byte)

• serialisation SyncStore.js:L40: data.toString() // "1,2,3,4..."
• deserialisation SyncStore.js:L31: Uint8Array.from(data.split(','), Number)

Base64 fix: (storage size = data length * 2.7) (about 1.33 ascii char by data byte)

• serialisation: window.btoa(Array.from(data, x=>String.fromCharCode(x)).join(''))
• deserialization: Uint8Array.from(window.atob(data), x=>x.charCodeAt(0))

Utf16 fix: (storage size = data length * 1) (need to handle odd data lenght)

• itoc = (x)=>String.fromCodePoint(x>>11!==27?x:x|0x10000)
• ctoi = (x)=>x.codePointAt(0) // Uint16Array will discard high bit
• serialisation: Array.from(new Uint16Array(data.buffer), itoc).join('')
• deserialisation: new Uint8Array(Uint16Array.from(data, ctoi).buffer)

With utf16, to handle odd data length:

• on serialization: (add last odd byte, and count of extra data bytes; be carefull with LE/BE)
  • ...new Uint16Array(data.buffer, 0, length/2)... // discard odd last byte
  • if (length%2)
  • then append String.fromCodePoint(data[length-1] * 0x101, 0x303)
  • else append String.fromCodePoint(0x202)
• on deserilization: (remove extra bytes)
  • len = data.length, extra = data[len -1]
  • return new Uint8Array(data, 0, len - extra)

`readFile` on directory should throw meaningful errors

I tested reading file with WebAccess backend which was actually a directory and it gave me the following error:

(in promise) TypeError: Cannot read properties of undefined (reading 'close')
at _readFile (promises.js:471:20)

For example, calling readdir on a file gives the following error: ENOTDIR: File is not a directory

Remote IndexedDB storage proposal (like RemoteStorageJS or SQLite.JS)

@ptsource:

It would be nice to have the abillity to set up remote IndexedDB databases, this can be done with RemoteStorageJS, also PHP IndexedBD servers can be set up. Just a enhancement suggestion. Another option can be SQLite.JS Keep up the great work. Cheers

@jvilk:

I don't see the benefit of a backend for SQLite.js, as that just mutates an in-memory SQL database (what are the advantages over the in-memory file system?).

RemoteStorageJS could be a cool file system backend addition; I'd welcome PRs for that, so long as we can set up unit tests for it somehow.

@ptsource:

Well since SQLite.js it a Emscripten fork of SQLite the purpose was to achieve total compatibility with any SQL management tool besides storing the data hosted where the script is runned by simply adding SQLite.JS to the application, RemoteStorageJS is indeed a cool feature. Keep up the great work. Cheers

jvilk/BrowserFS#211

class heritage L is not an object or null

When loading browser.min.js
It result in an Uncaught TypeError: class heritage L is not an object or null
at browser.min.js:3:14236 / SyncStore.js:83:7

It seems that class is defined after first reference.
So BrowserFS_DOM is not defined and lib is unusable.

Workaround:

With npm --save-dev esbuild-plugin-external-global
Update scripts/build.mjs as following:

import {externalGlobalPlugin} from "esbuild-plugin-external-global";

function externalGlobals(pkg, global, mods) {
  const entries = mods.map(m => [`${pkg}/${m}.js`, global]);
  return Object.fromEntries(entries);
}

const core_externals = externalGlobals('@browserfs/core', 'BrowserFS', [
  'ApiError', 'FileIndex', 'backends/AsyncStore', 'backends/SyncStore',
  'cred', 'file', 'filesystem', 'inode', 'mutex', 'stats', 'utils'
]);

const ctx = await context({
	entryPoints: ['src/index.ts'],
	globalName: 'BrowserFS_DOM',
        /* ... */
        plugins: [
          externalGlobalPlugin(externalGlobals),
          { name: 'watcher', setup(build) {
              /*...*/ 
          }}
	],
});