redprl / bantorra Goto Github PK

Unix.chdir and Sys.getcwd are currently used to normalize file paths, but the stateful chdir becomes a major reason why bantorra is not thread-safe. It's possible to normalize file paths without chdir, but the code would be too ugly. Thankfully, OCaml 4.13 will bring in a real Unix.realpath that does the job. I believe with some locking the entire library can be made thread-safe. For these reasons we should switch to OCaml 4.13 as soon as it is officially released.

• git
• archive
• public index

Depending on #70.

This is needed for #62.

Options to turn off git auto-fetch

Bantorra appears to be using an outdated version of Asai, with Logger instead of Reporter.

Currently every library needs to be specified in the anchor files. Perhaps it is useful to pre-mount the standard library.

For whatever reasons, the json-data-encoding was moved to https://gitlab.com/nomadic-labs/data-encoding

There was actually a API for this, but is hidden now. This is useful for initializing routers that wish to create some directories at the root of the library.

YAML is beautiful but JSON is much more well-defined and well-implemented, especially in the area of implicit types. JSON has its own issues, for example duplicate keys, but overall it's easier to find quality encoders and decoders.

Currently, the library lazily loads libraries. This is bad and should be changed.

1. There should be eager initialization that loads everything.
2. However, once everything is loaded, the checking should done in the offline mode as in #73.

It is wrong to load libraries lazily because (1) there's no way to detect whether some library is bad and (2) there's no way to solve version constraints without loading everything.

We don't really need a package manager now, but it's fun to create trolling PRs (for good reasons). Needs #52.

Probably LRU as the policy.

This avoids specifying the anchor file name multiple times.

This means the path resolver would have to check the absence of the anchor files.

idea: let's just have different copies!

The module UserConfig cannot serialize library versions correctly because the underlying OCaml binding of libyaml is serializing the string "1.0" as the floating point 1., "null" as null (unquoted), etc. This means users have to manually edit the configuration files.

The bug report to the upstream: avsm/ocaml-yaml#39

• Exec
• File
  • XDG

The code is there, but the interface needs some Thoughts.

This is the spec of waypoints that point to local library roots. A library reference in this scheme is a string library.name. Waypoints are stored in YAML files of a name specified by the client. Maybe it's waypoint. This special file name is referred to as ${waypoint} below.

Direct waypoints

These are the basic waypoints pointing to the roots of libraries. The resolver will keep moving up in the file system until some matching waypoint is found. If the root directory is reached and there's still no matching waypoint, the resolution fails.

- "basis":
    at: ["src", "basis"]
- "frontend":
    at: ["src", "frontend"]

Indirect waypoints

Indirect waypoints delegate the resolution to other waypoints, which can also be indirect. The resolution is thus recursive. The library name must be explicitly listed in the next waypoint file. That is, it will not move up in the file system when no applicable waypoint is found. Otherwise, one can easily have unintentional infinite looping due to a typo.

- "bantorra":
    "next": ["mylib", "bantorra"] # the resolver will look up the library in "mylib/bantorra/${waypoint}"
- "cool.basis":
    "next": ["mylib", "cool"] # the resolver will look up the library in "mylib/cool/${waypoint}"
    "rename": "basis" # optional field for the new name to look up

Mixed waypoints

Of course, you can have both direct and indirect waypoints in a single file.

- "cool.basis":
    "next": ["mylib", "cool"]
    "rename": "basis"
- "bantorra":
    "next": ["mylib", "bantorra"]
- "builtin":
    "at": ["runtime", "builtin"]

Name uniqueness

The YAML format demands unique library names within a waypoint file, so there should not be any ambiguity. The order of waypoints in a file should not matter.

A version system should at least define a join-semilattice over all versions.

It would be useful to introduce a mechanism for

1. Private units only visible to a library, and
2. Putting all code under a subdirectory (such as src/)

Like rewrite in go modules

New plan: drop camlzip and LMDB.

• Use ezgzip + ezjson and reimplement the compression later, if necessary.
• Drop LMDB and write something inspired by bap-cache, except that GC is done by fuzzy LRU (like relatime). Unfortunately we cannot use bap-cache directly because it will pull in a prohibitively large dependency tree.
• This library still intends to support using JSON everywhere.