elizagamedev / mujmap

Bridge for synchronizing email and tags between JMAP and notmuch

mujmap

mujmap is a tool to synchronize your notmuch database with a server supporting the JMAP mail protocol. Specifically, it downloads new messages and synchronizes notmuch tags with mailboxes and keywords both ways and can send emails via a sendmail-like interface. It is very similar to Lieer in terms of design and operation.

Disclaimer

mujmap is in quite an early state and comes with no warranty. I use it myself, it has been seeing steady adoption among other users, and I have taken caution to insert an abundance of paranoia where permanent changes are concerned. It is known to work on Linux and macOS with at least one webmail provider (Fastmail).

If you do decide to use mujmap, please look at the list of open issues first. If you are installing the latest Cargo release instead of the latest git revision, also consider looking at the issues in the changelog that have been found and resolved since the latest release.

Installation

Please first read the Disclaimer section.

Install with cargo:

cargo install mujmap

You may instead want to install from the latest main revision as bugs are regularly being fixed:

cargo install --git https://github.com/elizagamedev/mujmap

There is also an official Nix package. A home-manager module is underway.

Usage

mujmap can be the sole mail agent in your notmuch database or live alongside others, it can manage two or more independent JMAP accounts in the same database, and be used across different notmuch databases, all with different configurations.

In the directory that you want to use as the maildir for a specific mujmap instance, place a mujmap.toml file (example). This directory must be a subdirectory of the notmuch root directory. Then, invoke mujmap from that directory, or from another directory pointing to it with the -C option. Check mujmap --help for more options. Specific

Syncing

Use mujmap sync to synchronize your mail. TL;DR: mujmap downloads new mail files, merges changes locally, preferring local changes in the event of a conflict, and then pushes changes to the remote.

mujmap operates in roughly these steps:

1. mujmap gathers all metadata about emails that were created, potentially updated, or destroyed on the server since it was last run.

   JMAP does not tell us exactly what changes about a message, only that one of the very many properties of the JMAP Email object has changed. It's possible that nothing at all that we care about has changed. This is especially true if we're doing a "full sync", which can happen if we lose the state information from the last run or if such information expires server-side. In that case, we have to query everything from scratch and treat every single message as a "potential update".

2. mujmap downloads all new messages into a cache.

3. mujmap gathers a list of all messages which were updated in the database locally since it was last ran; we call these "locally updated" messages.

4. mujmap adds the new remote messages to the local notmuch database, then updates all local messages except the locally updated messages to reflect the remote state of the message.

   We skip updating the locally updated messages because again, there is no way to ask the JMAP server what changes were made; we can only retrieve the latest state of the tags as they exist on the server. We prefer preserving local tag changes over remote changes.

5. We push the locally updated messages to the remote.

   Unfortunately, the notmuch API also does not grant us any change history, so we are limited to looking at the latest state of the database entries as with JMAP. It seems possible that Xapian, the underlying database backend, does in fact support something like this, but it's not exposed by notmuch yet.

6. Record the first JMAP Email state we received and the next notmuch database revision in "mujmap.state.json" to be read next time mujmap is run back in step 1.

For more of an explanation about this already probably over-explained process, the slightly out-of-date and not completely-accurately-implemented-as-written DESIGN.org file goes into more detail.

Pushing without Pulling

Besides what is described above, you may also use mujmap push to local push changes without pulling in remote changes. This may be useful when invoking mujmap in pre/post notmuch hooks. You should only use push over sync when specifically necessary to reduce the number of redundant operations.

There is no mujmap pull, since pulling without pushing complicates the design tenet that the mujmap database is the single source of truth during a conflict. (The reason being that pulling without pushing changes the notmuch database, and now mujmap thinks those changes are in fact local revisions which must be pushed, potentially reverting changes made by a third party on the remote. If that's confusing to you, sorry, it's not easy to describe the problem succinctly.) It's possible to sort of work around this issue, but in almost every case I can think of, you might as well just sync instead.

Sending

Use mujmap send to send an email. This subcommand is designed to operate mostly like sendmail; i.e., it reads an RFC5322 mail file from stdin and sends it off into cyberspace. That said, this interface is still experimental.

The arguments -i, -oi, -f, and -F are all accepted but ignored for sendmail compatibility. The sender is always determined from the email message itself.

The recipients are specified in the same way as sendmail. They must either be specified at the end of the argument list, or mujmap can infer them from the message itself if you specify -t. If -t is specified, any recipient arguments at the end of the message are ignored, and mujmap will warn you.

Emacs configuration

(setq sendmail-program "mujmap"
      message-sendmail-extra-arguments '("-C" "/path/to/mujmap/maildir" "send"))

Quirks

• If you change any of the "tag" options in the config file after you already have a working setup, be sure to heed the warning in the example config file and follow the instructions!
• No matter how old the change, any messages changed in the local database in-between syncs will overwrite remote changes. This is due to an API limitation, described in more detail in the Behavior section.
• Duplicate messages may behave strangely. See #13.
• This software probably doesn't work on Windows. I have no evidence of this being the case, it's just a hunch. Please prove me wrong.

Migrating from IMAP+notmuch

Unfortunately, there is no straightforward way to migrate yet. The following is an (untested) method you can use, ONLY after you make a backup of your notmuch database, and ONLY after you have verified that mujmap works correctly for your account in an independent instance of a notmuch database (see the notmuch manpages for information on how to do this):

1. Ensure you're fully synchronized with the IMAP server.
2. Add a maildir for mujmap as a sibling of your already-existing maildirs. Configure it as you please, but don't invoke mujmap sync yet.
3. Create a file called mujmap.state.json in this directory alongside mujmap.toml with the following contents:

{"notmuch_revision":0}

4. Run mujmap --dry-run sync here. This will not actually make any changes to your maildir, but will allow you to verify your config and download email into a cache.
5. Run mujmap sync here to sync your mail for real. This will the downloaded email to the mujmap maildir and add them to your notmuch database. Because these messages should be duplicates of your existing messages, they will inherit the duplicates' tags, and then push them back to the server.
6. Remove your old IMAP maildirs and run notmuch new --no-hooks. If everything went smoothly, notmuch shouldn't mention any files being removed in its output.

Limitations

mujmap cannot and will never be able to:

• Modify message contents.
• Delete messages (other than tagging them as deleted or spam).

Troubleshooting

Status Code 401 (Unauthorized)

If you're using Fastmail (which, let's be honest, is practically a certainty at the time of writing), you may have recently encountered errors with username/password authentication (HTTP Basic Auth). This may be caused by Fastmail phasing out username/password-based authentication methods, as described in this blog post.

While this is objectively a good thing, and while it seems the intention was to roll this change out slowly, the API endpoint advertised by Fastmail DNS SRV records has almost immediately changed following the publication of this blog post, causing 401 errors in existing mujmap configurations. You have two options:

• Switch to bearer tokens by following the guide in the blog post. mujmap supports bearer tokens via the password_command config option in the latest main branch revision but not yet in a versioned release.
• Remove fqdn from your config if it's set, and add or change session_url to explicitly point to the old JMAP endpoint, located at https://api.fastmail.com/.well-known/jmap.

If your 401 errors are unrelated to the above situation, try the following steps:

• Ensure that your mail server supports either HTTP Basic Auth or Bearer token auth.
• Verify that you are using the correct username and password/bearer token. If you are using HTTP Basic Auth, Fastmail requires a special third-party password specifically for JMAP access.
• Verify that you are using a password_command which prints the correct password to stdout. If the password command fails, mujmap logs its stderr.
• If using Fastmail, check your login logs on the website for additional context.

Invalid cross-device link

This error will occur if your mail directory is stored on a different device than your cache directory. By default, mujmap's cache is stored in XDG_CONFIG_HOME on Linux/FreeBSD and ~/Library/Caches on macOS. You can change this location by setting config_dir in mujmap.toml.

The rationale for downloading messages into a cache instead of directly into the maildir is because mujmap is designed to be able to roll-back local state changes in the event of a catastrophic failure to the best of its ability, which includes not leaving mail files in the maildir which haven't been fully integrated into notmuch's database. As an alternative, mujmap could have depended on notmuch being configured to ignore in-progress downloads, but this is much more prone to user error.

mujmap's Issues

Add option to convert DOS newlines to Unix newlines

Unable to create large (300+) number of tags at one time

After running the following (it's ugly..I'm sure there's a better way but it worked) to create all the missing tags:

notmuch address --output=recipients --deduplicate=address --format=json to:@<my-domain> tag:inbox | jq -r '.[].address' | grep <my-domain> | xargs -L1 -I %s echo %s | cut -d@ -f1 | xargs -L1 -I %s notmuch tag -inbox +%s to:%s@<my-domain>

I am unable to synchronize those changes back up to Fastmail:

[ocelotsloth@ocelotsloth-xps mark]$ mujmap -v sync
Retrieving metadata... (7 possibly changed)
Applying changes to notmuch database... (0 new, 7 changed, 0 destroyed)
error: Could not sync mail: Could not create missing mailboxes for tags `[<<long list of 317 tag names>>]': Could not complete API request: https://api.fastmail.com/jmap/api/: status code 400

I'll take a closer look later today (after work), but my current guess is there's a limit to how many tags can be created in a single request and it may need to be paginated. I haven't dug into the mujmap code yet to see if it does that already though.

If I find it I'll try and get a patch written this week....I'd prefer that over manually adding all these tags in the web interface.

Clarify scary and confusing message about missing state

This message is confusingly worded and triggered when mujmap is added to an already-existing database. Either:

1. Clarify that this is okay if mujmap is new to the database
2. Detect this case and don't trigger it

Failure when maildir and cach dir are on separate devices

This can be fixed by introducing a cache_dir config option. A workaround is to set XDG_CONFIG_DIR to a location on the same device as the maildir.

Support for bearer tokens

Fastmail is adding support for authenticating with bearer tokens (personal and OAuth-negotiated). Basic auth (username+password) will be removed on 1 October.

https://jmap.topicbox.com/groups/fastmail-dev-beta/Tc47db6ee4fbb5451/updates-to-jmap-api-and-authentication
https://beta.fastmail.com/developer/integrating-with-fastmail/

Maildir flags are not set

I've been trying to use mujmap with neomutt today, which has some support for notmuch but its not entirely there.

It treats the maildir flags as the source of truth, and so when reading a mujmap-generated maildir thinks that all messages are unread and not flagged.

When a message is changed (eg user reads one) it triggers a flag update in notmuch. Because it didn't know the original flags, it sends the wrong set out to notmuch, and from there to the JMAP server.

(and because I accidentally touched the whole inbox in neomutt, all my inbox is unread. Fortunately I'm used to my email being chaos, hah).

(arguably neomutt is doing the wrong thing by using the maildir flags, and I might yet go back to it but I've been inside it before and its .. not pretty).

So! I thought to make mujmap set the file flags right. My first dumb attempt was to just plug in notmuch's own support for this:

diff --git a/src/local.rs b/src/local.rs
index a664a32..80288ba 100644
--- a/src/local.rs
+++ b/src/local.rs
@@ -308,6 +308,7 @@ impl Email {
         for tag in tags {
             self.message.add_tag(tag)?;
         }
+        self.message.tags_to_maildir_flags()?;
         self.message.thaw()?;
         Ok(())
     }

Alas, that really seems to assume that we're letting notmuch manage the maildir, but since mujmap is it just results in this madness:

$ ls -l Mf5091b2cd9e2f9c3fe4f4166.Gf5091b2cd9e2f9c3fe4f4166b1e7877cbfad1b5e*
-rw-r--r-- 1 robn robn 54315 May 15 12:52 Mf5091b2cd9e2f9c3fe4f4166.Gf5091b2cd9e2f9c3fe4f4166b1e7877cbfad1b5e
lrwxrwxrwx 1 robn robn   117 May 15 17:16 Mf5091b2cd9e2f9c3fe4f4166.Gf5091b2cd9e2f9c3fe4f4166b1e7877cbfad1b5e:2,F -> '/home/robn/.cache/mujmap/!home!robn!mail!grue!cur!Mf5091b2cd9e2f9c3fe4f4166.Gf5091b2cd9e2f9c3fe4f4166b1e7877cbfad1b5e'

That is, a dangling link to the message file in the cache, since moved into the maildir.

So maybe notmuch's own support for this can be made to work but I think it would require changes to how mujmap puts together its storage, which is rather beyond my knowledge (I actually had never used notmuch before today).

So thing mujmap would have to do is add flags when making the filename, but it would also need to grow support for renaming files when the flags change. I don't think that's too hard to do, but I definitely have no idea whether this is something mujmap should be doing. That's likely your call.

I will play around with this and with neomutt a bit more over the next week as time permits to try and figure out what the best solution is. Your thoughts welcome!

Ignore unrecognised mailbox roles

RFC8621 says mailbox roles MUST come from this list: https://www.iana.org/assignments/imap-mailbox-name-attributes/imap-mailbox-name-attributes.xhtml

As a safety mujmap should either ignore any non-null roles that aren't on the list or, maybe, only permit ones that we know we can deal with.

(servers might accidentally present non-standard roles and we don't want to assume that they're just normal mailboxes lest we make modifications that cause additional actions on the server).

Add `identities` command for external MUAs

When replying to a message with an MUA, said MUA should default to using the same sender email address/username as the original addressee. mujmap should implement an identities command which prints a machine readable (json) list of all available JMAP identities, so software can determine the correct identity to reply with.

Could not sync mail: 'InvalidProperties' error

After successfully syncing all mail, I opened in mutt, marked some as read, and then tried to sync again. I received this error:

❯ mujmap sync
Retrieving metadata... (13 possibly changed)
Applying changes to notmuch database... (0 new, 13 changed, 0 destroyed)
Applying changes to JMAP server... (104 changed)
error: Could not sync mail: Could not push changes to JMAP server: Failed to update messages on server: {Id("Mc4b0d8a389c8c1e2a682ab7d"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M1e05672cd7ca16a4cedcb1dc"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M7e03f950c8d35a83491044fe"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M154e4cf3ee4227b2ef210271"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M44b8aaf73d64735b3a9d01c2"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M621a6820aa5282f44ccc8093"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mdd0834c3479fa78a65398df1"): MethodResponseError{ kind: InvalidProperties, description: None }, Id("Md3feb2aaf36b17b95c861ffd"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M0bcf74807bc20cbb90c07448"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M3940ecadff220196d6a9fe79"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M1a6701f903cb1688ca97756e"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M8886a152fe1bc715922c7938"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mba1cf08956ac162b70c83a0b"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mcc986fc026c79dd88da4c2db"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mf8a8bd90898a0e15bc4b3e83"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Md4d23dceb24e26fbebadec18"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M3a04a76e915fdcd1109989d1"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mcd5999855b80040e94a48f82"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M71b13cbbacd075c53d68a84d"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mf01459c0ec3f28a13c02884f"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M0d43feb01e3ea324a241986f"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Ma2703c7b5e9be78f6c91179a"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M352f9146c051abc2b70ade1f"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M71a07a2d950dfe5b487a9c94"): MethodResponseError {kind: InvalidProperties, description: None }, Id("M138498064d7310aed497ca4a"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Me5985f33c7d6fab66e456485"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M9e582f6db29cd584a9f98ebd"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Ma4694c4b821955194ff3c97f"):MethodResponseError { kind: InvalidProperties, description: None }, Id("M28df96b1de9bd9319117dd60"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M6f2937e7b2d9b873d9d566d6"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Md6befaeefa32dde04ec81f5b"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mab481f02fafab2965d4eab53"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M77364a02804229bddce27358"): MethodResponseError { kind: InvalidProperties, description: None },Id("M66cfad3decd32a25ded42bbf"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M6d0772496175d7cdf81f0474"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M541a97b187815af1e4eb0a48"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M49e92578f14b35226c4542e8"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Ma6246b3fd0a837806c25ecf2"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mb1f0e147146444af3a44e999"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mc02ae577d51603ce62c67b4d"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M47c0b837c62159b8d0c9b30e"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M237e68f277350dc34ac31446"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Md0d8003d304d1ee21e5aaec4"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mba4a26f282f5876755c5c749"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M02eaa2ce574754c21d7e8973"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M8aa8b35494faf8ad0c2443d6"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M7213507179a20a249a82da00"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M8313a852361a5b39ab2e3640"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Md74e992c7b642cf4cf3c7bfc"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mda5fb3f530529a546bef0d9e"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M3f5b7b388067c13bc0b0294b"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M39e7e95396d897ebf142a6b8"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Me85df195e46cdd1c0c93bf7d"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Md0205dc3023febb4cb4ba14a"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M66a7091db0a19b0e2be5ae7b"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M8c868c337fb906292da03a95"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M5595ee92768119173bb64b04"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M70739bfd45293422a0133c76"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M2e89774b1797e89bd5c4b360"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M11f409a01ec22e986cf9e4e1"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M6a1895284cb94bc95301b190"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M3f3250412db24d8be5370c27"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M78e74cb75c0425ab4482add6"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Md5ab6ada12cbb5d0856638af"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M7949156a5a118ce587dc39bb"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M96b12bf35fd0b2ee57a07cc2"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M0f1b7baccd98b87c938fa481"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M616b81454510aead3e47bb1a"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M43ee1fe8f03a117a898f9167"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Md08c74a54e1924a73b918dcc"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mbbe8b0950a96e2c56c0409e6"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Me59ca90950607da36b393a45"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M4e6e9d9e8b3100edafc8b226"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mce1df9dd3e4bc1eb36e488e8"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mb3507685cf9c53613dd56465"): MethodResponseError { kind:InvalidProperties, description: None }, Id("M745f91ac08399592b0ed1867"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mb9209f3599899be342347ebd"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mba5d2742d1b7d837b4a5ee53"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M3f897a3b256cacd249613348"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mb1b83ee3c4c59ddc73c7baab"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M19451545cd4cdba32449b6b7"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M91d02b954e252c421ebe7dd5"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mb92198e34378729cedd764bb"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Me73c6234cce30d2d1247bbdd"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M1328deaec15337c907aa30fa"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mfa238ef28163cf170a2332c0"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M59da8fefccbc7451859f89e8"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M36990bc3159e7c36456a8ef1"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M1a04390239011d4d3a1f0bc3"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M00c0b47372aa7ce14966b2e1"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M0963e36497a5204df2d24221"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mebaebe8149a4b386f3f8af69"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M1db08d807bf7d451b3c92536"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Me830117c26f4c33776ee317b"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M5a11f2e38055a01a13bed285"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M0cebb7c63a9216344b671bf5"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M6806f9a8ebb10bb5bce00e6b"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M76eff48bbd2e02e3abc43471"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mefb037798a542a4fe44dc411"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M5a16981055db9d4024feaff0"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M3820b5975fa757c31f9e45bf"): MethodResponseError { kind: InvalidProperties, description: None }, Id("M7a11ccb42328c0a3f19b510d"): MethodResponseError { kind: InvalidProperties, description: None }, Id("Mb5d7c8c743c01b6a1e44a439"): MethodResponseError { kind: InvalidProperties, description: None}, Id("M45fb6a04f363d9f170f711f3"): MethodResponseError { kind: InvalidProperties, description: None }}

The JMAP docs say:

invalidProperties: (create; update). The record given is invalid in some way. For example:

• It contains properties that are invalid according to the type specification of this record type.
• It contains a property that may only be set by the server (e.g., “id”) and is different to the current value. Note, to allow clients to pass whole objects back, it is not an error to include a server-set property in an update as long as the value is identical to the current value on the server.
• There is a reference to another record (foreign key), and the given id does not correspond to a valid record.
  The SetError object SHOULD also have a property called properties of type String[] that lists all the properties that were invalid.

Individual methods MAY specify more specific errors for certain conditions that would otherwise result in an invalidProperties error. If the condition of one of these is met, it MUST be returned instead of the invalidProperties error.

This happens every time I sync now.

Aside from deleting these messages completely from disk, I'm not sure of another way to reset my state so that I can reproduce how I got into this state more consistently. Any suggestions on how I can help trace the root issue?

After sending mail, broken symlinks found in maildir leads to notmuch error

[vb@sita ~/mail/fastmail/cur]% ls -la ./M2ad4cf6ca935b78e20aa4b3a.G2ad4cf6ca935b78e20aa4b3a36e1f0c3f9670475:2*
lrwxrwxrwx 1 vb users 117  6 août  23:37 ./M2ad4cf6ca935b78e20aa4b3a.G2ad4cf6ca935b78e20aa4b3a36e1f0c3f9670475:2, -> '/home/vb/.cache/mujmap/!home!vb!mail!fastmail!cur!M2ad4cf6ca935b78e20aa4b3a.G2ad4cf6ca935b78e20aa4b3a36e1f0c3f9670475'
-rw-r--r-- 1 vb users 231  6 août  23:37 ./M2ad4cf6ca935b78e20aa4b3a.G2ad4cf6ca935b78e20aa4b3a36e1f0c3f9670475:2,S
[vb@sita ~/mail/fastmail/cur]% ls -al ~/.cache/mujmap
total 44
drwxr-xr-x  2 vb users 33542144  6 août  23:37 .
drwxr-xr-x 49 vb users     3452  1 août  19:39 ..

A link pointing into ~/.cache/mujmap exists while ~/.cache/mujmap is empty

Duplicate messages do not synchronize properly

mujmap currently assumes that one notmuch message corresponds to one and only
one remote message. I thought I had taken care of this assumption, but in
reality, this results in issues like in the description of 955bb9. I thought
was due to database corruption, but in reality was probably actually just
notmuch being blind to the duplicate message. I think what happened was...

• mujmap builds a map of all local IDs to local files. There was a Message
  associated with two JMAP IDs, but mujmap only considers one of them.
  (Email::from_message)
• mujmap gets a list of updates from the server, which includes an updated ID
  that mujmap accidentally skipped over. mujmap thinks there must be some kind
  of error since it doesn't "know" about this ID, so does a full sync.
  (sync.rs:263)
• mujmap re-downloads the email and tries to link it into the maildir. There's
  already a file there, so mujmap terminates with an error. (was sync.rs:409
  before the aforementioned commit)

With the "hotfix" in 955bb9, mujmap should continue to operate without fatal
errors, but will do more full syncs than necessary and will sometimes fail to
synchronize message tags for duplicate messages.

Can we ignore certain tags for syncing?

My notmuch database pulls email from my personal (via mujmap) as well as two work accounts (via lieer). When I try to sync changes with mujmap, it tries to create tags related to work on my Fastmail account and fails.

While debugging while it fails would be useful (I can paste a trace with -vvv if needed), ideally I wouldn't be syncing those tags to Fastmail at all. What do you think about adding a configuration option to ignore certain tags from sync altogether?

If this feature already exists or is possible another way, I do apologize for the duplicate issue!

Support distinct push and pull features

lieer and mbsync both have a nice feature where you can independently push local changes to the server as well as a pull remote changes, instead of doing a full sync.

What I often used and miss in mujmap is a workflow where I will do a full sync, then read, tag, and archive messages but then push up the changes to the server. push is usually a quicker operation and avoids any new messages getting pull down into my inbox until the next time that I sync. Yet I'm able to get the local changes pushed up to the server incase I later, for example, check my email on my phone.

Support the upcoming blob management RFC for integrity checking

https://datatracker.ietf.org/doc/draft-ietf-jmap-blob/

We should provide some integrity checking command to verify the integrity of blobs in the maildir. This gets a bit hairy given the Unix newline transformations.

Can't use mujmap with Stalwart mail-server

I'm pretty new the JMAP scene, so this may be a noob question.

I wanted to sync a JMAP server running on stalwart-mail but I don't really understand how is mujmap meant to connect to it.

The /.well-known/jmap path does not do anything, and the stalwart OAuth path (/.well-known/oauth-authorization-server) can't be parsed (Failed to read JSON: missing field capabilities at line 1 column 405).

I'm not really sure if I've mis-configured either the mail-server or mujmap, or is mujmap just compatible with fastmail?

Update documentation and sample config file for current Fastmail configuration

See the information at Updates to JMAP API and authentication on [Topicbox - Fastmail API](https://jmap.topicbox.com/groups/fastmail-api

In the config file, you need:-

session_url = "https://api.fastmail.com/.well-known/jmap"

The same document shows how to get an API key.

`invalidProperties` deserialization swallows `properties` field

This also likely affects lots of other error types. We should probably strongly type error variants that contain additional information like this.

Optimize all_mail_query

I currently have ~70k messages in my index which is... notmuch. 🥁

Each time I run mujmap it runs the format!("path:\"{}/**\"", relative_mail_dir.to_str().unwrap()) query, which takes 6+ seconds for me.

NOTMUCH_PROFILE=personal notmuch search path:"jmap/**"  6.19s user 2.07s system 98% cpu 8.355 total

With my basic understanding of how mujmap works, I don't see a straightforward way to use a different query or optimize this query so this can be a place to discuss.

Messages tagged 'sent' removes 'inbox' tag

I sometimes send emails to myself and it seems like mujmap applies the sent tag but also remove the inbox tag (and I'm guessing but haven't verified that it'll remove any other tag as well).

The outcome is that this message is only shown in Sent and doesn't show in my Inbox in neomutt, which is configured to use the inbox tag.

I've tested using sent = "" in mujmap.toml which does keep the inbox tag on the message but has the expected side-effect of not syncing the tag with the Sent folder on Fastmail.

Expected outcome is that a message could be both sent and have other tags.

Ignore read-only mailboxes

This is related to #19. The error seems to have been caused by a attempting to write to mailbox without write permissions (Scheduled). mujmap should altogether ignore read-only mailboxes like this.

Set default sender or list of default senders with Emacs

When using Emacs and notmuch-mua-new-mail function the sender always defaults to my
email@fastmail instead of
email@customdomain which is what I use and keep changing it to
Grabbing the sender emails address from jmap or Fastmail would be best
If not possible is their a way of to set this to a default value or list of values?

compose-mail command takes the value of user-mail-address for sender address while notmuch-mua-new-mail does not

Potential related issues #33 #35 #30

As a side note is their a way to set compose-mail command to notmuch-mua-new-mail
for other Emacs apps that use mail commands? or other commands to set to use Notmuch?

weird behavior when run from a symlinked maildir

I am trying to setup mujmap on my own before checkjing out the homemanager module.
I have my maildir folder as a symlink towards /mnt/ext/maildir and I think this confuses mujmap

✖2 ❯ mujmap -C ~/maildir sync                       
[2022-05-16T22:22:47Z WARN  mujmap::sync] Could not read mujmore state file `/home/teto/maildir/mujmap.state.json': No such file or directory (os error 2)
error: Could not sync mail: Could not open remote session: Could not open session at https://jmap.fastmail.com:443/.well-known/jmap: https://jmap.fastmail.com/session: status code 401 (redirected from https://jmap.fastmail.com/.well-known/jmap)
~/.config took 4s 
✖1 ❯ mujmap -C ~/maildir sync
~/.config 
✖130 ❯ cd ~/maildir
~/maildir 
➜ cp mujmap.toml .notmuch/
~/maildir 
➜ mujmap -C .notmuch sync 
[2022-05-16T22:25:25Z WARN  mujmap::sync] Could not read mujmore state file `.notmuch/mujmap.state.json': No such file or directory (os error 2)
error: Could not sync mail: Could not open local database: Given maildir path `/mnt/ext/maildir/.notmuch' is not a subdirectory of the notmuch root `/home/teto/maildir'
➜ mujmap -C . sync       
[2022-05-16T22:25:57Z WARN  mujmap::sync] Could not read mujmore state file `./mujmap.state.json': No such file or directory (os error 2)
error: Could not sync mail: Could not open local database: Given maildir path `/mnt/ext/maildir' is not a subdirectory of the notmuch root `/home/teto/maildir'
~/maildir 
➜ ls -l maildir 
lrwxrwxrwx 1 teto users 16 2021-10-02  maildir -> /mnt/ext/maildir

as seen previously, if I run it from another directory with -C ~/maildir it seems to work.

mujmap -C maildir sync -v
error: Could not sync mail: Could not open remote session: Could not open session at https://jmap.fastmail.com:443/.well-known/jmap: https://jmap.fastmail.com/session: status code 401 (redirected from https://jmap.fastmail.com/.well-known/jmap)

My credentials look ok (btw I can access my mail over jmap with meli https://meli.delivery/#) so I wondered if the newline in my password could be an issue (I run password_command = "pass-show perso/fastmail_mc" which returns my password plus a newline apparently). Is there a way to just dump my password in the config file to make sure that's not the cause for the 401.

Daemon mode using JMAP push notifications

https://datatracker.ietf.org/doc/html/rfc8620#section-7

add home-manager to flake

I've written a home-manager module in my config and I ponder about adding it straight to home-manager or as part of the flake.
Considering the limited number of mujmap users, I thiunk it would make more sense to have it here. What do you think ?

Multiple account IDs on the same server

JMAP supports multiple account IDs, but mujmap only considers the "main" account. Multiple accounts are described in the spec as a way to potentially share read-only information via subscriptions across accounts, but I am unaware of its usage in practice.

Support partial maildirs?

I would like to use mujmap as part of a larger maildir/notmuch database, aggregating mail from multiple accounts at different providers. Is this use case supported? E.g., lieer (and mbsync) can be configured to sync just a subdirectory of the maildir, instead of the whole notmuch database.

mujmap sync => Error encountered in the status line: timed out reading response

My initial sync worked well, I now tried to change tags and reran mujmap sync and I hit:
Error encountered in the status line: timed out reading response
I wonder if its due to the size of the request, I've got sthg like 11000mails and I get in the log a seemingly long request with lines of

"keywords/$answered": Null, "keywords/$forwarded": Null, "keywords/$draft": Null, "keywords/$phishing": Null, "keywords/$seen": Bool(true), "keywords/$flagged": Null, "mailboxIds": Object({"86a2d9f5-c010-45a0-82ac-67c2e55da6d3": Bool(true), "a0b0232f-b611-4eaf-b185-6ed047a97c96": Bool(true), "e33b24e7-4faf-46e9-83fd-d3c97ce02868": Bool(true)})}, Id("Maa48c356ce4142cfb28d5c97"): {"keywords/$forwarded": Null, "keywords/$answered": Null, "keywords/$phishing": Null, "mailboxIds": Object({"24e4de2e-a4e2-4cd9-822e-08efbf5efab9": Bool(true), "a0b0232f-b611-4eaf-b185-6ed047a97c96": Bool(true)}), "keywords/$flagged": Null, "keywords/$seen": Bool(true), "

This is the end of the log

24e4de2e-a4e2-4cd9-822e-08efbf5efab9": Bool(true), "a0b0232f-b611-4eaf-b185-6ed047a97c96": Bool(true)}), "keywords/$forwarded": Null, "keywords/$seen": Bool(true), "keywords/$draft": Null, "keywords/$answered": Null, "keywords/$phishing": Null, "keywords/$flagged": Null}, Id("M672a30cce2ed0a77ea870958"): {"keywords/$phishing": Null, "mailboxIds": Object({"24e4de2e-a4e2-4cd9-822e-08efbf5efab9": Bool(true), "a0b0232f-b611-4eaf-b185-6ed047a97c96": Bool(true)}), "keywords/$seen": Bool(true), "keywords/$flagged": Null, "keywords/$answered": Null, "keywords/$forwarded": Null, "keywords/$draft": Null}}
[2022-07-02T20:11:32Z DEBUG ureq::stream] connecting to jmap.fastmail.com:443 at 66.111.4.141:443
[2022-07-02T20:11:32Z DEBUG rustls::client::hs] Resuming session
[2022-07-02T20:11:32Z DEBUG rustls::client::hs] Using ciphersuite Tls13(Tls13CipherSuite { suite: TLS13_AES_256_GCM_SHA384, bulk: Aes256Gcm })
[2022-07-02T20:11:32Z DEBUG rustls::client::tls13] Resuming using PSK
[2022-07-02T20:11:32Z DEBUG rustls::client::tls13] TLS1.3 encrypted extensions: [ServerNameAck]
[2022-07-02T20:11:32Z DEBUG rustls::client::hs] ALPN protocol is None
[2022-07-02T20:11:32Z DEBUG ureq::stream] created stream: TcpStream { addr: 192.168.1.13:36722, peer: 66.111.4.141:443, fd: 9 }
[2022-07-02T20:11:32Z DEBUG ureq::unit] sending request POST https://jmap.fastmail.com/api/
[2022-07-02T20:11:32Z DEBUG ureq::unit] writing prelude: POST /api/ HTTP/1.1
    Host: jmap.fastmail.com
    User-Agent: ureq/2.4.0
    Accept: */*
    Authorization: ***
    Content-Type: application/json
    accept-encoding: gzip
    Content-Length: 1288766
[2022-07-02T20:11:33Z DEBUG rustls::client::tls13] Ticket saved
[2022-07-02T20:11:37Z DEBUG ureq::stream] dropping stream: TcpStream { addr: 192.168.1.13:36722, peer: 66.111.4.141:443, fd: 9 }
error: Could not sync mail: Could not push changes to JMAP server: Could not complete API request: https://jmap.fastmail.com/api/: Network Error: Network Error: Error encountered in a header: timed out reading response

the funny thing is that the tag I changed was successfully sent to fastmail as I can now see it in the UI, which is fantastic , it means mujmap has been solving my single biggest issue in my setup so thanks for that. Let me know how I can help (testing etc)

Synchronize sieve scripts

Still in draft: https://datatracker.ietf.org/doc/draft-ietf-jmap-sieve/

Automatically create an archive mailbox?

mujmap requires an archive mailbox, which is used to synchronize mail without any other tags in notmuch. If this mailbox does not exist, the synchronization process fails.

Would it be possible and in scope for mujmap to create this mailbox itself if it cannot find one? If implemented, this should likely be gated either behind a command line flag or a config setting.

Adding unread tag to message does not change message on server to unread

The changes to sync unread states in neomutt is working nicely! There's one issue I've found: If I mark a message as unread locally, either through neomutt's <toggle-new> or update the notmuch tag to unread, then sync, the change isn't reflected on the server.

notmuch search tag:unread properly shows the message, so it's not a neomutt to notmuch issue.

This is the the trace log for Email/changes. Does this mean mujmap is not attempting to push any changes to the server?

[2022-05-30T17:19:23Z TRACE mujmap::remote] Post response: {"sessionState":"cyrus-581349;p-72a2823c15","methodResponses":[["Email/changes",{"destroyed":[],"updated":[],"newState":"612383","hasMoreChanges":false,"oldState":"612383","created":[],"accountId":"u41f441db"},"0"]]}

Alternate authentication methods

mujmap currently only supports HTTP Basic Auth. As of the time of writing, this is the only authentication method supported for JMAP clients by Fastmail.

Support for notmuch profiles

I think mostly this means getting the maildir path from notmuch config item database.mail_root and then using that for path: queries, sync destination, etc. Which might mean changing/removing the cur/new/tmp setup, but maybe not if clients are really expecting it to look like that.

I'm not sure if its better or worse to explicitly set a profile name in mujmap config and pass that into Database::open_with_config, or just to require NOTMUCH_PROFILE to be set and the magic just happens. The latter is more notmuchy, the former is much more pleasant. But support first I think.

Timeouts not handled well

I've been hacking on mujmap today on a very flaky network connection, which is leading to the program pretty regularly exiting with:

[2022-05-15T06:59:20Z DEBUG ureq::unit] response 200 to GET https://www.fastmailusercontent.com/publicjmap/download/ud0004100/G1c3c459181941951be1590e6fa0261b918a8919c/G1c3c459181941951be1590e6fa0261b918a8919c?type=text%2Fplain
[2022-05-15T06:59:20Z DEBUG ureq::stream] dropping stream: TcpStream { addr: 192.168.1.105:33020, peer: 66.111.4.88:443, fd: 11 }
[2022-05-15T06:59:20Z DEBUG ureq::stream] dropping stream: TcpStream { addr: 192.168.1.105:33022, peer: 66.111.4.88:443, fd: 14 }
[2022-05-15T06:59:20Z DEBUG ureq::stream] dropping stream: TcpStream { addr: 192.168.1.105:54542, peer: 66.111.4.87:443, fd: 7 }
[2022-05-15T06:59:20Z DEBUG ureq::stream] dropping stream: TcpStream { addr: 192.168.1.105:54534, peer: 66.111.4.87:443, fd: 8 }
error: Could not sync mail: Could not save email to cache: Could not create mail file `/home/robn/.cache/mujmap/!home!robn!mail!grue!cur!in_progress_download.4': timed out reading response

It continues from where it left off when restarted, so its mostly just an inconvenience.

I thought to catch timeouts and retry (much like remote::get_reader does) but I can't quite figure out where this error is actually coming from.

I'll have another run at it once I'm finished with with the current thing I'm doing, but maybe you will just say "oh, its just this" and flip a single bit and save me some time!

Failure on symlinked maildir

See #11

auto-tags like `attachment` inadvertently removed by mujmap

This is caused by the call to remove_all_tags(). mujmap should not remove
or otherwise touch the following tags:

• attachment
• signed
• encrypted

Additionally, mujmap should warn the user if any JMAP mailbox exists with one of
these names and refuse to synchronize with them.

Send mail via JMAP

Support sending emails via Fastmail's masked email addresses

This functionality is currently unavailable to third parties, but once it is, mujmap should support it.

Add progress indicator for notmuch updates

There is currently a progress indicator for downloads. However the updating into the notmuch database can be at least as long (on my initial import of nearly 600K emails the download took a couple of hours, the update into notmuch took around 14 hours).

This stage all appears to be within https://github.com/elizagamedev/mujmap/blob/main/src/sync.rs#L441-L536

Is there the capability to add a progress bar in this section?

[Even after the initial import, the notmuch db update appears to take a significant time - multiple minutes for changes of 10s of emails]

Cut new release including Bearer support

@elizagamedev Would you be willing to cut a new release, say v0.2.1, that includes 56c3824? That commit is necessary to support Fastmail today. The latest release, v0.2.0, always fails with a 401.

Improve duplicate message handling

notmuch combines all duplicates it detects into a single virtual message entry. If notmuch detects such duplicates among JMAP emails, the "source of truth" for which JMAP email entry maps to the local notmuch message is non-deterministic. When pushing changes, all remote duplicate messages are updated to match.

Maybe the best course of action would be not to store duplicate messages in the first place. Fastmail has a duplicate message detector which can help find and delete these problematic messages. That said, mujmap could come up with a scheme to handle these cases more robustly.

Compile issue on mac OS

Very excited about this project! Thanks for the work.

I'm on mac OS version 10.15.7 and I've tried a few different install methods, cargo install mujmap, cargo install --git https://github.com/elizagamedev/mujmap, and cloning then cargo build and each get the following error:

   Compiling mujmap v0.1.1 (/Users/nick/Projects/opt/mujmap)
error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "-arch" "x86_64" "/Users/nick/Projects/opt/mujmap/target/x86_64-apple-darwin/debug/deps/mujmap-b527ed80e46a5b62.11ysm0zvaufyx36t.rcgu.o" "/Users/nick/Projects/opt/mujmap/target/x86_64-apple-darwin/debug/deps/mujmap-b527ed80e46a5b62.12i1i1o4anqdi2px.rcgu.o" "/Users/nick/Projects/opt/mujmap/target/x86_64-apple-darwin/debug/deps/mujmap-b527ed80e46a5b62.12nofqtye063tz74.rcgu.o" "/Users/nick/Projects/opt/mujmap/target/x86_64-apple-darwin/debug/deps/mujmap-b527ed80e46a5b62.13sc98w46l98n4ap.rcgu.o" "/Users/nick/Projects/opt/mujmap/target/x86_64-apple-darwin/debug/deps/mujmap-b527ed80e46a5b62.14pi6ct8r8is04ca.rcgu.o" "/Users/nick/Projects/opt/mujmap/target/x86_64-apple-darwin/debug/deps/mujmap-b527ed80e46a5b62.15mhs55uyeyc9d23.rcgu.o"
......
"/Users/nick/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcore-701d43bb5146c80b.rlib" "/Users/nick/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-7e89f88408f280c2.rlib" "-framework" "Security" "-lnotmuch" "-liconv" "-lSystem" "-lresolv" "-lc" "-lm" "-liconv" "-L" "/Users/nick/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "-o" "/Users/nick/Projects/opt/mujmap/target/x86_64-apple-darwin/debug/deps/mujmap-b527ed80e46a5b62" "-Wl,-dead_strip" "-nodefaultlibs"
  = note: Undefined symbols for architecture x86_64:
            "_notmuch_database_open_with_config", referenced from:
                notmuch::database::Database::open_with_config::h808d9d521a989158 in mujmap-b527ed80e46a5b62.1u5gq9nhfi7k23dq.rcgu.o
          ld: symbol(s) not found for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

I'll keep looking for a resolution and will update with findings.

send mail: error: Found argument '-o' which wasn't expected, or isn't valid in this context

error: Found argument '-o' which wasn't expected, or isn't valid in this context

	If you tried to supply `-o` as a value rather than a flag, use `-- -o`

USAGE:
    mujmap [OPTIONS] <SUBCOMMAND>

For more information try --help

Now my emacs seems to want -o ? :(

help description for `-C` suboptimal

I find the description confusing as in I first tried mujmap -C ~/maildir/fastmail/mujmap.toml

    -C, --path <PATH>
            Path to config file.
            
            Defaults to the current working directory.

I would rephrase to

    -C, --config-dir <PATH>
            Directory where to find config file "mujmap.toml".
            
            Defaults to the current working directory.

XDG base directory spec compliance

mujmap is based on lieer Lieer in terms of configuration layout, and has the following requirements:

• The config file (mujmap.toml) is in a directory which is at the root of or a subdirectory of a notmuch-managed mail directory.
• The state file (mujmap.state.json) and lock file (mujmap.lock) generated by mujmap are placed in this same directory.

According to the XDG base directory spec, the correct locations for these files are:

mujmap.toml
: $XDG_CONFIG_HOME/mujmap/

mujmap.state.json
: $XDG_DATA_HOME/mujmap/

mujmap.lock
: $XDG_RUNTIME_DIR/mujmap/

However, I’m not sold on the usefulness of changing mujmap to comply with this spec, as the following problems arise:

• Filesystems shared between two operating systems on the same device would have to maintain independent configurations of the same maildir. You could argue that given mujmap’s purpose, you could just synchronize instead, but if you have gigabytes and gigabytes of mail then this could be wasteful.

• Similar to above, the current system “just works” on networked filesystems.

• This might be a matter of opinion, but the configuration process might become more confusing. mujmap would need to support multiple maildir configurations in the same config file, and these configuration options would need to point to specific mail directories. The -C option would need to be replaced with something like --name.

  Perhaps I specifically am biased to think that the way lieer does it is easier because I am used to using it already for my work email. But I do think there is something somewhat intuitive to the concept of “this mail directory contains the mujmap config/state, therefore the mail files here are owned by mujmap”, and it also communicates to the user “you can have separate mujmap setups, and they are all independent by design” in a clear way.

This issue exists both as a space for me to explain why mujmap is the way it is and also a place for people to try and convince me that I’m wrong.