Unexpected behavior: inbox emptied about lieer

[gauteh]

Hi Karl, this sees weird, answers below: Karl F writes on mars 29, 2018 18:20:

# Unexpected behavior: inbox emptied I've just switched to gmailieer, and followed the instructions on the webpage, and did an initial download, and there are several things that seem odd to me, (I've tried finding a IRC-channel or FAQ to ask these questions).

You could try #notmuch. Also, there's a few notes at the end of the README. Otherwise, issues are your best bet.

1. After running `gmi sync` from the folder it has emptied my inbox. Prior to the sync, I had +2000 conversation threads in the web-interface of gmail, after the sync I have 7. The rest seems to reside in `.mail/gmail/mail/cur` (about 20k of files).

First, gmailieer can not delete any e-mail (on the remote): * https://github.com/gauteh/gmailieer/blob/master/lieer/remote.py#L12 * https://developers.google.com/gmail/api/auth/scopes So your e-mail has to be somewhere. Have you checked Trash and All Mail in GMail? Note that if your messages somehow ended up in Trash or Spam they will be deleted after 30 days.

2. Most importantly for me: Is it possible to push the messages back up to web-gmail some how? And why/how can this happen?

gmi does not push messages, only label changes to the remote. If you create files in the gmi repository you will get an exception.

3. When sending a mail, and using mbsync and notmuch, I'd have a fcc of the mail to my "sent" folder. Should I do something similar for gmailieer, or it should not be fcc-ed anywhere?

GMail automatically adds a copy to the Sent folder, you will get your copy back on the next pull.

4. Looking at the email in notmuch (emacs), not all messages that I've trashed, using web-interface of gmail, are marked with the `trash` tag, only some are.

Check the caveats in the readme: you have to use lower case 'trash', anything else will be ignored (because of GMail limitations). Also; only one of 'spam', 'inbox', 'trash' may be applied to a message.

Note, I ran gmi sync several times, to properly propagate tag changes I had done to the downloaded mail in notmuch.el, and on one occasion, gmailieer produced thousands of rows like the following: ``` update: gid does not match any file name of message, probably a draft, skipping: 12e247f736244708 update: gid does not match any file name of message, probably a draft, skipping: 12e2479a1d3a6ec0 update: gid does not match any file name of message, probably a draft, skipping: 12e2475f9eba8394 ```

There seems to be some general funkyness going on: * Do you have several gmail accounts that could have gotten mixed up? * Are you using other synchronization methods to fetch your email, e.g.: OfflineIMAP / mbsync? This could cause a lot of confusion. * Did you start with a fresh account in notmuch? i.e: no e-mail from that account in your notmuch database on the first gmi pull?

[impaktor]

Hi!

Thanks for reply. I've now been experimenting with this further and I
understand better what is going on, and I think all my 4 points above are OK
now. Just user error on my part (thus closing).

Have you checked Trash and All Mail

Right you are, they're in "All Mail", and the error was mine, as I think at
some point during my experimentation I must have removed the "inbox"-tag in notmuch and
didn't realize it's a 1:1 with gmail's "INBOX"-folder; which I now realize is
just a tag, and of course syncing tags is exactly what gmailieer does.

Specifically, I am/was hazy on how labels work in gmail, and never even noticed
there's an "All mail" in gmail.

• Do you have several gmail accounts that could have gotten mixed up?

No, however, there are other parallel non-gmail accounts that I sync with mbsync, e.g:

~/.mail/gmail
~/.mail/work
~/.mail/other

...but problem solved now.

• Did you start with a fresh account in notmuch? i.e: no e-mail from
  that account in your notmuch database on the first gmi pull?

Nope. In fact, I have a more general, gmail, question, you might
have pointer on:

1. Sent mail tag?

In notmuch the mails sent from gmail have the "sent" tag, but for me some have
a "[Gmail]/Sent Mail"-tag. I believe this comes from when I initially tried to
get gmail and mbsync to play nice, and in that process I mapped gmail's IMAP
folder "[Gmail]/Sent Mail" (linked to the "Sent Mail"-folder/tag in the web
interface) into a "sent/"-folder on my local machine. That worked in my
Wanderlust setup.
Now, I suspect the following:

1a. This isn't normal, all the "[Gmail]/Sent Mail"-tags should be "sent" (in notmuch).

1b. Thus, in the web version of gmail, I can remove the "[Gmail]/Sent Mail" label/tag.

1c. But the README seems to indicate it's not possible to push the "sent" tag?

2. Unread/inbox tags on sent mail

Right now, I think I need to sync twice, to get sent email downloaded, "inbox", "unread"-tags
removed, and then pushed back up again and shown properly in both notmuch and
gmail.com, and not shown as "unread" on either side.

I have in my notmuch config:

[new]
tags=inbox;unread;new;
ignore=*.json;

and a post-new hook:

notmuch tag -inbox tag:sent and tag:inbox and tag:new
notmuch tag -unread tag:sent and tag:unread and tag:new

# gmail arbitrarily marks mails as "important",
# note: not the same as "flagged". 
notmuch tag -important tag:important

# gmailieer needs the "new" tag initially, can no be removed: 
notmuch tag -new tag:new

When I used the fcc dir, notmuch ran this code:

(setq notmuch-fcc-dirs '(("work.com" . "\"work/Sent Items\" -unread -inbox")
                         ;;("gmail.com" . "\"gmail/Sent Mail\" "-unread -inbox") ;; using gmailieer
                         ("other" . "\"other/Sent\" -unread -inbox")))

Which was handy, as it removed the "unread" and "inbox" tags right when
sending the mail.

3. trash vs. delete?

I notice the default tag for thrashing messages in notmuch, is "deleted". Is
it possible to map "deleted" to be alias for "trash"?

Closing

If you have any thought on what I've written, I'd be interested in hearing them, but I'm closing this now, as the main "issue" has been clarified now. Thanks!

[gauteh]

Karl F writes on april 1, 2018 20:25:

Hi! Thanks for reply. I've now been experimenting with this further and I understand better what is going on, and I think all my 4 points above are OK now. Just user error on my part (thus closing). > Have you checked Trash and All Mail Right you are, they're in "All Mail", and the error was mine, as I think at some point during my experimentation I must have removed the "inbox"-tag in notmuch and didn't realize it's a 1:1 with gmail's "INBOX"-folder; which I now realize is just a tag, and of course syncing tags is exactly what gmailieer does. Specifically, I am/was hazy on how labels work in gmail, and never even noticed there's an "All mail" in gmail. > * Do you have several gmail accounts that could have gotten mixed up? No, however, there are other parallel non-gmail accounts that I sync with mbsync, e.g: ``` ~/.mail/gmail ~/.mail/work ~/.mail/other ``` ...but problem solved now. > * Did you start with a fresh account in notmuch? i.e: no e-mail from > that account in your notmuch database on the first gmi pull? Nope. In fact, I have a more general, gmail, question, you might have pointer on: ## 1. Sent mail tag? ## In notmuch the mails sent from gmail have the "sent" tag, but for me some have a "[Gmail]/Sent Mail"-tag. I believe this comes from when I initially tried to get gmail and mbsync to play nice, and in that process I mapped gmail's IMAP folder "[Gmail]/Sent Mail" (linked to the "Sent Mail"-folder/tag in the web interface) into a "sent/"-folder on my local machine. That worked in my [Wanderlust](https://github.com/wanderlust/wanderlust/) setup. Now, I suspect the following: 1a. This isn't normal, all the "[Gmail]/Sent Mail"-tags should be "sent" (in notmuch).

This must be a left-over from your IMAP setup. The long label is therefore (probably) treated as a new label and re-created in the GMail side by gmailieer.

1b. Thus, in the web version of gmail, I can remove the "[Gmail]/Sent Mail" label/tag.

As far as I can see, yes. Then gmi pull afterwards.

1c. But the README seems to indicate it's not possible to push the "sent" tag?

Yes, the sent tag will be applied by GMail to messages you send through that account. It will therefore be applied to the message on the next pull. This is the only way that messages can get the 'sent' label/tag.

## 2. Unread/inbox tags on sent mail ## Right now, I think I need to sync twice, to get sent email downloaded, "inbox", "unread"-tags removed, and then pushed back up again and shown properly in both notmuch and gmail.com, and not shown as "unread" on either side.

Probably yes, since the sent message is added by the GMail side and we do not have the gid-mid message map before you have done a pull. This is how it works for me as well. gmi sync is essentially just gmi push followed by gmi pull. If you add on another gmi push at the end you should hopefully be all set.

I have in my notmuch config: ``` [new] tags=inbox;unread;new; ignore=*.json; ``` and a post-new hook: ``` notmuch tag -inbox tag:sent and tag:inbox and tag:new notmuch tag -unread tag:sent and tag:unread and tag:new # gmail arbitrarily marks mails as "important", # note: not the same as "flagged". notmuch tag -important tag:important # gmailieer needs the "new" tag initially, can no be removed: notmuch tag -new tag:new

NOTE: gmailieer doesn't need it, it just ignores the tags listed as new.tags.

``` When I used the fcc dir, notmuch ran this code: ``` elisp (setq notmuch-fcc-dirs '(("work.com" . "\"work/Sent Items\" -unread -inbox") ;;("gmail.com" . "\"gmail/Sent Mail\" "-unread -inbox") ;; using gmailieer ("other" . "\"other/Sent\" -unread -inbox"))) ``` Which was handy, as it removed the "unread" and "inbox" tags right when sending the mail.

Yes, so there is probably some confusion because the message is added on the GMail side automatically. But has already been added to the local mail store in some way (but _not_ inside the gmailieer repository, which is not supported).

## 3. trash vs. delete? ## I notice the default tag for thrashing messages in notmuch, is "deleted". Is it possible to map "deleted" to be alias for "trash"?

Not currenly: #62

## Closing ## If you have any thought on what I've written, I'd be interested in hearing them, but I'm closing this now, as the main "issue" has been clarified now. Thanks!

Hope this helps!