Mail check fires with 1 unread email even if required count > 1

[dkozinn]

I'm running Blink1Control2 2.2.4 under Windows 10 20H2 (build 19042.508). I have an event source configured against my IMAP email server to trigger if I have 4 or more unread emails. Even though it shows "Play pattern when Unread email count >=4" the pattern plays even with a single unread email.

Here's the section of the configuration file (redacted) showing how it's configured:

{
    "type": "mail",
    "enabled": true,
    "name": "My Email",
    "patternId": "pattern3",
    "mailtype": "IMAP",
    "host": "somewhere.example.com",
    "port": "993",
    "username": "[email protected]",
    "password": "secret",
    "useSSL": true,
    "actionType": "play-pattern",
    "triggerType": "unread",
    "triggerVal": "4",
    "triggerOff": true,
    "errormsg": ""
  },

It does turn off when the number of unread goes to zero.

[todbot]

Hi,

What kind of IMAP server are you connecting to? (Gmail, Fastmail, etc) That is, is it a service I could sign up for and try out?

It may be the server is not returning numeric values to the UNSEEN IMAP command or Blink1Control2 isn't parsing what is being returned correctly.

It could be instructive to open up the Developer Tools by going to the menu option Blink1Control2 -> Toggle Developer Tools, then clicking on the "Console" tab in the developer window and look for the lines ImapSearcher.searchMailDo. One of those will have a results output that should be useful.

You may need to shut down Blink1Control2 and add this section to your blink1control2-config.json file:

  "logger": {
    "showDebug": true
  },

[dkozinn]

It's hosted on a private service but if needed I can set you up with an account for debugging. However, I think I've found a few things.

Playing around with Wireshark then manually simulating what I think B1C2 is doing. Here's what I sent and what the responses were:

A14 UID SEARCH UNSEEN
* SEARCH
A14 OK Search completed (0.001 + 0.000 secs).
**A15 UID SEARCH UNSEEN**
* SEARCH
* 29 EXISTS
A15 OK Search completed (0.001 + 0.000 secs).

(Hopefully it's obvious that I typed the lines starting with A15 and A15). It looks like the response was saying that 29 was a new message. I then told it to idle and got this:

+ idling
* 31 EXISTS
* 1 RECENT

Here's what I found in the debug console:

1600135173505: ImapSearcher.on-mail: unread 1
bundle.js:8 1600135173517: ImapSearcher.searchMail
bundle.js:8 1600135175525: ImapSearcher.searchMail: searchtimer
bundle.js:8 1600135175526: ImapSearcher.searchMailDo
bundle.js:8 1600135175601: ImapSearcher.searchMailDo: SEARCH criteria: ["UNSEEN"] pattId: pattern3 results: (4) [227973, 227974, 227975, 227976]

If it helps, I know that the backend is running what should be a reasonably recent version of Dovecot.

Let me know if you want me to set you up with an account to test.

[todbot]

Thank you for the detailed debug sleuthing, it's incredibly helpful. Let me do some tests on my end first before I ask for a test account.

There was a least one previous issue (#55) about looking only at new messages vs all messages. The current behavior is to look at only new messages but perhaps that is not the correct approach for unread count. (I assume this is what the above is representing too, if not let me know)

[dkozinn]

I've been looking to tinker with the code a little but I'm not that familiar with JS. In imapSearcher.js is this what actually does the search

self.imap.search( searchCriteria, function(err, res)

Is "imap" like a library function?

[todbot]

Hi @dkozinn, yes imap is a Node js IMAP library https://github.com/mscdex/node-imap
Sorry I've not been able to look much more into this recently. Some other work commitments that end by next week have been keeping me occupied.
I believe the issue stems from the distinction of "new msgs total" vs "new msgs since last checked". I think I changed it to be the latter when working on #55

[dkozinn]

No worries at all, I was just trying to understand the flow a bit better. I don't know if I'd ever get around to it, but I was thinking about a few other ways of dealing with email, most around being able to use a regex. But I haven't thought this through and don't know if it's worth the effort, I just wanted to get a basic idea of how the pattern matching worked.

As far as the number thing, I did see the comments and some of the code that got changed as a result of #55. I think I might have sent you an email about this, but I realize that both "total unread" vs. "unread since last checked" have value. My personal preference would be "total number of unseen") which you can get from STATUS INBOX (unseen). The problem with that is that some folks like to keep messages unread as a kind of a todo list (I've done this in the past myself), so not sure what the best way to deal with that is.

[todbot]

Yes, this is going to take a bit of thought. :)

But if you're wondering about searching, check out the IMAP SEARCH command, which is what imap and imapSearcher.js is using. The searching is all done on the server, so it's faster & more secure. In fact, IMAP SEARCH doesn't even return the messages themselves, just their UIDs. But this server-side search is also more limited, being only able to do substring matching (as far as I know). I did a quick survey of docs just now and these two are pretty quick summaries of what's possible: http://www.marshallsoft.com/ImapSearch.htm https://www.php.net/manual/en/function.imap-search.php