Unseen email search on Office365 email account

Question

We are using Mail.dll to connect to an 'outlook.office365.com' mail account.

The Inbox consists of a collection of Seen and Unseen emails.

We are trying to return a list of emails that have a specific email address and have not been read. We are trying to achieve this using the where clause on the imap.Search() function.

The search returns a number of results which doesn't match whats in the search criteria.

If we interrogate the returned results we observed that all of the emails have the flag 'Seen' set.

This was working previously, we have only noticed this after moving across to Office365.

using (Imap imap = new Imap())
{
    imap.ConnectSSL(InboundServerName);
    imap.UseBestLogin(InboundUserName, InboundPassword);
    imap.SelectInbox();

    List<long> uids = imap.Search().Where(
        Expression.And(
            Expression.HasFlag(Flag.Unseen),
            Expression.Or(
                Expression.To(EmailAddress),
                Expression.Cc(EmailAddress))
        ));

    foreach (var uid in uids)
    {
        IMail email = new MailBuilder().CreateFromEml(
        imap.PeekMessageByUID(uid));
        var flags = imap.GetFlagsByUID(uid);
    }
    imap.Close();
}

Answer 1

First I need to explain that entire search process is done by the IMAP server.
Mail.dll creates and sends a search query and the retrieves the results. If the server is performing search incorrectly, there is not much you can do.

Try using:

Expression.Not(Expression.HasFlag(Flag.Seen))

instead of:

Expression.HasFlag(Flag.Unseen)

You can also turn on logging to see the query send by Mail.dll IMAP library:
https://www.limilabs.com/blog/logging-in-mail-dll

Comments

We have tried this, same problem.

---

What happens if you remove other search criteria:

List<long> uids = imap.Search(Expression.HasFlag(Flag.Unseen));

-or-

List<long> uids = imap.Search(Expression.Not(Expression.HasFlag(Flag.Seen)));

---

This returns unseen emails for all the To addresses inside this mailbox.

---

Could you please test with the latest version:
https://www.limilabs.com/mail/download

It would be good to get the simplest query that fails. Try:

   Expression.And(
       Expression.HasFlag(Flag.Unseen),
       Expression.To(EmailAddress),
   ));

Can you also experiment with Parentheses setting:

    List<long> uids = imap.Search()
        .Parentheses(CriterionParenthesesMode.Always)
        .Where(...);

---

We have followed your guidance and were unsuccessful. We have rewritten our search expression to exclude the AND's. This has brought us closer to our goal but it's still missing a few of the unseen emails. Previously the results returned were entirely incorrect (Seen instead of Unseen). Now with the rewritten query, the results returned are correct but incomplete. The first time we ran the query it was correct. After adding another email to the inbox, the query returned the same number of results without the additional email.

imap.Search().Where(Expression.Not(
    Expression.Or(
        Expression.Not(Expression.HasFlag(Flag.Unseen)),
        Expression.Not(Expression.To(CommitteeEmailAddress))
    )
));

---

Your query is correct in my opinion, although I'd say this is more readable:

    Expression.Not(
        Expression.Or(
            Expression.HasFlag(Flag.Seen),
            Expression.Not(Expression.To(CommitteeEmailAddress))))


If the server is not returning the newest message it may be because of some server caching. Make sure the message is unseen and try in few minutes.

If it doesn't work you should report this error to the server administrator (logs may be helpful to him).

---

We have waited 12 hours before running the query again, once again we are not receiving the correct amount of unseen emails. We have also run a basic test that returns unseen emails, and this returns the expected list ( including the newly added email). We have programmatically set the emails as seen expecting the query to return the remaining emails on the next run, but unfortunately we received the same amount of emails, just with the flag Seen set.

We have added logging, is there anything particular you require or we could supply from the output?

---

You can show the query generated by Mail.dll here (UID SEARCH line), but as we are not responsible for how your server performs search, we can't help you.

You should report this issue to your server administrator.

---

For additional testing, we created a new @outlook.com email address on a different server. We sent this address 3 emails. When running the query ALL 3 emails appear. After marking 1 as read, we still received the original 3 emails. 1 email returned with the flag 'Seen'.

We continually ran the query for 30 minutes without change. After modifying the query back to the original (the query posted in our initial question) we received our expected result. But after sending it a fourth email, the new email in not included in the results.

---

Each query returns correct result only the first time around. This look like caching issue, isn't it?

---

We did some tests on our own and I can confirm this is a server problem.

S: * 4 FETCH (UID 1888 FLAGS ())           // <- 1888 message is unseen
S: 17266eca7d4c4ef3 OK FETCH completed.
C: ab5175b61f664486 UID SEARCH SEEN TO test@example.com // <- we search for seen email
S: * SEARCH 1888                           // <- incorrect: 1888 is unseen

The only way to workaround it is to perform 2 searches and process the results on the client side.