The Robservatory

Robservations on everything…

 

A possible fix broken search in macOS Mail

Over the weekend, I wrote about my totally useless search in Mail. I got so frustrated by my inability to search in Mail that I decided it was time to for a complete rebuild. I exported all my locally-saved mail, deleted my accounts, quit Mail, trashed its prefs and data files, rebooted, then rebuild it mailbox by mailbox, account by account.

I started with my iCloud account, which I barely use for anything—it has a total of seven messages in the inbox (four of which are iTunes Store receipts), and only 121 sent messages. As a test, I searched for Linea, an excellent drawing app that I had recently purchased. No matches.

At that point, I decided to quit Mail and force Spotlight to rebuild its index overnight. In Terminal, sudo mdutilmdutil -E / will do just that (and take many hours). Today, opened Mail, and search was still dead. Argh! (I had also tried this suggested fix, but it made no difference.)

But doing some random testing today, I discovered a fix! It’s a weird fix, but it seems to work:

If I move all the messages from an inbox or local storage folder into a different local storage folder, they’ll be indexed and findable. I can then move them back into the inbox or source folder, and they remain findable.

Even more important, newly-added messages seem to be properly indexed, in both the inboxes and the local storage folders.

This doesn’t make any sense to me, as any one of my recent actions—rebuilding mailbox indexes, reimporting, and redoing the entire Spotlight index—should have been enough to force a rebuild. But for whatever reason, only manually moving the messages seems to force a rebuild.

Now pardon me while I go back to manually dragging a quarter-million email messages around…

14 Comments

Add a Comment
  1. So you need to move all the email to a local folder (thereby deleting it off the IMAP/Exchange/GApps server) and then push it all back up to the server – just to get Mail to see them in the Spotlight index? Yet, if you use a Spotlight search or 3rd party utilities, the emails are indexed in the Spotlight database? Nice going Apple!

    1. Most of my archive is already in local storage, so it’s mainly just moving it around on the hard drive. But yes, the few messages still sitting on servers had to be copied down to the Mac, then copied back to the IMAP server.

      -rob.

  2. Rob, you of course know about the excellent utility MsgFiler, yes? http://msgfiler.com — this will let you semi-automate the ‘manual’ moving of tons of messages… with its ‘Engine’ addon, it’s super fast. I think I’ve experienced a variant of this problem you’ve seen. Somehow, sometimes, a message won’t get ‘seen’ by my filtering rules, and I’ll have to move it *back to the Inbox* (or, as you’ve discovered, any other mailbox) – then re-apply filtering rules, for it to actually get filtered. Seems like this is related. Somehow the database gets out of sync w/ the UI (or something), and manually (i.e., via the UI) moving something around, *re-alerts* the database to… the existence of these messages, in terms of indexing, or in terms of filtering.

    1. Yea, I used to use MsgFiler, but when I was trying to clean up this issue a while back I purged everything out of Mail, just to see if that helped. I should probably add it back :).

      -rob.

  3. The same work around also works for Notes. When search doesn’t work, move them all to a new folder, then copy them back to the original folder and trash the folder you just created. Search will work again. Don’t know how this fixes anything but it works, in other apps as well it seems.

    1. There is a relationship, at some level, between Notes and Mail. Remember that in some version of OS X, Notes were integrated (by default) in Mail. Now, when you create a new Mail account, you’re asked if you’d like to use it for Notes, Mail, or both. I imagine they must use the same database on the backend.

      Good to know the same fix works.

      -rob.

      1. Exchange, Google and iCloud all have Notes as a part of the account; back in the day when we ran our own Dovecot imap server, Notes objects were stored in a specific account folder on the server as ordinary mail messages.

        When we transitioned to Exchange we had to download the notes and re-add them into the new account.

        They’re treated as separate things in Outlook or Notes, but notes in an exchange account are just essentially email messages, just like everything else in an Exchange server. I suspect the same holds for Google and iCloud accounts as well.

  4. It worked! Thanks so much for letting us know. I’ve told others, as well. I even have to move my Sent mail to a folder and back in order to search that, too. Sigh. Seriously, thanks a million!! :)

  5. This is crazy, but dam, it works. Thank you for saving me from losing my mind!! My email is really my management system, frail as it is, its the most convenient way to track all my clients and projects. If that doesn’t work, I can’t work. You saved me!

    Gratefully yours, almost-200K-messages-to-move.

    1. Just create a new folder in Mail, in the “On my Mac” section, and move the messages into it. Let them sit for a bit, then move them back.

      -rob.

  6. Hi Rob, Just to clarify, when I do this does it force a reindex of only the messages that I move, or does it force a reindex of all messages? i.e. do I have to do this trick for all the messages in each folder I have separately in order to fix search for all of them?

    Mail search hasn’t worked reliably for years. Yet it’s a basic essential feature. Apple shouldn’t allow Mail to ship like this, let alone for years running without fixing or even acknowledging the problem… I’ve wasted so much time trying to fix it… incredibly frustrating…

    1. Just tried it and I can answer my own question for the benefit of anyone else reading this: every mailbox and sub-mailbox has to be done separately. It only reindexes what you move. This is going to be time consuming seeing as I have a ton of sub-mailboxes, but at least this is a hack that works. After a year fighting with Mail to get it to do basic searches properly this is the only thing I’ve seen that does. I guess that’s progress, though not on Apple’s part.

Leave a Reply

The Robservatory © 2017 Built from the Frontier theme