Apple Intelligence v1.0 … is not good

Apple hasn't delivered on the promise of LLMs so far.

Apple's OS releases this week opened up Apple Intelligence to the public for the first time. Much of what it promises could eventually be useful. Unfortunately, after trying the very limited number of features in the initial release, I'd put Apple Intelligence into the bucket of "bad, won't use"; at least in its current state.1

Writing Tools

Screenshot of a MacOS application with text selected showing the new Writing Tools flyout menu

This seems to be the flagship feature of v1.0. Let's group the features into two buckets: okay and bad.

  • Okay
    • Summary: the summaries are accurate, but it doesn't do a good job of drawing conclusions.
    • List: If you select a bunch of things separated by punctuation like a comma it makes a bulleted list.
  • Bad
    • Proofreading: actually, this mostly does a good job proofreading, but on the mac its UI is busted. You can only revert a suggestion if you click through all them sequentially. If you click on the suggestion directly, it un-reverts the revert. This could be useful if fixed.
    • Rewrite: Changes facts, tons of cliches. Doesn't understand that quotes shouldn't be altered. Untrustworthy.
    • Friendly: holy exclamation points batman!
    • Professional: cliche alert
    • Concise: if you don't mind that it doesn't understand document structure/formatting, this will make the writing shorter, though not what I'd call concise.
    • Key Points: also gets confused very easily. Often elevating sub-points to the top level, or treating something as a conclusion when it wasn't.
    • Table: it gets confused very easily—I'm not sure who this is supposed to be for?

Notification Summaries

Hello. Is this on? I have decent notification hygiene, but I have gotten a number of notification stacks and no summary spotted yet.

Priority Email

Why are you showing me something that’s been read and archived? How do I get you to stop showing me?

Bummer

This is just the initial release, but it's unfortunate to see so many of these features just not work. The core issue with LLMs is that are not trustworthy. But, if productized in the right way, they can be trustworthy enough. Apple's strategy seems to be to break things down into very small problem sets—aiming to spread the trust surface area and to narrow the problem space. As of now, they've not succeeded.

Footnotes

  1. It's possible for the models used to change by device, I tested these features on an iPhone 16 Pro Max and an M2 MacBook Pro. ↩