Generative UI

Earlier this month I made a money app in a day to try out generative UI, where the app builds the screen for each answer out of components. The model picked one or two blocks from a set of ten, which was fine for a sketch. I wanted to do it properly in Muesli, with smaller pieces the model arranges itself and everything it shows coming from your notes.

Design system

Before starting on the AI side I set up a design system for the model to use. I based the structure on Interfere's design system, where every token and prop is a short list of named options, so there's nothing to guess. I took that structure and kept Muesli's own look.

That means the model never gets a colour or a size to choose from. It picks a role, like card for a surface or caption for text, and the app maps each role to a value Muesli already used. Tone only has two options, neutral and attention. Attention is the dark red from the Start fresh button.

On top of that there are twelve pieces. Cards and rows hold other pieces, and the rest are things like headings, stats, lists, people, meetings, quotes, drafts and buttons. Each one has a short write-up of when to use it and when to avoid it. That same text is what the catalogue in the app shows and what goes into the model's instructions, so the two are always the same.

The rule I cared about most is that the model never writes anything that came from your notes. Those come in by reference instead, like m1.3.2 for the second line in the third section of meeting one, or m1.q4 for the fourth sentence of its transcript. The app looks each one up, and if it doesn't exist the piece is dropped. A stat counts the references that exist, so the model can't make a number up.

The layout comes back as a flat list where each piece names its parent. A nested tree isn't complete until its last bracket, but each piece in a flat list is complete once its own object closes, which means the app can draw it while the rest is still being written. If the model breaks a rule, like putting a card inside a card, the app fixes it before drawing. In that case the inner card flattens into its contents.

The working states are part of the system too. Each kind of piece has its own skeleton, and once the answer's in, the steps collapse into a line like "Claude · 8 lines sent · 1 step · 3.4s" that you can tap to see each step.

On device

Muesli writes its notes with Apple's on-device model, so I tried that first. It ran on my Mac and answered a test prompt in 3.4 seconds. I didn't take it further than that. In the money app sketch it had refused most of the finance questions, and here the bigger problem was size.

The on-device model has 4,096 tokens for everything, what you send and what it writes back. A request here came to about 3,600 tokens by Claude's count before it had found anything, with the pieces, the rules, the tools and a list of only four meetings. That would have left about 500 tokens for the lines from your notes and the answer itself.

So I switched to Claude for this, with your own API key. Five tools run on the phone and search your notes and calendar. From your notes, only the meeting list and the lines the tools return are sent.

Testing

To compare engines properly I wrote 22 questions about four made-up meetings, each with the kind of layout a good answer should have and the lines it should find. "What do I owe people this week?" should be a checklist with my two action items, for example. One question is about the weather, where the right answer is to show nothing. The app runs them in the simulator and draws every answer on one screen, so I can scroll through and compare.

The first engine had no model at all. It matches keywords to a few fixed layouts, and it's what Muesli falls back on when there's no key or no connection. It got the right kind of layout 68% of the time and found 61% of the lines. It showed meeting prep for the weather question because it had the word tomorrow in it, and for "What did Bob say about the budget?" it showed a quote that wasn't Bob's.

Haiku's first run found 93% of the lines, but 15 of the 22 answers had no sentence. I'd only asked for the sentence in the instructions. On the low effort setting I'd picked for speed, Haiku often went straight to the layout. So I made the sentence a required field in the tool call, ahead of the pieces. Every answer had one after that, and because it comes first, the sentence now shows up before the pieces do.

Two models

I started with Haiku 5.5, which is what the money app used, then ran the same questions on Sonnet 5.5 to see what a bigger model would add.

They came out pretty much the same on the numbers. Both got the shape right on nearly every question, and Sonnet found 95% of the lines to Haiku's 91%, which is within what changes from one run to the next. Haiku's pricing question found all its lines in one run and half of them in the next. They were the same speed too, about 2.8 seconds for the whole answer. Sonnet costs about 18 times as much, though, at 1.7 cents a question against a tenth of a cent.

The difference is in the writing. Sonnet's sentences are shorter, and Haiku tends to repeat what the pieces already show, like listing everyone in the checklist underneath. When I asked for a checklist for the launch, Sonnet made one from the planning meeting's action items, while Haiku said the notes didn't have one and then put a decision in a checklist anyway. Sonnet got one wrong the other way, reading "What's still open across my meetings?" as only my own items.

I haven't picked one yet. Next I'd like to bring the same answers into the chat inside each note, and save the pieces into the note's markdown, so a ticked checklist stays ticked when you open it in Obsidian.