A ticket landed on my desk with a deceptively simple ask: design a clearer visual separator between an AI-powered recommendation module and the rest of a resort search results page. Travelers were confused about which results were their actual search and which ones the AI had inserted.
The obvious fix was a label, maybe a divider, a small "Recommended for you" treatment. I drafted ten versions of the copy. But underneath the copy sat a real fork nobody had resolved. One option intermixed the AI picks with the organic results and positioned them higher, where they'd get more clicks. The other kept them clearly separated, even if that meant fewer people ever noticed them.
Those aren't the same goal. One optimizes for people clicking on a recommendation. The other optimizes for people understanding what they're looking at. No amount of typography solves that, because it was never a typography problem.
It's tempting to treat that as a design decision waiting on stakeholder input. I think it was something else: a strategy question that had never been settled, wearing a design ticket as a costume. Once I looked at the rest of the recommendation program, I found the same unresolved question sitting underneath more than one ticket.
What the review actually found
When the team stepped back and looked at where the recommendation engine stood, two related gaps showed up.
No shared definition of "what a model is for"
Which model should own this behavior? Two models had drifted close enough together that nobody could clearly say what made them different.
How should the UI distinguish it? A design ticket was asked to resolve a strategy question it had no way to answer.
First, several models didn't have fully settled logic. One model, built to surface options similar to a traveler's past bookings, and a newer one meant to broaden personalization further, had drifted close enough together that nobody could clearly say what made them different.
Second, and this was the bigger one: there was no shared definition of what a model was actually for. Every model was evaluated on its own internal statistics — precision, recall, training performance. None of that told anyone whether a recommendation shown to a traveler ever led to a booking. Data science, engineering, and design were each judging success by a different yardstick, and none of those yardsticks connected back to what a traveler actually did.
Those two gaps compound each other. Without a clear definition of what a model does, it's hard to build toward a shared goal. Without a shared goal, there's no pressure to define the model clearly in the first place. The program had grown this way for over a year, which is a fairly normal way for a recommendation system to develop, and also exactly how a design team ends up litigating tone of voice on a shelf label instead of the question underneath it.
The fix that caught something real
The fix we tried wasn't a process rollout. It was closer to a one-page form: for every recommendation model, write down what data it uses, how it ranks results, who sees it, what makes it different from every other model, and how you'd measure whether it worked. Nothing more elaborate than that.
| Field | What it has to answer |
|---|---|
| Data | What inputs does this model actually use? |
| Logic | How does it rank or select results? |
| Audience | Who sees this, and in what context? |
| Differentiation | What makes it different from every other model in the program? |
| Goal / measurement | How would you know if it worked? |
Writing it down is what caught the problem. The two overlapping models turned out to have almost identical inputs and no clear answer for how they differed. That overlap had been sitting there for a while. It just wasn't visible until someone had to write a plain sentence explaining why a traveler would see one shelf instead of the other, and couldn't.
Personalization systems tend to accumulate this kind of debt quietly, especially once the people building a model, the people naming it, and the people measuring it all sit in different parts of the org chart. It's not unique to travel, or to one team.
What this isn't
I want to be honest about what this is, and isn't. It wasn't a finished success story with a conversion lift to point to. It was a proposed direction, tested against one real example of the ambiguity it was meant to catch.
It's also not an argument for more process. The point of the one-pager was to add just enough friction to catch a real problem, not to build an approval chain that slows down every future model. If it takes more than a paragraph to explain what a recommendation is for, that's worth noticing. If it takes a committee to sign off on it, that's a different problem.
"Most alignment problems don't announce themselves in a strategy meeting. They show up quietly, in a ticket that looks like it's about a divider."
What I'd carry forward
A few things I'd apply on the next recommendation system, or any feature with a model quietly making decisions behind it:
Write the one-paragraph definition, covering data, logic, audience, differentiation, and goal, before a model goes anywhere near naming or visual design.
If two models need nearly the same sentence to describe them, treat that as a signal to merge or retire one, not a copywriting problem.
When a design decision feels like a genuine coin flip between two reasonable goals, ask whether the strategy that should have decided it was ever actually made.
Treat "we can't tell if this recommendation worked" as a blocking issue, not a metric to backfill later.
Closing
The ticket about the visual separator eventually got resolved with a label and a clear boundary. But the more useful outcome was smaller and less visible: a habit of writing the sentence down before building around it.