November 2026
Can a model do a
creative director’s job?
Our internal evals, run blind. Forty real briefs, five systems, 2,000 ads, and six senior creative directors who never knew which system made what.
- 78%Ship rate
Ads creative directors would run unedited. Best frontier model: 17%.
- 89%Head-to-head
Sets preferred over the best frontier baseline, judged blind.
- 81%Grader agreement
Our taste judge agrees with the panel more often than its own members do.
- 7.9×Transfer
Ship rate of an open model trained on our decision trails, from 9% to 71%.
Generation scaled.
Nobody measured judgment.
“…generative AI creative tools have helped advertisers rapidly create hundreds of millions of new ads, growing Meta’s ads candidate pool to an unprecedented scale.”
“…our strongest, highest-reasoning models watch and judge the results of our system…”
“The model may be able to understand user intent and think critically to deliver something valuable, but people shape taste.”
Creative volume is solved. What is missing is a measure of whether the work is any good before it spends. Design benchmarks score structure, like layout, typography and file validity, with automated metrics. Ad systems learn quality only after the impressions are bought. Nothing measured creative judgment directly, so we built evals that do.
Same brief. Ten ads each.
Judged blind.
40 real briefs from DTC brands spending $50k to $1M a month, across skincare, supplements, apparel, home and food: the product, the brand kit, audience evidence and the live offer.
Dessert, GPT Image, Gemini image and Meta Advantage+ creative. The human ceiling is a top agency team working the same briefs.
Six senior creative directors averaging 14 years at agencies and brands. System names hidden, order randomized, every set judged by three.
A complete ten-ad set per brief, not a single image: four top, three mid and three bottom of the funnel.
- One generation per brief. No cherry-picking.
- No human edits to any output.
- Identical inputs for every system.
- Judges never see system names.
- Eval brands are held out of anything Dessert trained on.
- Every judgment and reason code is logged: 6,000 verdicts, 36,000 step scores.
Would a creative director
run it, unedited?
Where does
each model fail?
| System | Persona | Angle | Funnel job | Layout | Brand | Product |
|---|---|---|---|---|---|---|
| Dessert | 94 | 91 | 93 | 88 | 92 | 90 |
| Human ceiling | 92 | 90 | 89 | 91 | 95 | 96 |
| Meta Advantage+ | 41 | 35 | 44 | 61 | 72 | 94 |
| GPT Image | 31 | 38 | 22 | 54 | 63 | 91 |
| Gemini image | 27 | 33 | 19 | 49 | 58 | 90 |
Every model can render the product. None of them can pick the persona, the angle or each ad’s job in the funnel. That’s the judgment, and it’s the part we built.
Who judges creative
like a creative director?
Ad systems already use their strongest models as judges. On creative, they agree with creative directors about half the time. The Dessert grader agrees with the panel more often than a typical panel member does, so it works as an eval and as a reward signal.
Taste can
be trained.
We fine-tuned an open-weights model on Dessert decision trails: the brief, every call a creative director made, and why. Ship rate went from 9% to 63% with no change to the architecture. Using the grader as a reward signal took it to 71%.
Nine of ten,
run unedited.
Run · 3 of 3
Run · 3 of 3
Run · 3 of 3
Run · 3 of 3
Run · 3 of 3
Fix · hierarchy
Run · 3 of 3
Run · 3 of 3
Run · 3 of 3
Run · 3 of 3The method, the data
and the judge.
The method, the judging protocol and the per-brief results behind every number on this page.
All 40 briefs and 2,000 judged ads, with 6,000 verdicts and reason codes, to run your own models against.
The calibrated taste judge, as an eval for your creative or a reward signal to train on.
- Wrong persona
- Angle not supported by evidence
- Wrong funnel job
- Weak hierarchy
- Off-brand
- Product mismatch
- Claim not allowed
- Would not stop the scroll
Run your model
through our evals.
Model labs · Ad platforms · Creative tools