We graded 1,902 clips from 6 reference-to-video models against a fixed human rubric to see how often they turn a multi-character story into a clip that gets everything right.
Say you give a video model a reference image of a fox, one of a rabbit and one of a red umbrella, plus a few lines of story: the fox does a backflip, lands beside the rabbit and hands it the umbrella. The clip that comes back can look polished and still get the story wrong. The fox may match its reference perfectly and never actually flip, or the rabbit may do the flip instead. The umbrella may turn blue halfway through or end up in the wrong paws, and during the handover the two may never touch, or a paw may pass straight through the handle. A cartoon fox is allowed to squash and stretch mid-flip; it is not allowed to lose track of who did what to whom. CASTBench grades clips on exactly these points, one story and one clip at a time, so a result shows which model is ahead and also what each model can be relied on to do.
A story in CASTBench gives the model one written prompt and a reference image for every character and prop, and asks for one short clip in which several of those characters act and touch. Trained annotators then grade every clip on the 13 overall rubrics, and on the capability mode its story was written to test.
Seedance 2.5 leads. Even so, it gets 3.1 questions not fully right in an average clip and produces a fully clean clip, with every question and every scripted action a clean Yes, only 9.5% of the time. For Grok Imagine those figures are 7.2 and 0.6%.
Who leads, and how far every model is from a clean clip
We report the Rubric Pass Rate (RPR). On each clip, every rubric that applies to it (a capability mode applies only to the stories written to test it) scores one if every question and every item in it, such as each scripted action or each listed detail, is answered Yes; one half if the worst answer is Partial; and zero if any answer is No. A model's RPR is the mean of these scores over its clips for each rubric, then the unweighted mean over the rubrics.
The rule is strict on purpose: a clip in which one character's hand passes through another's arm, or in which a scripted hand-off never happens, is not a usable clip, however many other questions it passes.

On RPR over all 317 stories the field spreads from 60% to 82%. Seedance 2.5 leads clearly: it comes first in every one of thousands of re-draws of the stories. Two pairs sit too close to call, Wan 2.7 vs Kling O3 and MiniMax-H3 vs HappyHorse 1.1, so their order is not a finding.
Price separates the models too. We accessed all six on 18–19 August 2026. At their list prices as of 9 October 2026, one 10-second clip from Seedance 2.5 costs $4.73, about 7.9 times the $0.60 of MiniMax-H3, and one clip for every story costs $1,499 against $190. The best value comes from MiniMax-H3, HappyHorse 1.1 and Seedance 2.5: no other model is both cheaper and better than these. Kling O3, by comparison, costs $4.20 a clip for a held-out RPR of 64.1%.

The radar below uses a second measure, with partial credit: the Item Pass Rate (IPR) of each rubric. Each question on a clip scores the mean of its answers over the items it asks about, with Yes as one, Partial as one half and No as zero. Each question is averaged over the model's clips, and a rubric's IPR is the unweighted mean over its questions. One wrong answer lowers the IPR but does not zero it.

What CASTBench measures
CASTBench uses its 8 capability modes to design the stories, with each story written to test one of them, and its 13 overall rubrics to grade every clip. Together they measure six broad families of capability, in the order a story is read:
- Who is on screen: the right characters and props appear, each exactly once, and stay distinguishable from look-alikes and twins, including after being hidden (M4 Occlusion, M5 Look-alike, M6 Twin, OR1 Cast & props present, OR4 Cast count).
- Looking like the reference: characters and props keep the appearance on their reference sheets, down to small details (M2 Prop binding, OR3 Appearance consistency, OR10 Fine details, OR14 Prop appearance stability).
- Doing the scripted action: hard motions are performed and every scripted action happens, clearly framed (M1 Hard-motion integrity, OR9 Action coverage, OR13 Action staging).
- Characters acting together: scripted contact between characters happens, in the right direction, and looks physically real (M3 Interaction, M7 Handover).
- Physical plausibility: bodies and objects sit, rest, move and deform the way the physical world allows (OR2 Motion smoothness, OR6 Placement & spatial sense, OR7 Within-shot stability, OR11 Physics & deformation).
- Story continuity: the story holds together across cuts and over time, with sensible pacing and props that see their arc through (M8 End-to-end Persistence, OR8 Cross-cut continuity, OR12 Pacing).

Four parts of the rubric go wrong on more than half of all clips: Action coverage (OR9) on 82%; Interaction (M3) on 80%; Placement & spatial sense (OR6) on 56%; Hard-motion integrity (M1) on 55%. The least failed, Cast & props present (OR1), still goes wrong on 6%. Seedance 2.5 is the best model, alone or level with others, on 19 of the 21. The exceptions are Interaction (M3), led by HappyHorse 1.1, and Cross-cut continuity (OR8), led by MiniMax-H3.
Where each model falls short
Seedance 2.5 is the strongest model overall, but it is not the strongest everywhere, and each of the other models fails in a recognisably different place.
HappyHorse 1.1 and Seedance 2.5 make scripted contact happen most often (72% and 71% of scripted contacts), and these stories cannot tell the two apart. Yet HappyHorse 1.1 lets something pass through something solid more often than any other model (clips free of it: 56%, against 76% for Seedance 2.5). MiniMax-H3 is third on whether contact happens (63%) but, in the handover stories, scores highest for contact that looks physically real (91%) and is clearly staged (96%). Wan 2.7 and Kling O3 keep every character on screen and recognisable almost perfectly while doing far less of what the story asks: Wan 2.7 completes 55% of hard motions and Kling O3 41% of scripted contacts.
Each question in the chart below sits in its rubric's family, with four exceptions. For these, the rubric belongs to one family but what the question asks belongs to another, so we place the question by what it asks. That keeps each family about one kind of failure:
- M1.2 Identity through motion, from M1 Hard-motion integrity (Doing the scripted action), asks whether a character stays recognisable through a hard motion, so it sits under Who is on screen.
- M2.4 Prop use sensible, from M2 Prop binding (Looking like the reference), asks whether a prop is used in a sensible way, so it sits under Doing the scripted action.
- M1.3 Natural deformation, from M1 Hard-motion integrity (Doing the scripted action), asks whether bodies deform naturally during a hard motion, so it sits under Physical plausibility.
- OR14 Prop appearance: nothing comes loose, from OR14 Prop appearance stability (Looking like the reference), asks whether a held prop comes loose and floats free, so it sits under Physical plausibility.
Each row is a question on which the models differ: the best and the worst model are at least 10 percentage points apart, and the gap holds up across stories. Questions where all six models are close, or the gap is too uncertain to call, are left out. Rows are grouped by the six families above, in the order a story is read: who is on screen, how they look, what they do, how they act together, whether it is physically plausible, and whether the story holds together. Every model sits on the same scale, low on the left and high on the right, and models with close values sit side by side. Use the arrows to move between families.
Two kinds of story are comparatively easy in clips this short. Questions about a character who is hidden for a moment fail 7% of the time, and questions about telling look-alikes and twins apart 7%, while 40% of scripted contacts never happen at all. We did not vary how long a character stays hidden or how alike two characters look, so we do not claim either is solved in general. Some of the gaps between neighbouring models are small and should be read as a pattern rather than a ranking.
Models keep who is on screen yet they lose what the characters do
A character almost always stays recognisable through a hard motion, but the motion itself, and the story's actions, are where clips go wrong.
In the stories built around a demanding motion, the character stays recognisable through it in 97% of cases, while the motion is performed in only 71%. The chart below follows each model's clips on the interaction stories through a series of checks, keeping only the clips that pass every check so far: 69% to 97% of clips have the right characters on screen, and only 5% to 28% still pass once every scripted action must also happen.

Stories get harder as they ask for more. Averaged over models, a story with 1 reference image has an RPR of 90% and one with 4 has 64%; by number of characters the drop is from 91% to 63%. Longer prompts and more scripted actions show the same direction. These factors rise together across stories, so we do not separate them.

Characters often fail to act together
When two characters are supposed to touch, the usual failure is that the contact never happens. When it does happen, it is usually a real touch.
In the 39 interaction stories, 40% of the 840 scripted contacts between characters never happen, and only 21% of clips show every one. The cast is not the obstacle: in 87% of these clips every character is on screen and counted correctly. Contact is more likely in those clips (on average 60% of a clip's scripted contacts happen, against 33% when the cast is wrong), but still falls well short.

The models differ far more on whether contact happens than on how it looks once it does. The share of scripted contacts that never happen runs from 24% for HappyHorse 1.1 to 56% for Kling O3; the real-touch score stays between 82% and 92% for every model.
Props appear, then fall apart
Getting the right prop into the scene is easy; keeping it the same object, in the right hands, through the story is not.
In the stories with props, 92% of clips have every character and prop present, correct and in the right hands. Requiring the prop to keep its look leaves 64%. Requiring as well that nothing worn, held or in the scene pops or vanishes leaves 41%. In the stories that follow a prop's whole life, from handed over to used to its final fate, 20% get every stage right.

Failures stack
Most clips go wrong in more than one way at once.
Of all clips, 91% show at least one of 6 common kinds of failure and 68% show two or more. The most common pattern is action missing on its own (19% of clips), followed by action missing + object pops or vanishes (8%).

What this benchmark produces
CASTBench is released as 50 public stories, each with its prompt and the scripted actions and contacts it asks for, and the human grades of all 6 models' clips: 300 clips and 7,902 graded answers, with annotator identities removed. The other 267 stories are held out, so a new model can be scored on stories no one has tuned to. The scoring code reproduces every number in this post from the grades.
Because every score breaks down into what was asked of the clip, a result can be read at the level that matters for a fix: whether a model fails to start a contact or starts it and breaks the physics; whether a prop never appears or appears and drifts. A single overall score hides these differences.
What this does not show
This is not a general ranking of video models. Each story is one prompt and one short clip per model, so the results describe short, multi-character animation from reference sheets. RPR is deliberately strict: it rewards clips that get everything right, not clips that get most things right. Two pairs of models cannot be separated on it; the held-out stories are the reference.
Some comparisons are made on subsets: contact findings on the stories that script contact, prop findings on stories with props. Prices are list prices, not what any run was billed. How long a character stays hidden, how alike two characters look, and when in a clip a failure starts were not annotated, so we make no claim about them.







.png)
.png)