Matching a shot to a reference frame got within 10.0 ΔE of a colourist's grade on 18 Tears of Steel shots, against 17.7 when the look was described in words, and our best grading agent reached 13.4 against 19.2 for no grade. This article reports the agent designs, the judge calibration and the mask tests; it's a small test against one film's colourist.
Method. Our own tests, run in late September 2026 and written up on 29 September: 18 artist pairs for baselines that use no model, an 8-item comparison of grading-agent designs (4 for design variations), mask tools scored on public datasets; total model spend $3.54. Vinc makes an AI film studio, and these are our own tests of our own pipeline.
What this means for you
Bring a reference frame. Matching a shot's basic colour balance to a reference still moved it from 18.0 to 10.0 on our distance score, while picking a preset look from a written description only reached 17.7. The match is free and needs no AI.
If you grade from words, iterate and look, and stop as soon as it works. A loop that renders, checks the scopes and stops at the first round that hits its targets beat every one-shot design and broke no frames. Forcing an extra round made one shot worse again.
Don't trust a single absolute score from an AI judge on a stylised grade. Ours marked heavily stylised film frames about 3 out of 5 on their own. Ask it to compare against an ungraded version instead.
Check skin keys by eye, especially on darker skin. Our improved sky key overlapped the true sky at 0.75, but the skin key only reached 0.50, and its colour error on darker skin was twice as large on a sample of three.
For object-level colour changes on video, check every detail. A generative video editor was strong at background replacement and inserting objects, but it reinvented subtle recolours: a watch was replaced and black gloves came back brown.
You can describe a look in words ("cool teal shadows, warm skin, lifted blacks") or show a reference frame that has it. In late September 2026 we tested both on Vinc's colour-grading agent, which drives the same node-based engine a user drives by hand (primaries, curves, qualifiers, power windows, look nodes). Our short answer: show it a frame, which got closer to a professional colourist's grade than any description and needed no model call.
The footage was Tears of Steel (Blender Foundation), an open film whose camera plates and colourist-graded release frames are both available, so each shot has a professional target.
Item
Detail
Artist pairs, no-model baselines
18 Tears of Steel shots: ungraded plate plus the released graded frame
Agent comparison items
8 items: 6 Tears of Steel artist pairs, plus 2 practice items with faces (a real Sony F65 S-Log3 plate and a group with four skin tones), graded toward a filmmaker's stated taste
Design variations (one change at a time)
4-item subset
Grading model (the AI that sets the grade)
An OpenAI language model
Judge
gemini-3.8-flash, blind (images shuffled and labelled Image 1…N), separate prompt and call path
Total model spend
$3.54 for every experiment (agent comparison $2.96: OpenAI $2.81, Gemini $0.15; judge calibration $0.58)
The main measure was ΔE2000, a colour difference between the agent's frame and the colourist's (lower is closer). Pixel ΔE compares every pixel; stats ΔE compares the average colour of shadows, mid-tones and highlights, and is the one we trust more, because the release frames include VFX and cropping the plates don't. Failures were clipped skin, crushed faces, broken exposure, or more than 5% of the frame clipped.
Reference frame against words, with no model at all
First, the simplest baselines on all 18 artist pairs, with no language model involved.
Distance to the colourist's grade, 18 Tears of Steel shots (stats ΔE2000, lower is closer)
Plain conversion (no grade)18.0
Look recipe, chosen by look picker17.9
Look recipe, chosen by intent search17.7
Histogram curve match to reference15.8
Reference frame, primaries match10.0
Our test, late September 2026. No AI model involved. 'Look recipe' = the closest of 14 preset looks, picked from a description of the intent.
Picking a preset look from a description barely moved the frame toward the colourist's grade, from 18.0 to 17.7. Matching the shot's primaries (the basic colour-balance controls) to the reference frame took it to 10.0, at no cost.
On the 8-item set, "loop" means the agent grades, renders, reads the scopes and a judge's critique, and adjusts, for up to three rounds. "One-shot" grades once without looking.
Design
n
Pixel ΔE
Stats ΔE
Judge technical / intent (1–5)
Failures
Cost per item
Seconds per item
Plain conversion
8
20.2
19.2
3.20 / 2.88
0%
$0
–
Target = look recipe
8
21.0
19.0
3.22 / 3.50
13%
$0
–
Target = reference frame (supervised match)
6
14.6
13.2
3.30 / 2.67
0%
$0
–
One-shot, raw engine controls, text intent
8
20.8
17.7
3.18 / 3.38
13%
$0.05
27
One-shot, colourist-style controls
8
17.1
15.4
3.38 / 3.25
13%
$0.02
11
Loop, stop when targets are hit
8
16.1
13.4
3.68 / 3.63
0%
$0.14
70
Loop, fixed 3 rounds
8
16.1
13.4
3.58 / 3.63
13%
$0.14
70
Loop, stop when judge plateaus
8
20.1
17.7
3.37 / 3.38
13%
–
–
Loop, pick round with fewest issues
8
19.9
17.7
3.52 / 3.50
0%
$0.14
70
Agent designs, distance to the colourist's grade (stats ΔE2000, lower is closer)
Plain conversion19.2
Look recipe19.0
One-shot, text intent17.7
One-shot, colourist controls15.4
Loop, stop at targets13.4
Reference frame match13.2
Our test, late September 2026. 8 items (6 Tears of Steel artist pairs + 2 practice plates with faces); the reference-frame match has n 6 because it needs a reference frame.
The best design, a loop that stops at the first round hitting its targets, reached 13.4 stats ΔE against 19.2 for no grade, with no failures and the highest judge scores. A plain reference-frame match lands about there (13.2 on its 6 items) with no model calls. The match gets colour closest; the loop gets the best-judged frames and catches broken ones, so Vinc matches to a reference first, then runs the loop.
Why the loop needs to look at its own work
One-shot grading broke 13% of frames. On the F65 S-Log3 plate, round 1 crushed 56% of the frame, round 2 passed every target (judge 4.0 / 4.0), and a forced round 3 made it worse again. So Vinc stops at the first round that passes, else takes the last one.
The stopping rule mattered as much as the loop (see the table above): always doing 3 rounds matched on ΔE but failed 13%, and waiting for the judge to plateau stopped too early, at 17.7. Picking the round with the fewest open issues also scored 17.7, because issue counts are noisy. Other designs, on the 4-item subset:
Design (4 items)
Stats ΔE
Judge T / I
Failures
Loop, feedback from scopes only
13.8
3.25 / 3.50
0%
Loop, feedback from judge only
14.0
3.50 / 3.50
0%
Best-of-3 in one call, judge picks
13.5
3.55 / 3.50
0%
Technical pass first, then a separate look pass
17.8
2.95 / 3.00
25%
Splitting into a technical pass and a look pass was worst: the look pass undid the technical one.
Can an AI judge grade a grade?
We tested the judge against the colourists' own work first:
Asked which of two frames was better, it picked the colourist's grade over the plain conversion 18 of 18 times on Tears of Steel, and 92% of the time on a second set (FiveK).
In pairwise comparisons of the colourist against the agent, it was 100% consistent in both orders, with 0% order flips and no position bias.
Its absolute scores were biased against stylised finals. Shown alone, heavily stylised film frames scored about 3.0 out of 5 on technical quality. The first rubric flagged 6% of the colourists' finished Tears of Steel frames as technically faulty.
So Vinc never asks the judge for a grade in isolation: it always shows the candidate next to the plain conversion. Pass bars come from where the colourists' frames land: technical ≥ 3.0, no sub-score of 1, intent ≥ 4.
Masks: the sky key went from 0.43 to 0.75
Secondaries (corrections limited to skin, sky or background) need masks. We scored the engine's mask tools against hand-drawn public datasets; IoU is the overlap with the true mask (1.0 = perfect).
Tool
Test data
Before
After
Sky key
30 COCO-Stuff masks
IoU 0.43
0.75 (precision 0.83, recall 0.90)
Skin key
40 Pratheepan masks
IoU 0.44
0.50 (0.48 on held-out images); hue error inside the key 4.1°, 8.6° on dark skin (n 3)
Depth band
8 DIODE indoor scenes
IoU 0.44
1.00, once near/far is set in the clip's own depth percentiles
Window tracking
4 DAVIS sequences
box IoU 0.97 / 0.78 / 0.44 / 0.05
0.97 / 0.78 / 0.63 / 0.07. A small, fast object (kite-surf) still fails
The sky key improved by combining blue-sky, overcast and upper-frame-gradient keys. The skin result is the one to watch: 0.50 overlap is not a finished key, and the error on darker skin was twice as large on a sample of three.
On moving footage
On 3 October 2026 we checked the grades on video. Whole-frame looks carried over with no flicker, and dropping object windows and mattes cost less than 1 ΔE on most clips; tracking-based per-object control wasn't good enough and was dropped for video.
Object-level video changes now go to a generative editor, Runway Aleph, which we tried 5 times for about $4.76. It was strong at background replacement and object insertion, but reinvented subtle recolours: a watch was replaced, black gloves came back brown, and haze became a fog bank.
What we didn't measure
Small n. 8 items in the agent comparison, 4 for the design variations, 18 shots for the baselines. These numbers set product defaults; they aren't general truths.
One film's colourist. Every ΔE target comes from the Tears of Steel grade. A different colourist or film would give different numbers.
One model per role. One director model and one judge model. We didn't compare model families.
Video in the main comparison. The comparison was on still frames. The video follow-up was a check by eye on a handful of clips, not a scored test.
Human preference. No colourist or audience rated the agent's frames for this test.
How we compiled this
We ran the mask, judge and agent tests in late September 2026 (written up 29 September) and the video check and Runway Aleph tests on 3 October 2026. We quote our own results tables and didn't re-run anything for this article.
Public evaluation sets used for the mask tools: COCO-Stuff (sky), Pratheepan (skin), DIODE (depth), DAVIS (tracking), FiveK (judge calibration).
Read more →
Research··4 min read
What an AI Shot Really Costs, and Why Credits Are the Wrong Unit
Price AI film work by the cost of a shot the client accepts, not by the credits it takes to render one. This article puts three public figures from November 2026 side by side and shows that each one points the same way: generation is a small, visible line item, and judgement, revision and review are where the money goes.