Artificial Viewer
A machine that watches film. It receives only a foveated view, weighs what stands out against what a language model finds meaningful, selects its own fixation, and then sees the world from wherever that choice left it.
7
Experiments
75
Film clips
223
Human observers
1
Artificial viewer
A closer look
at looking
A different question
Most models predict human gaze: they are handed a frame a person reached, and scored on where that person looked. They are never asked to survive their own mistakes.
JOY is given the frame only through a foveated percept, computes its evidence on that percept and never on the full image, arbitrates between competing cues, samples a fixation, and begins again from wherever it landed. The four panels are one real frame through those four stages.
Read the full story →
Research at a glance
Acuity falls with eccentricity, and both evidence models see only what survives it.
Salience, semantics, oculomotor history and a centre bias, each with its own weight.
It updates its own history from its own choices. No human fixation reaches inference.
Free film viewing and task-directed search, paired and bootstrapped, folds reported.
Featured results
+0.0099 bits as a predictor, positive in five of five folds. As an observer, a median of −0.919 and three wins in eight clips.
A global weight set fitted for held-out likelihood gives the semantic map zero in four of five folds, while per-frame fits give it 0.0475.
Fixations of 214 to 220 ms against a human 179, saccades of 2.4 to 4.7 degrees against 1.44. The gap no semantic mechanism touched.
| What each mechanism bought | bits per fixation | folds |
|---|---|---|
| Semantic map over no semantic map | +0.0030 | 4 of 5 |
| Story state over no story state | +0.0041 | 5 of 5 |
| Best arbitration policy over fixed weights | +0.0099 | 5 of 5 |
| Real story state over the same state from another clip | +0.0214 | 5 of 5 |
| Gaze generated from the semantic map alone | −4.726 | excludes 0 |
| All the headroom there was | +0.2108 | 5 of 5 |
Publications
Element 1 · 60%
Seven experiments, nine figures, every number drawn from a stored result file rather than typed.
Element 2 · 20%
Four studies over 36 months, from motor dynamics to a dramatic-irony eye-tracking study.
Element 3 · 20%
Twelve slides with the full spoken script and the questions it should expect.
Data
| resource | detail |
|---|---|
| Film corpus | DIEM · 75 clips, 73 videos, 4,492 sampled frames, 223 observers, 4,619,922 identified fixation samples |
| Search corpus | COCO-Search18 · 7,697 held-out per-fixation examples over 313 images and all ten observers |
| Validation | 45 named required tests on the film dataset, 28 on the search dataset, all passing |
| Grounding quality | 0.6% refusals, median overlap 0.855 against the annotated target |
About
A page that reports only what worked is not reporting a result. Adaptive arbitration is constrained here, not refuted. Foveation was not shown to humanise the viewer. The semantic map may be operating through high-level image structure rather than meaning. Narrative memory failed outright, and the claim that it helped was withdrawn. The closed-loop comparisons use eight clips and are underpowered.
MRes Creative Computing, Creative Computing Institute, University of the Arts London. Supervised by Tim J. Smith. Both corpora are public research datasets used within their terms and are not redistributed.
| behaviour | audience | viewer |
|---|---|---|
| Fixation duration | 179 ms | 214–220 ms |
| Saccade amplitude | 1.44° | 2.4–4.7° |
| Returns within one degree | 46% | 4–24% |