Revised September 19, 2026 to replace broad claims about model interchangeability with a task-specific evaluation approach.
A model may fail because the evidence is missing, because its representation loses relevant detail, or because it cannot use the evidence correctly. These failures can look similar at the output. They require different interventions.
Separate the failure modes
Consider a system reading a financial document. A wrong value might originate in image resolution, OCR, table structure, retrieval, arithmetic, or an outdated source. Improving retrieval will not necessarily repair a table parser. A larger model may help with reasoning while leaving a source-data error untouched.
A useful evaluation preserves intermediate artifacts: the page region, extracted text, retrieved evidence, numerical operation, and final answer. These make it possible to localize a failure before choosing a fix.
Treat representation and data together
In multimodal learning, the training examples and objectives shape what a representation retains. Object identity, geometry, text, and temporal relationships may demand different forms of supervision.
A controlled experiment compares interventions under a fixed task and budget. One comparison might hold the model fixed while improving source data. Another might hold the data fixed while changing resolution, representation, or adaptation. Report the interaction between them rather than attributing every improvement to one layer.
Keep evaluation separate
Use held-out documents, scenes, or sequences that reflect the intended use. Avoid distributing near-duplicate frames or document templates across training and test splits. Specify how missing, contradictory, and stale inputs are handled.
Freshness is task-dependent: a stock price and an archived contract have different validity windows. Record when evidence was observed and whether it is still applicable to the requested decision.
Our technical report applies these questions to visual workflows, controllable generation, and spatial memory. The diagnosis separates improvements to the data, the model, and the surrounding system.