DocAtlas

DocAtlas

Long-document understanding as mutable-state interaction.

Paper Code

Hongchen Wei1,†,‡, Yuanzhe Wang2,†,‡, Bei Liu2,*, Yifan Yang2, Qi Dai2, Kai Qiu2, Yunsheng Li2, Dongdong Chen2, Chong Luo2, Zhenzhong Chen1, Baining Guo2

1 Wuhan University2 Microsoft† Equal contribution‡ Work done during an internship at MSRA* Project leader

A document becomes more useful as the agent interacts with it

DocAtlas turns long-document understanding into a stateful process: search proposes where to look, reading exposes multimodal evidence, notes write findings back, and review recalls what matters next.

71.4%

MMLongBench-Doc accuracy

GPT-5.4 with DocAtlas exceeds the 65.8% human-expert reference.

63.7%

Compact policy after RL

A Qwen3.5-4B policy improves from 54.4% direct input to 63.7%.

+20.5

FinRAGBench-V gain

GPT-5.4 improves from 55.1 to 75.6 LLM-as-judge.

4

Tools in one environment

Search, Read, Note, and Review can be selected in any order.

Three benchmarks, one interaction strategy

Select a benchmark to compare DocAtlas with direct long-context input.

71.4All accuracy

MMLongBench-Doc

Why long documents should be mutable state

Static retrieval commits before reasoning begins. DocAtlas lets evidence change what the agent can find and do next.

The initial tree contains titles, page ranges, and summaries. When the agent records a source-grounded note, that finding updates both structured memory and the document tree. Later search operates on an enriched state rather than the same frozen index.

The mutable-state loop
1

Search

Navigate the visual-aware tree.

2

Read

Inspect selected multimodal pages.

3

Note

Write grounded findings back.

4

Review

Recall evidence when it matters.

state v0evidencestate v1

One document harness for inference and end-to-end RL

The same environment supports frontier VLM agents at inference time and trains compact open policies with outcome rewards.

DocAtlas mutable document harness
DocAtlas exposes Search, Read, Note, and Review over a mutable document tree and active memory.
01 · SEARCH

Search the tree

Retrieve candidate regions from hierarchy, summaries, and prior findings.

02 · READ

Read selectively

Choose the pages and multimodal views that enter active context.

03 · NOTE

Write grounded notes

Compress evidence with source attribution and update the tree.

04 · REVIEW

Review on demand

Retrieve relevant findings from structured memory before answering.

Search broadly. Read selectively.

Search is optimized for evidence recall; Read converts those candidates into a compact, high-precision evidence set.

87.45

Full Search All-Hit

Coverage of all gold evidence pages before selective reading.

5.79

Pages after Full Read

The agent consumes only a small subset of candidate pages.

+7.40

Tree-annotation gain

All-Hit improvement from writing findings back into the tree.

MethodAvg. pagesAll-HitPage F1
ColQwen top-22.0064.1238.75
ColQwen top-66.0076.4224.36
ColQwen top-1010.0083.6018.38
Full Search16.6587.4538.33
Search w/o Annotation16.9480.0529.40
First Read2.6759.9056.04
Full Read5.7978.0158.99
Read w/o Annotation6.0176.9358.17

Mutable-state interaction improves frontier and compact agents

DocAtlas reaches 71.4% on MMLongBench-Doc with GPT-5.4 and trains a 4B policy to 63.7% in the same environment.

71.4%MMLongBench-Doc accuracy
1

Above the human reference

DocAtlas + GPT-5.4 reaches 71.4%, compared with 65.8% for human experts.

2

One harness, two scales

The same tools support frontier inference and end-to-end RL for compact VLMs.

3

Gains transfer across benchmarks

GPT-5.4 improves by 20.5 points on FinRAGBench-V and 11.9 on LongDocURL.

GPT-5.4 direct62.4
Human expert65.8
DocLens + Gemini Pro67.6
DocAtlas + GPT-5.270.6
DocAtlas + GPT-5.471.4

What changes when state is mutable

Component ablations and tool allocation show that performance comes from the interaction loop rather than a single retrieval call.

DocAtlas component ablation
DocAtlas average tool calls

Inside a complete DocAtlas trajectory

Follow how the agent searches, reads, records source-grounded evidence, reviews memory, and produces a final answer.

Search · Read · Note · ReviewEvidence writes back to the tree

Multi-hop evidence gathering

DocAtlas multi-hop trajectory

Complete results across 27 systems and three benchmarks

The full table retains evidence-type breakdowns, aggregate metrics, compact-policy RL, and results on FinRAGBench-V and LongDocURL.

Best non-human values in the selected view are highlighted in red; second-best values are blue. Missing results are shown as dashes.

Long-document understanding as mutable-state interaction

DocAtlas: Long-Document Understanding as Mutable-State Interaction

AUTHORS & AFFILIATIONS

Hongchen Wei1,†,‡, Yuanzhe Wang2,†,‡, Bei Liu2,*, Yifan Yang2, Qi Dai2, Kai Qiu2, Yunsheng Li2, Dongdong Chen2, Chong Luo2, Zhenzhong Chen1, Baining Guo2

1 Wuhan University2 Microsoft

† Equal contribution   ‡ Work done during an internship at MSRA   * Project leader

DocAtlas turns a long document into a mutable environment: the agent searches a visual-aware tree, reads selected pages, writes source-grounded notes, reviews prior findings, and uses the updated state to make later decisions.

71.4%MMLongBench-Doc
GPT-5.4 + DocAtlas
+9.0points over
GPT-5.4 direct input
63.7%4B policy after
end-to-end RL

Abstract

Long documents should be stateful environments.

Existing retrieval systems choose evidence from a static index, while many document agents use tool calls around frozen backbones. DocAtlas instead models long-document understanding as mutable-state information seeking.

A document harness exposes search, reading, note-taking, and review tools; evidence recorded during exploration updates both a hierarchical tree and a structured memory store. The same environment supports large VLM inference and end-to-end RL for compact VLM agents.

71.4%
MMLongBench-Doc with GPT-5.4
63.7%
RL-trained Qwen3.5-4B
+9.3
points over 4B direct input

27-second product story

Why should long-document understanding be
a mutable-state interaction?

DOCATLAS / FILM 0101 / 06 · REAL-WORLD NEED
01
REAL-WORLD DOCUMENT WORK

The answer is rarely
on one page.

Financial reports, legal contracts, scientific papers, and government filings mix text, layouts, tables, charts, and figures.

02
WHY CURRENT APPROACHES STRUGGLE

Static retrieval.Frozen before reasoning begins.

Full context.Attention diluted by irrelevant pages.

Prompted agents.Tool use without a trainable policy.

The document stays fixed.
The problem does not.

INTRODUCING

DocAtlas

The document becomes state.

MutableGroundedFlexibleTrainable
03
THE DOCATLAS STRATEGY

Evidence changes
what happens next.

STATEv0v1enriched by evidence
SearchReadNoteSearch+ReviewAnswer
04
ONE ENVIRONMENT · TWO SCALES
INFERENCE71.4GPT-5.4
END-TO-END RL63.7Qwen3.5-4B

The same harness powers frontier agents and trains compact open policies.

05
MMLONGBENCH-DOC · ALL ACCURACY
GPT-5.4 direct62.4
Human expert65.8
DocLens + Gemini Pro67.6
DocAtlas + GPT-5.270.6
DocAtlas + GPT-5.471.4
+9.0MMLongBench-Doc+20.5FinRAGBench-V+11.9LongDocURL
DOCATLAS

Find evidence.
Update state.
Learn to interact.

71.4% · above human expert

Framework

A document harness for
inference and RL.

The same tool environment supports powerful VLM agents at inference time and end-to-end reinforcement learning for compact VLM policies.

FIG. 01DocAtlas mutable document harness
Overview of DocAtlas mutable document harness

System principles

Three mechanisms,
one mutable environment.

DocAtlas is defined not by a fixed tool sequence but by a stateful loop in which later retrieval and reasoning depend on earlier evidence.

01

Self-improving retrieval

Source-attributed notes are written back into a hierarchical tree, enriching the retrieval state for subsequent search.

02

Selective evidence access

Search proposes candidate regions; Read decides the final pages and multimodal views consumed by the agent.

03

Active working memory

Structured notes compress page-grounded evidence while Review retrieves relevant findings on demand.

Main results

Mutable-state interaction
improves long-document agents.

Across MMLongBench-Doc, FinRAGBench-V, and LongDocURL, DocAtlas improves both large VLM agents and compact Qwen policies trained in the same environment.

GPT-5.4 + DocAtlas71.4%

MMLongBench-Doc ALL — above the 65.8% human-expert reference.

Qwen3.5-4B + RL63.7%

Compared with 54.4% for the direct-input compact baseline.

GPT-5.4 + DocAtlas78.8%

LongDocURL LLM-as-judge score, up from 66.9% direct input.

TABLE 01 · COMPLETE MAIN RESULTS

Three benchmarks, evidence-type breakdowns, and compact-policy RL.

Green cells mark the best result and pale yellow cells mark the second best. Asterisks indicate performance above the MMLongBench-Doc human-expert reference.

FIG. 02Component ablation
DocAtlas component ablation results
FIG. 03Tool-use allocation
Average tool calls per question for DocAtlas agents

Evidence efficiency

High-recall search.
Selective reading.

Search is deliberately broad; Read turns candidate regions into compact, high-precision evidence consumption.

Full Search All-Hit87.45

coverage of all gold evidence pages before selective reading.

Full Read page F158.99

best evidence precision after the agent chooses which pages to consume.

Tree-annotation gain+7.40

All-Hit points from writing evidence findings back into the tree.

Search–Read evidence efficiency on MMLongBench-Doc
MethodAvg. pagesAll-HitPage F1
ColQwen top-22.0064.1238.75
ColQwen top-66.0076.4224.36
ColQwen top-1010.0083.6018.38
Full Search16.6587.4538.33
Search w/o Annotation16.9480.0529.40
First Read2.6759.9056.04
Full Read5.7978.0158.99
Read w/o Annotation6.0176.9358.17

Trajectory case studies

Inspect the complete
interaction trajectory.

Select a case to follow how DocAtlas searches the tree, reads evidence, records source-grounded notes, reviews memory, and answers.

SELECTED TRAJECTORY

Multi-hop evidence gathering

DocAtlas iteratively searches the document tree, reads selected pages, records intermediate findings, and combines them through review before answering.

Tools used
Search · Read · Note · Review
State update
Evidence writes back to tree
Output
Page-grounded answer
CASEMulti-hop trajectory
DocAtlas multi-hop evidence gathering trajectory