Unified Multimodal Models/ BAGEL/Cross-View Spatial Reasoning

How and What to Imagine?
Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning

A unified model can generate an intermediate thinking-image — a panorama, top-down, or point-matching render — before answering a cross-view spatial question. But under standard fine-tuning it ignores the picture it just drew. We introduce View Dropout to force the thinking-image to matter, and a Learnability–Informativeness framework to choose the right one.

Qian Yang^1,2 · Ankur Sikarwar^*,1,2 · Huy Le^†,1,2 · Le Zhang^1,2 · Zhuan Shi^1,3 · Perouz Taslakian^1,3,4 · Aishwarya Agrawal^1,2,5

¹ Mila – Québec AI Institute · ² Université de Montréal · ³ McGill University · ⁴ ServiceNow AI Research · ⁵ Canada CIFAR AI Chair
^*,† Equal contribution · {qian.yang, aishwarya.agrawal}@mila.quebec

↗ Paper { } Code ▤ Data

Overview: two input views and a cross-view question; three thinking-image strategies (panorama, point matching, top-down) with a learnability-informativeness quadrant; and View Dropout forcing the answer to route through the thinking-image.

Figure 1. Given two input views and a cross-view spatial question (left), a UMM generates one of three thinking-image types (middle) before answering. Right: without View Dropout the answer shortcuts through the input views, leaving the thinking-image unused; with View Dropout, part of one input view is masked, forcing the answer to route through the thinking-image and making visual thinking causally load-bearing.

01 Contributions

What this paper contributes

Method

View Dropout (VDrop)

A training-time intervention that hides part of one input view from the answer span while leaving it visible to the thinking-image. No architecture change; agnostic to the image type; consistently improves OOD reasoning across all three variants.

Framework

Learnability–Informativeness

We disentangle two axes prior work conflated: a thinking-image helps only if it is both informative (reveals spatial structure) and learnable (the model can actually render it). Neither axis alone is enough.

Empirical

8K beats 3× more data

On 1 in-domain and 5 real-world OOD benchmarks, the thinking-image becomes causally used only after VDrop. Panoramic + VDrop, trained on 8K samples, is the best config and beats prior BAGEL methods trained on ≥3× the data.

02 Abstract

Thinking with images that actually get used

Cross-view spatial reasoning remains a weak spot for vision-language models: they reason in language and discard the fine-grained geometry the task requires. Thinking with images aims to fix this by generating an intermediate thinking-image — but recent work shows the visual evidence in these traces is largely ignored.

We ask how to make visual thinking matter and what kind works best, for unified multimodal models that natively generate interleaved image–text. For the how, we propose View Dropout (VDrop), which hides parts of one input view from the answer span while leaving it visible to the thinking-image tokens — incentivizing the model to route its answer through the generated image. For the what, we frame the choice as a Learnability–Informativeness (L–I) tradeoff and compare top-down, panoramic, and point-matching renderings. Trained on synthetic scenes and evaluated on five real-world OOD benchmarks, panoramic visual thinking with VDrop is the only configuration that is simultaneously informative and learnable, and achieves the best out-of-domain generalization.

03 Method · How to imagine

View Dropout removes the shortcut

Standard SFT supervises both the thinking-image and the answer, but never forces the answer to depend on the image. The full layout is already recoverable from the two input views, so the model takes a shortcut — the generated image becomes a decorative by-product.

Attention mask

Hide one view from the answer only

Sample one view and a contiguous rectangle covering a fraction ρ of its patches. Block the answer span from attending to that region — in both the ViT and VAE token streams — while the thinking-image keeps full attention to both views. The missing layout is now recoverable only through the generated image.

Curriculum

Warm up, then anneal the pressure

Masking from step 0 collapses learning. We keep p_mask=0 for a 500-step warmup, ramp linearly over 1500 steps, then hold at 1 — so a working image→answer route exists by the time the view is fully masked. Default ρ=0.5, contiguous region, one view.

Compatibility

Only the mask changes

VDrop modifies the answer-side attention mask and nothing else: the SFT objective is unchanged, the thinking-image is still generated from the full views and supervised toward its ground-truth render. It works with any thinking-image strategy.

VDrop attention mask over the token sequence: thinking-image queries attend to all of V1, V2 and VT, while answer queries cannot attend to a red-hatched contiguous region of V1.

VDrop attention mask. Answer queries Q_a cannot attend to the masked region of V1 (red hatched), while thinking-image queries Q_vt retain full access to both input views — so the masked evidence reaches the answer only through the generated thinking-image.

04 What to imagine · three strategies

Three ways to bridge two views

Panorama

A wide-angle rendering from the observer's pose that reconstructs the full scene, so V1 and V2 become sub-regions of one unified visual field.

Top-down

A high-angle rendering from a top corner that lifts to a shared external frame, exposing global layout while still revealing object sides and depth.

Point matching

The two views side by side with coloured markers on corresponding objects — makes cross-view identity explicit without changing the camera frame.

05 Training data

An 8K training set with ground-truth thinking-images

We construct the training set from Infinigen Indoors procedural 3D scenes — so every sample comes with an unambiguous ground-truth answer and, uniquely, ground-truth renders of all three thinking-image types. Each scene provides two egocentric views with overlapping fields of view; questions follow the COSMIC benchmark's four cross-view types. This gives a clean, controlled training signal for every visual-thinking strategy within a single dataset.

7,921QA pairs

1,584unique 3D scenes

4cross-view question types

3GT thinking-images per scene
(pano · top-down · point-match)

The four cross-view question types. Every question requires integrating spatial information from *both* views.
Type	Task	Example question
Anchor	Identify an object visible in both views.	“Which object appears in both views?”
Counting	Count total instances of a given object across views.	“How many chairs are in the scene?”
Rel-Distance	Identify the closest or farthest object from a reference.	“Which object is closest to the desk?”
Rel-Direction	Identify the direction of an object relative to a reference.	“Which side of the sofa is the lamp on?”

Each training trace is the full interleaved sequence (V₁, V₂, q) → thinking-image → answer, supervising the UMM end-to-end. With only these 8K synthetic samples, the best configuration surpasses prior BAGEL-based methods trained on 3×–23× more data — the bottleneck is the training signal, not scale.

06 Main results

Panoramic + VDrop generalizes best out-of-domain

Accuracy (%). Everything below the prior-work block is BAGEL fine-tuned on the same 8K Infinigen data. Green = best per column, gold = second. OOD is the mean over five real-world benchmarks — our primary, unsaturated metric.

		COSMIC · in-domain					Real-world · out-of-domain
Model	VDrop	Anchor	Count	R-Dist	R-Dir	ID	MMSI	MindC.	Omni-CL	Omni-PT	STARE	BLINK	OOD
Understanding-only VLMs
Qwen3-VL-4B	✗	62.8	44.4	40.4	22.4	42.5	27.4	29.0	28.6	40.1	29.2	48.9	33.8
Qwen3-VL-8B	✗	64.8	54.4	40.8	26.8	46.7	28.0	34.4	25.4	45.5	31.6	55.6	37.0
BAGEL-based prior work
BAGEL (vanilla)	✗	18.0	42.1	24.4	21.2	26.4	26.9	31.7	31.1	38.9	28.0	45.1	33.3
BAGEL-Zebra-CoT (182K)	✗	8.8	24.4	28.4	24.8	21.6	23.2	21.7	29.0	43.0	28.4	24.8	26.8
ThinkMorph (24K)	✗	49.6	43.6	32.0	30.0	38.8	26.5	39.2	33.7	44.6	28.8	52.6	37.2
BAGEL + 8K · non-visual baselines
No-Think	✗	86.8	82.4	67.6	85.6	80.6	27.4	41.1	30.2	44.0	24.8	45.1	35.1
Text CoT	✗	56.0	60.4	44.4	40.8	50.4	24.6	25.3	29.4	35.8	26.0	54.9	32.7
BAGEL + 8K · visual thinking (ours)
Panoramic	✗	93.6	83.6	76.4	87.2	85.2	24.9	36.9	32.9	43.9	32.4	55.6	37.6
Panoramic	✓	89.2	78.8	74.8	93.2	84.0	26.0	34.1	38.9	45.3	35.6	62.4	40.0 +2.4
Point Matching	✗	92.8	81.2	73.2	94.4	85.4	27.8	35.2	31.0	41.7	24.0	52.6	35.2
Point Matching	✓	92.8	78.8	78.0	91.6	85.3	28.3	34.1	33.7	43.3	34.4	45.1	36.1 +0.9
Top-down	✗	93.6	83.2	68.4	92.0	84.3	28.8	35.2	31.4	39.8	28.0	58.7	37.3
Top-down	✓	94.4	76.8	72.0	89.2	83.1	32.0	36.5	32.5	44.2	26.0	57.1	38.0 +0.7

Finding 1

Visual thinking beats non-visual baselines on both ID and OOD. VDrop then lifts the OOD average for every strategy — panoramic 37.6→40.0, a +6.7 gain over vanilla BAGEL and above Qwen3-VL-8B, using only 8K samples. Text CoT actually hurts: inconsistent textual supervision is worse than none.

07 Is the thinking-image causally used?

Generate, then blind

The model draws the thinking-image as usual; then, before answering, we mask the answer span's attention to it. It still "thinks" — but answers without consulting the image. A model that genuinely uses the image should lose accuracy; one that ignores it stays flat.

Lollipop chart: performance drop from blinding the thinking-image. VDrop-trained model drops up to 12.8 points on BLINK; standard SFT stays near zero across BLINK, MindCube, OmniSpatial, and STARE.

Generate-then-blind. Accuracy drop when the generated thinking-image is blinded at answer time; a larger drop means more dependence. The VDrop model drops substantially (BLINK −12.8) while standard SFT is nearly invariant.

Line chart of answer-token attention share on the thinking-image across 28 decoder layers for vanilla BAGEL, SFT, and VDrop; SFT and VDrop attend more than vanilla, VDrop highest in early-mid layers.

Attention by layer (BLINK). The thinking-image's share of the answer span's visual attention rises from 55.3% (vanilla) to 63.6% (SFT) to 65.2% (VDrop), with the gap widening in early and mid decoder layers.

Finding 2

VDrop makes the thinking-image load-bearing: blinding it drops accuracy substantially while standard SFT is nearly invariant, and answer-token attention to the image rises — confirming the causal dependence VDrop is designed to induce.

08 What to imagine · the L–I tradeoff

Informative and learnable — panorama is the only one

A thinking-image helps only if a perfect instance would reduce the reasoning burden (informativeness) and the model can actually render it faithfully (learnability). We measure each on a paired COSMIC subset.

The two axes are laid out as a quadrant in Figure 1 (middle-right): panorama sits in the informative-and-learnable corner; top-down is informative but hard to draw; point matching is learnable but adds little new geometry. The two tables below measure each axis directly.

**Informativeness I(T)** — Δ accuracy when a *ground-truth* render is given to a frozen reader (Qwen3-VL-235B).
Oracle Δ	Anchor	Count	R-Dist	R-Dir	Overall
Input views only	73.2	66.1	49.5	39.1	56.7
+ Panorama	+0.6	+9.8	+15.3	+16.8	+10.8
+ Top-down	−6.5	−1.1	+10.0	+8.4	+2.9
+ Point matching	0.0	−9.3	0.0	+6.2	−0.8

**Learnability L(T)** — same test, but the render is *generated by BAGEL* after VDrop training.
Generated Δ	Anchor	Count	R-Dist	R-Dir	Overall
+ Panorama (gen)	−13.7	+3.3	+2.6	+6.7	0.0
+ Top-down (gen)	−13.1	−2.2	+8.4	+2.8	−0.7
+ Point match (gen)	−0.6	−2.2	−3.7	−0.6	−1.8

Finding 3

Top-down is informative but only partially learnable — BAGEL captures its layout only in part. Point matching is limited by low informativeness. Panorama alone scores high on both axes: its generated renders still carry spatial structure a frozen reader can use, and it is the only strategy that consistently beats prior methods on OOD.

09 Citation

Cite this work

@article{yang2026and,
  title     = {How and What to Imagine? Visual Thinking in Unified
               Multimodal Models for Cross-View Spatial Reasoning},
  author    = {Yang, Qian and Sikarwar, Ankur and Le, Huy and Zhang, Le and
               Shi, Zhuan and Taslakian, Perouz and Agrawal, Aishwarya},
  journal   = {arXiv preprint arXiv:2605.27310},
  year      = {2026}
}

Built on BAGEL (ByteDance-Seed, Apache-2.0). Training data rendered with Infinigen Indoors. View Dropout modifies only the answer-side attention mask and is compatible with any thinking-image strategy.

How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning