How and What to Imagine? — Visual Thinking for Cross-View Spatial Reasoning

In short

Draw a picture before answering — and make sure it gets used

Unified multimodal models can generate an image mid-reasoning. We study how to train them so that this “thinking-image” actually drives the answer, and what the image should show, on cross-view spatial reasoning.

Visual thinking helps

Generating a thinking-image beats no-thinking and text-CoT baselines, both in-domain and on five real-world out-of-domain benchmarks.

VDrop makes it matter

Standard training lets the model ignore its own image. View Dropout makes the thinking-image causally used, not decorative.

Panorama wins

Of three image types, the panorama is the only one that is both informative and learnable — and it generalizes best out-of-domain.

The task · cross-view spatial reasoning

Two photos of one room, one spatial question

The model sees two images of the same 3D room taken from different camera positions, and answers a multiple-choice question that needs both views — for example, where something seen in one view lies from the viewpoint of the other. To answer, it has to mentally stitch the two partial views into one consistent layout.

Left: an isometric render of a bedroom and a floor plan showing two camera positions with overlapping fields of view. Right: the two rendered input views and the question 'From the second view, in which direction is the window with beige curtain?' with options behind-right, front-right (correct), behind-left, front-left.

Example. Both views come from one shared 3D scene (left); the two cameras see overlapping but different parts of the room. Answering “front-right” requires locating the beige-curtained window, which is clearer in View 1, relative to the camera of View 2.

The idea · visual thinking

Think in images before answering

Reasoning only in words loses geometry: a text description of a room drops the exact positions and directions the question depends on. A unified multimodal model (here, BAGEL) can instead generate an intermediate image that merges the two views — a thinking-image — and then answer by reading it.

Pipeline: the two input views and question go into the model, which first draws a thinking-image — a panorama, a point-matching image, or a top-down view — and then answers 'B) front-right'.

Visual thinking. Before answering, the model draws one of three kinds of thinking-image: a panorama that unifies both views, a point-matching image marking the same objects in both views, or a top-down view of the room. Which kind to draw is the “what to imagine” question we return to below.

Method · How to imagine

Training with images ≠ using them

The standard recipe supervises both the thinking-image and the answer, but never forces the answer to depend on the image. Since everything is already visible in the two input views, the model learns a shortcut: delete its thinking-image at test time and accuracy barely moves.

Three panels. Training: the model learns to draw a thinking-image before answering. Inference: with or without the thinking-image, the answer is still correct. Accuracy: removing the generated image changes accuracy by only -0.8, -0.3, -1.0 and +2.4 points on BLINK, MindCube, OmniSpatial and STARE. — **The shortcut.** A model fine-tuned to think with images still answers correctly when its thinking-image is skipped: removing it changes accuracy by at most a few points.

View Dropout: make the thinking-image causally necessary

During training, hide part of one input view from the answer — but not from the thinking-image. The missing spatial evidence can now reach the answer only by going through the image the model draws.

Without VDrop, thick arrows go from both input views straight to the answer and only a thin arrow through the thinking-image. With VDrop, View 2 is partly masked, and the main path runs from the views through the thinking-image to the answer. — **Without VDrop** the answer reads the input views directly. **With VDrop** part of View 2 is masked for the answer, so the answer routes through the thinking-image.

Attention mask

Hide one view from the answer only

Sample one view and a contiguous rectangle covering a fraction ρ of its patches. Block the answer span from attending to that region — in both the ViT and VAE token streams — while the thinking-image keeps full attention to both views.

Curriculum

Warm up, then anneal the pressure

Masking from step 0 collapses learning. We keep p_mask=0 for a 500-step warmup, ramp linearly over 1500 steps, then hold at 1. Default ρ=0.5, contiguous region, one view.

Compatibility

Only the mask changes

The SFT objective is unchanged; the thinking-image is still generated from the full views and supervised toward its ground-truth render. VDrop works with any thinking-image type.

VDrop attention mask over the token sequence: thinking-image queries attend to all of V1, V2 and VT, while answer queries cannot attend to a red-hatched contiguous region of V1. — **How it is implemented: one attention mask.** Answer queries *Q_a* cannot attend to the masked region of a view (red hatched), while thinking-image queries Q_vt keep full access to both views.

Contributions

What this paper contributes

Method

View Dropout (VDrop)

A training-time intervention that hides part of one input view from the answer span while leaving it visible to the thinking-image. No architecture change; agnostic to the image type; consistently improves OOD reasoning across all three variants.

Framework

Learnability–Informativeness

We disentangle two axes prior work conflated: a thinking-image helps only if it is both informative (reveals spatial structure) and learnable (the model can actually render it). Neither axis alone is enough.

Empirical

8K beats 3× more data

On 1 in-domain and 5 real-world OOD benchmarks, the thinking-image becomes causally used only after VDrop. Panoramic + VDrop, trained on 8K samples, is the best config and beats prior BAGEL methods trained on ≥3× the data.

What to imagine · three strategies

Three ways to bridge two views

Point-matching render with colored markers

Training data

An 8K training set with ground-truth thinking-images

Built from Infinigen Indoors procedural 3D scenes — so every sample comes with an unambiguous ground-truth answer and, uniquely, ground-truth renders of all three thinking-image types. Each scene provides two egocentric views with overlapping fields of view; questions follow the COSMIC benchmark's four cross-view types.

7,921QA pairs

1,584unique 3D scenes

4cross-view question types

3GT thinking-images / scene
(pano · top-down · point-match)

The four cross-view question types. Every question requires integrating spatial information from *both* views.
Type	Task	Example question
Anchor	Identify an object visible in both views.	“Which object appears in both views?”
Counting	Count total instances of a given object across views.	“How many chairs are in the scene?”
Rel-Distance	Identify the closest or farthest object from a reference.	“Which object is closest to the desk?”
Rel-Direction	Identify the direction of an object relative to a reference.	“Which side of the sofa is the lamp on?”

Each training trace is the full interleaved sequence (V₁, V₂, q) → thinking-image → answer, supervising the model end-to-end. With only these 8K synthetic samples, the best configuration surpasses prior BAGEL-based methods trained on 3×–23× more data — the bottleneck is the training signal, not scale.

Main results

Panoramic + VDrop generalizes best out-of-domain

Accuracy (%). Everything below the prior-work block is BAGEL fine-tuned on the same 8K Infinigen data. Green = best per column, gold = second. OOD is the mean over five real-world benchmarks — our primary, unsaturated metric.

		COSMIC · in-domain					Real-world · out-of-domain
Model	VDrop	Anchor	Count	R-Dist	R-Dir	ID	MMSI	MindC.	Omni-CL	Omni-PT	STARE	BLINK	OOD
Understanding-only VLMs
Qwen3-VL-4B	✗	62.8	44.4	40.4	22.4	42.5	27.4	29.0	28.6	40.1	29.2	48.9	33.8
Qwen3-VL-8B	✗	64.8	54.4	40.8	26.8	46.7	28.0	34.4	25.4	45.5	31.6	55.6	37.0
BAGEL-based prior work
BAGEL (vanilla)	✗	18.0	42.1	24.4	21.2	26.4	26.9	31.7	31.1	38.9	28.0	45.1	33.3
BAGEL-Zebra-CoT (182K)	✗	8.8	24.4	28.4	24.8	21.6	23.2	21.7	29.0	43.0	28.4	24.8	26.8
ThinkMorph (24K)	✗	49.6	43.6	32.0	30.0	38.8	26.5	39.2	33.7	44.6	28.8	52.6	37.2
BAGEL + 8K · non-visual baselines
No-Think	✗	86.8	82.4	67.6	85.6	80.6	27.4	41.1	30.2	44.0	24.8	45.1	35.1
Text CoT	✗	56.0	60.4	44.4	40.8	50.4	24.6	25.3	29.4	35.8	26.0	54.9	32.7
BAGEL + 8K · visual thinking (ours)
Panoramic	✗	93.6	83.6	76.4	87.2	85.2	24.9	36.9	32.9	43.9	32.4	55.6	37.6
Panoramic	✓	89.2	78.8	74.8	93.2	84.0	26.0	34.1	38.9	45.3	35.6	62.4	40.0 +2.4
Point Matching	✗	92.8	81.2	73.2	94.4	85.4	27.8	35.2	31.0	41.7	24.0	52.6	35.2
Point Matching	✓	92.8	78.8	78.0	91.6	85.3	28.3	34.1	33.7	43.3	34.4	45.1	36.1 +0.9
Top-down	✗	93.6	83.2	68.4	92.0	84.3	28.8	35.2	31.4	39.8	28.0	58.7	37.3
Top-down	✓	94.4	76.8	72.0	89.2	83.1	32.0	36.5	32.5	44.2	26.0	57.1	38.0 +0.7

Finding 1

Visual thinking beats non-visual baselines on both ID and OOD. VDrop then lifts the OOD average for every strategy — panoramic 37.6→40.0, a +6.7 gain over vanilla BAGEL and above Qwen3-VL-8B, using only 8K samples. Text CoT actually hurts: inconsistent textual supervision is worse than none.

Is the thinking-image causally used?

Generate, then blind

The model draws the thinking-image as usual; then, before answering, we mask the answer span's attention to it. It still "thinks" — but answers without consulting the image. A model that genuinely uses the image should lose accuracy; one that ignores it stays flat.

Lollipop chart: performance drop from blinding the thinking-image. VDrop-trained model drops up to 12.8 points on BLINK; standard SFT stays near zero across BLINK, MindCube, OmniSpatial, and STARE. — **Generate-then-blind.** Accuracy drop when the generated thinking-image is blinded at answer time; a larger drop means more dependence. The VDrop model drops substantially (BLINK −12.8) while standard SFT is nearly invariant.

Line chart of answer-token attention share on the thinking-image across 28 decoder layers for vanilla BAGEL, SFT, and VDrop; SFT and VDrop attend more than vanilla, VDrop highest in early-mid layers. — **Attention by layer (BLINK).** The thinking-image's share of the answer span's visual attention rises from 55.3% (vanilla) to 63.6% (SFT) to **65.2%** (VDrop), with the gap widening in early and mid decoder layers.

Finding 2

VDrop makes the thinking-image load-bearing: blinding it drops accuracy substantially while standard SFT is nearly invariant, and answer-token attention to the image rises — confirming the causal dependence VDrop is designed to induce.

What to imagine · the L–I tradeoff

Informative and learnable — panorama is the only one

A thinking-image helps only if a perfect instance would reduce the reasoning burden (informativeness) and the model can actually render it faithfully (learnability). We measure each on a paired COSMIC subset — panorama sits in the informative-and-learnable corner; top-down is informative but hard to draw; point matching is learnable but adds little new geometry.

Quadrant chart with learnability on the vertical axis and informativeness on the horizontal axis. Text CoT: learnable but words cannot capture the layout. Panorama: informative and learnable, marked with a star. Point matching: no new viewpoint and hard-to-place dots. Top-down: informative but too hard to draw. — **The L–I tradeoff.** Text CoT is easy to produce but cannot capture the layout; point matching adds no new viewpoint; top-down is informative but too hard to draw. Panorama sits in the informative-and-learnable corner.

**Informativeness I(T)** — Δ accuracy when a *ground-truth* render is given to a frozen reader (Qwen3-VL-235B).
Oracle Δ	Anchor	Count	R-Dist	R-Dir	Overall
Input views only	73.2	66.1	49.5	39.1	56.7
+ Panorama	+0.6	+9.8	+15.3	+16.8	+10.8
+ Top-down	−6.5	−1.1	+10.0	+8.4	+2.9
+ Point matching	0.0	−9.3	0.0	+6.2	−0.8

**Learnability L(T)** — same test, but the render is *generated by BAGEL* after VDrop training.
Generated Δ	Anchor	Count	R-Dist	R-Dir	Overall
+ Panorama (gen)	−13.7	+3.3	+2.6	+6.7	0.0
+ Top-down (gen)	−13.1	−2.2	+8.4	+2.8	−0.7
+ Point match (gen)	−0.6	−2.2	−3.7	−0.6	−1.8

Finding 3

Top-down is informative but only partially learnable — BAGEL captures its layout only in part. Point matching is limited by low informativeness. Panorama alone scores high on both axes: its generated renders still carry spatial structure a frozen reader can use, and it is the only strategy that consistently beats prior methods on OOD.

Citation

Cite this work

@article{yang2026and,
  title     = {How and What to Imagine? Visual Thinking in Unified
               Multimodal Models for Cross-View Spatial Reasoning},
  author    = {Yang, Qian and Sikarwar, Ankur and Le, Huy and Zhang, Le and
               Shi, Zhuan and Taslakian, Perouz and Agrawal, Aishwarya},
  journal   = {arXiv preprint arXiv:2605.27310},
  year      = {2026}
}

Built on BAGEL (ByteDance-Seed, Apache-2.0). Training data rendered with Infinigen Indoors. View Dropout modifies only the answer-side attention mask and is compatible with any thinking-image strategy.