VCG/Bench Read the paper
ICML 2026 arXiv:2605.15677

VCG-Bench · Diagram-as-Code

Can models recover editable structure from pixels?

VCG-Bench separates two capabilities that visual benchmarks often conflate: reconstructing executable mxGraph XML from a diagram image and structured caption, and modifying known structure through natural-language instructions.

Read the paper Benchmark code Dataset on Hugging Face

Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing

Xiaoyan Su · Peijie Dong · Zhenheng Tang · Song Tang · Yuyao Zhai · Kaitao Lin · Liang Chen · Yuhang Gai · Yuyu Luo · Qiang Wang · Xiaowen Chu

01 Paper overview Executable output
Raster + caption mxGraph XML Editable diagram
1,449verified diagrams
6 / 15domains / subdomains
14atomic edit types
2complementary benchmark tasks

01 Research contribution

A benchmark for diagrams that must work—not just look right.

Diagram understanding becomes useful when the result remains executable, editable, and structurally faithful. VCG-Bench turns that requirement into a reproducible evaluation protocol.

01

Taxonomized diagram data

1,449 quality-controlled diagrams spanning six domains and fifteen subdomains, with rich captions and structural metadata; the public Task 1 release includes 1,444 restored XML files.

02

Generation and editing, separated

Task 1 measures structure recovery from visual evidence. Task 2 measures reliable modification when the source structure is already available.

03

More than visual resemblance

Executability, structural correctness, semantic relations, visual similarity, and patch fidelity expose different—and actionable—failure modes.

02 Benchmark design

Two capabilities.
One executable constraint.

The tasks are complementary but evaluated independently. A model may be excellent at editing valid XML and still fail to infer that XML from pixels.

Figure 2 Benchmark framework
VCG-Bench tests whether visual-centric models can produce valid diagram programs and modify them while preserving untouched structure.
Task 01

Visual → Structure

Reconstruction

Recover the diagram program.

Given a raster image and a structured caption, generate complete, executable <mxGraphModel> XML that preserves topology, layout, text, and style.

InputImage + caption
OutputComplete XML

Protocol note Image + structured caption is the main setting; image-only reconstruction is reported as a harder ablation.

Task 02

Structure → Patch

Editing

Change only what was requested.

Given source XML, its rendering, and an instruction, return a JSON differential patch containing original and modified XML fragments. A deterministic applier performs the edit.

InputXML + image + instruction
OutputJSON XML patch

Protocol note 1,005 instructions cover 14 atomic edit categories at Easy, Medium, and Hard composition levels.

5,000+ candidate diagrams
1,449 verified benchmark samples

03 Dataset & construction

Broad coverage, filtered for executable quality.

Template libraries, research papers, anonymized enterprise artifacts, and permissively licensed web sources are normalized into a consistent mxGraph representation and verified through automated and human checks.

Figure 3 Data synthesis & verification pipeline
Filtering combines parser and rendering checks with OCR, semantic similarity, and manual verification. Reported acceptance rates describe distinct quality checks rather than a single sequential funnel.
Figure 4 Dataset distribution

Coverage map

Six domains. Fifteen subdomains.

01

AcademicArchitecture, neural networks, data visualization

02

SoftwareUML, sequence, ER diagrams

03

BusinessProduct, report, strategy

04

ManagementGantt, hierarchy, flow

05

UI/UXWeb and mobile interfaces

06

GeneralMind maps

0.85SigLIP2 threshold
>95%OCR consistency
2.3avg. manual edits
Read the dataset card

04 Evaluation protocol

How each task is scored.

Every output must run first. Then VCG-Bench asks a different, task-specific question: was the source reconstructed correctly, or was the requested edit completed correctly?

Shared gate00

ESR · Execution Success Rate

Can Draw.io open and render the output?

The generated XML must parse successfully and produce a valid rendering before any quality score is calculated.

PassContinue to task-specific scoring
FailAll downstream metrics = 0
Task 01Reconstruction

Image + caption → complete XML

Did the model reconstruct the source diagram correctly?

Three checks cover appearance, diagram content, and rendered similarity.

  1. 01

    Does the reconstruction look well aligned? SCS

    Judges visual style, layout consistency, and aesthetic quality.

  2. 02

    Are the elements and relationships correct? CodeXQA

    Checks counting, attributes, and connections directly from the XML.

  3. 03

    Does the rendered result resemble the source? SigLIP2

    Measures visual-semantic similarity between the two rendered images.

Task 02Instruction editing

Source XML + instruction → patch

Did the model make the requested edit correctly?

Two checks verify that the result stays polished and satisfies every requested change.

  1. 01

    Was the original style preserved? SCS

    Compares style consistency and aesthetic quality against the pre-edit diagram.

  2. 02

    Were all requested changes completed? XDRFR

    Verifies each decomposed edit requirement directly in the modified XML.

Why fewer checks? Task 2 already receives the source XML. It evaluates precise modification—not reconstruction from pixels.

05 Key findings

The bottleneck is seeing structure—not editing it.

Models can modify valid mxGraph programs with high reliability, yet the same models struggle to infer those programs from visual evidence. The gap is largest for open models and compositionally dense diagrams.

Task 1 · Best evaluated modelGemini-3-Pro
.9626

Executable Success Rate

SCS
.7790
CodeXQA
.9313
SigLIP2
.9162
Task 2 · Instruction complianceEvaluated range
.8873–.9405

XDRFR across evaluated models

ESR range
.9920–.9993
Best XDRFR
.9405

A revealing capability gap

Qwen3-VL-32B

Reconstruct.0000Task 1 ESR
Edit.9986Task 2 ESR

Reliable XML manipulation does not imply reliable visual-to-structure recovery.

Figure 5 CodeXQA across diagram complexity
Each panel reports CodeXQA accuracy for one evaluated model family as diagram complexity increases from Easy to Hard.
01

Counting is the dominant Task 1 failure for evaluated open models. They omit, duplicate, or merge repeated visual elements.

02

Captions materially help structure recovery. In a 100-example ablation, removing them reduced SCS from .805 to .690 and CodeXQA from .930 to .786.

03

Executability is necessary, not sufficient. Valid XML can still encode the wrong topology, text, or relationships.

06 Paper examples

Inspect what the metrics cannot summarize.

Qualitative results from the paper show both reconstruction behavior and instruction-driven edits. Select a task, then expand the figure for full-resolution inspection.

Figure 6Task 1 qualitative reconstruction
The ordering produced by the judge-based SCS metric closely follows expert rankings on these qualitative examples.

07 Project ecosystem

Two complementary workflows.
Two different promises.

The benchmark repository supports the research release and reproducible evaluation. The companion Skill turns the same diagram-as-code principle into a hands-on Codex reconstruction workflow.

Research release 01

sxy1499894281 / VCG-Bench

The paper, benchmark data, synthesis pipeline, and evaluation toolkit.

  • 1,449-diagram benchmark and annotations
  • Task 1 and Task 2 evaluation protocols
  • Data construction and evaluation scripts
Open benchmark repo Browse dataset
Companion workflow 02

sxy1499894281 / drawio-reconstruction-skill

A reusable Codex Skill for reconstructing individual diagrams into editable Draw.io files.

  • Guided image-to-Draw.io reconstruction workflow
  • Iterative rendering and visual correction
  • Practical, high-fidelity one-off cases
Open Skill repo View companion cases

Beyond the benchmark

Codex + Skill reconstruction cases

These are companion workflow demonstrations created with Codex and the reconstruction Skill. They are not leaderboard samples or benchmark model outputs.

AI agent architectureEditable Draw.io reconstruction
01A Original reference
01B Reconstructed & editable

08 Scope & limitations

What VCG-Bench does—and does not—measure.

Clear boundaries make the benchmark easier to interpret and extend.

In scope
  • Draw.io / mxGraph structured diagrams
  • Executable reconstruction with visual evidence
  • Objective, rule-based incremental edits
Outside the current scope
  • Ambiguous intent or subjective redesign
  • Full coverage of rare styles and non-English diagrams
  • Validated deployment or expert-free high-stakes use

09 Resources

Reproduce the benchmark.
Build on the workflow.

01
ResearchPaper on arXiv
 02
Research releaseVCG-Bench repository
 03
Research releaseDataset on Hugging Face
 04
Research releaseEvaluation scripts
 05
Companion workflowDraw.io reconstruction Skill

Citation

If VCG-Bench helps your research

@article{su2026vcgbench,
  title   = {Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing},
  author  = {Su, Xiaoyan and Dong, Peijie and Tang, Zhenheng and Tang, Song and Zhai, Yuyao and Lin, Kaitao and Chen, Liang and Gai, Yuhang and Luo, Yuyu and Wang, Qiang and Chu, Xiaowen},
  journal = {arXiv preprint arXiv:2605.15677},
  year    = {2026}
}