[🌐 Homepage] [💻 Github] [🤗 Huggingface Dataset]
[📊 Leaderboard ] [🔍 Visualization] [📖 Paper]
SolidGeo is the first large-scale benchmark specifically designed to evaluate the performance of MLLMs on mathematical reasoning tasks in solid geometry. SolidGeo consists of 3,113 real-world K–12 and competition-level problems, each paired with visual context and annotated with 3 difficulty levels and 8 fine-grained solid geometry categories. Our benchmark covers a wide range of 3D reasoning subjects such as projection, unfolding, spatial measurement, and spatial vector, offering a rigorous testbed for assessing solid geometry.Through extensive experiments, we observe that MLLMs encounter substantial challenges in solid geometry math tasks, with a considerable performance gap relative to human capabilities on SolidGeo .We hope SolidGeo serves as a catalyst for advancing MLLMs toward deeper geometric reasoning and spatial intelligence.
Performance of six MLLMs on SolidGeo benchmark across 8 solid geometry subjects (left), and trade-off between accuracy and average generated token length across 25 MLLMs (right).
Some examples of our datasets.
Click to expand/collapse more examples
{
"question": [string] The question text.
"choices": [list of string] Choice options for some questions.
"image": [list of string] A list of image file paths used in the question text.
"answer": [string] The final answer to the question, provided in LaTeX format for consistent mathematical formatting.
"solution": [list of string] A Optional list of solution steps.
"answer_type": [string] The format of the answer:"single_step", "multi_step", "choice"
"problem_type": [list of strings] A classification of the type of math problem :"Measurement of Solid Geometric Forms","Solid Shape Identification","Spatial Metric Relations","Multi-view Projection","Planar Unfolding and Configuration","Composite Solid Structures","3D Coordinate and Vector Reasoning","Solid Geometry Modeling".
"complexity_level": [string] An indicator of the difficulty level:"Level 1","Level 2","Level 3"
"source": [string] The origin of the question.
"qa_id": [integer] A unique identifier for the question-answer pair.
}
If you find this work useful, welcome to cite/star us.
@misc{wang2025solidgeomeasuringmultimodalspatial,
title={SOLIDGEO: Measuring Multimodal Spatial Math Reasoning in Solid Geometry},
author={Peijie Wang and Chao Yang and Zhong-Zhi Li and Fei Yin and Dekang Ran and Mi Tian and Zhilong Ji and Jinfeng Bai and Cheng-Lin Liu},
year={2025},
eprint={2505.21177},
archivePrefix={arXiv},
primaryClass={cs.CG},
url={https://arxiv.org/abs/2505.21177},
}