SpatialLM
An LLM fine-tuned on 3D point clouds that writes structured indoor models: walls, doors, windows and oriented object boxes as text.
About
SpatialLM fine-tunes an ordinary language model, starting from Qwen2.5-0.5B, to read raw 3D point clouds and emit structured indoor scenes as Python-like text: walls, doors, windows and oriented 3D object boxes with semantic labels. It was trained on 12,328 synthetic indoor scenes (54,778 rooms) and the authors report state-of-the-art layout estimation on Structured3D with competitive ScanNet object detection. The 1.1 update adds a Llama-1B variant and lets you condition detection on a subset of 59 furniture categories.
The approach matters more than the scores: it shows a standard multimodal LLM architecture can do structured 3D modeling without task-specific network designs, which opens the door for AR, robotics and embodied AI use cases. Code is Apache 2.0 but the weights are CC-BY-NC-4.0, so commercial use is off the table. Input point clouds are expected axis-aligned with z-up, and synthetic training data means real-scan robustness deserves testing. Accepted at NeurIPS 2025, and the eval pipeline is reproducible.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- State-of-the-art layout estimation on Structured3D (author-reported, arXiv 2506.07491); no independent eval found
- Competitive 3D object detection on ScanNet (author-reported)
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (TIER metrics, metrics-v1). Evidence: arXiv 2506.07491 (Jun 2025, v2 Nov 2025); accepted at NeurIPS 2025; github.com/manycore-research/SpatialLM with reproducible eval (SpatialLM-Testset on HF); author-reported SOTA on Structured3D layout estimation. Strengths: first successful LLM fine-tune for structured 3D indoor modeling, reproducible benchmark pipeline, active 1.1 releases. Docked for: (1) weights are CC-BY-NC-4.0, not commercial-open; (2) trained on synthetic data, real-scan robustness unverified; (3) benchmark claims author-reported. License: Apache 2.0 (code); CC-BY-NC-4.0 (weights).
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 6.0/10
- Maintenance · recency, releases, issue load
- 8.0/10
- Openness · license + docs
- 7.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · arXiv 2506.07491 (v1 2025-06-09, v2 Nov 2025); accepted at NeurIPS 2025
- · github.com/manycore-research/SpatialLM with reproducible eval (SpatialLM-Testset on HF); SpatialLM1.1 adds Llama-1B variant
- · Author-reported SOTA layout estimation on Structured3D; competitive ScanNet object detection
- · Weights CC-BY-NC-4.0 (not commercial-open); trained on synthetic data, real-scan robustness unverified