Multimodal Parametric Cad Ai

Research project: training a domain-specific AI model to generate parametric 3D CAD models from natural language using geometric data and human design workflow sequences.

The Problem

Large Language Models (LLMs) like GPT-4 and Claude are trained on text. They have no native understanding of 3D geometry, spatial constraints, or the procedural logic behind parametric CAD design. When asked to generate a CAD model, they hallucinate coordinates, produce topologically invalid geometry, and cannot reason about constraint propagation.

The core hypothesis of this research is:

A model trained on three aligned modalities — natural language descriptions, geometric B-rep data, and real human CAD workflow sequences — will dramatically outperform text-only LLMs at generating correct, editable, parametric 3D models.

This is not just about generating shapes. The output must be:

• Parametric — editable by changing dimensions
• Procedural — expressed as a sequence of CAD operations
• Topologically valid — openable in real CAD software
• Design-intent-aware — reflecting how a human engineer would approach the problem, not just what the final geometry looks like

---

Core Idea

Standard LLMs predict the next token in a flat text stream. They have no concept of:

• 3D spatial relationships
• Geometric constraint graphs
• Why an engineer picks a specific sequence of operations

This project proposes a multimodal architecture combining:

Modality	Encoder	What it teaches
Natural language	BERT / fine-tuned LLM	Design intent and requirements
B-rep geometry	UV-Net / BrepNet (GNN)	What valid 3D geometry looks like
Human workflow sequences	Transformer over op sequences	How engineers actually design

All three are fused via cross-attention in a shared transformer backbone, which then decodes to parametric CAD operation sequences.

---

Current Status

🔬 Phase: Early research / architecture design
📅 Started: 2025
👤 Lead: Solo research project (open to collaborators)

See ROADMAP.md for planned milestones.

---

Repository Structure

multimodal-parametric-cad-ai/
├── docs/                    Research documentation
│   ├── architecture.md      System design deep dive
│   ├── datasets.md          Dataset sources and preprocessing
│   ├── problems-and-challenges.md  Known obstacles
│   └── research-journal.md  Running log of findings
├── architecture/diagrams/   Visual system overviews
├── experiments/             Experiment configs and results
├── data/                    Dataset download scripts (coming)
└── src/                     Model source code (coming)

---

Related Work (Key Papers)

Paper	Year	What it does
DeepCAD	2021	Generative model over CAD sequences
Text2CAD	2024	NL → parametric CAD (NeurIPS Spotlight)
CAD-MLLM	2024	Multimodal CAD generation (text + images + point clouds)
BrepGen	2024	B-rep diffusion model (SIGGRAPH 2024)
CAD-Llama	2025	Fine-tuned LLaMA3 for parametric CAD (CVPR 2025)
TransCAD	2024	Point cloud → CAD sequence (ECCV 2024)
SkexGen	2022	Disentangled CAD sequence generation
Fusion 360 Gallery	2021	Human CAD workflow dataset

---

How to Contribute

See CONTRIBUTING.md — all skill levels welcome.

---

License

MIT — see LICENSE