Agentic Cad

✨ Overview

Agentic CAD explores how an AI agent can turn a natural-language design request into a sequence of structured CAD operations. Instead of asking an LLM to invent a complete model in one response, the agent can select from explicit CAD tools, execute operations through an MCP client-server workflow, and inspect the results.

The project brings together agent orchestration, engineering automation, structured tool calling, geometry validation, and LLMOps evaluation.

Current scope: CAD primitive creation and modification, Boolean operations, geometry measurement/validation, and STEP/STL export. This is an experimental prototype, not a replacement for engineering review or production CAD verification.

🎯 Why this project?

CAD workflows often involve many small, ordered operations. Natural-language interfaces can help translate design intent into those operations, but engineering output must be checked—not merely described.

Agentic CAD investigates three questions:

• Can an LLM select and sequence the appropriate CAD tools?
• Can tool-driven automation produce valid geometry and export useful CAD artifacts?
• How can we measure reliability, latency, token usage, and cost rather than judging the agent by its response text alone?

🏗️ Architecture

flowchart TD
    U["User: natural-language requirement"] --> API["FastAPI backend"]
    API --> G["LangGraph agent"]
    G --> L["LLM<br/>Groq: openai/gpt-oss-120b"]
    L --> G
    G --> C["MCP client"]
    C <-->|"Tool discovery / calls / results"| S["FastMCP server"]
    S --> T["CAD tool layer"]
    T --> CQ["CadQuery geometry operations"]
    CQ --> V["Measure and validate geometry"]
    V --> E["Export STEP / STL"]
    E --> R["Result returned to workflow"]
    G -. "Traces and evaluation" .-> LS["LangSmith"]

Request lifecycle

1. Understand — interpret the user's design requirements.
2. Select tools — choose from the CAD operations exposed to the agent.
3. Execute — send structured tool calls through the MCP client to the FastMCP server.
4. Inspect — measure or validate the resulting geometry where supported.
5. Export — generate a CAD artifact such as STEP or STL.
6. Trace and evaluate — inspect the execution path and measure end-to-end outcomes.

🔌 MCP client-server design

The Model Context Protocol (MCP) provides a standard interface between the agent and the CAD tool server.

• MCP client: connects the agent to the server and enables tool discovery and invocation.
• FastMCP server: exposes CAD operations through tool schemas and executes the corresponding functions.
• CAD tools: perform explicit operations and return structured results to the workflow.

This separation makes the system easier to extend: new operations can be added at the tool layer without embedding every CAD implementation detail into the LLM prompt.

Current CAD toolset

The available tool names in the current project include:

Category	Tools
Primitives	create_box, create_cyclinder, create_sphere
Features	create_hole, fillet, chamfer
Boolean operations	boolean_union, boolean_cut, boolean_intersection
Transformations	mirror_entity
Inspection	measure, validate_geometry
Export	export_step, export_stl

Implementation note: create_cyclinder is the current tool name in the toolset. Some operations, such as arbitrary entity placement/translations, may not yet be supported by the current tool schemas. Unsupported requests should be recorded as capability gaps rather than silently treated as successful designs.

🧰 Technology stack

Component	Role
Python	Core implementation language
LangGraph	Agent workflow and orchestration
LangChain	LLM and tool integration
Groq — openai/gpt-oss-120b	LLM used in the evaluated configuration
MCP / FastMCP	Standardized client-server interface for CAD tools
CadQuery	Programmatic CAD geometry
FastAPI	Backend API layer
Pydantic	Structured data and parameter validation where used
LangSmith	Execution tracing and LLMOps evaluation
STEP / STL	CAD exchange and mesh export formats

The project has also explored local inference with Qwen and vLLM; the metrics below refer to the evaluated configuration and should not be assumed to describe every model/backend.

---

📊 Initial LLMOps evaluation

The first evaluation used 10 experiments to establish a baseline for latency, task completion, tool use, geometry validation, token consumption, and API cost.

Metric snapshot

Metric	Initial result	Notes
Experiments	10	Small initial sample
Mean end-to-end latency	52.46 s	Average across the evaluated runs
P50 latency	34.99 s	Median / typical-run indicator
P99 latency	71.88 s	Highly uncertain with only 10 runs
Task success	90% (9/10)	Observed end-to-end result
Tool-call correctness	Almost 100%	Based on the initial evaluation; exact scoring denominator should be documented
Geometry validation	Validated for generated models	Validity does not alone prove every design requirement was met
Total token usage	191,638	~93% from Input Tokens
Total API cost	~$0.04	Rounded value
Cache hit rate	34%	Inference Report

Latency at a glance

xychart-beta
    title "Initial latency metrics (seconds)"
    x-axis ["P50", "Mean", "P99"]
    y-axis "Seconds" 0 --> 80
    bar [34.99, 52.46, 71.88]

The mean is notably higher than the P50, suggesting that slower runs pull up the average. Reviewing the slowest LangSmith traces is a practical next step to identify whether additional LLM calls, longer prompts, retries, or tool/workflow overhead contribute to the delay.

Important interpretation: P99 from 10 observations is not a stable estimate of production tail latency. It is retained here as the value shown by the dashboard, not as a statistically robust service-level guarantee.

What the first evaluation tells us

• Task completion: 9 of 10 experiments succeeded under the current evaluation criteria.
• Tool use: Tool-call correctness was close to 100% in the initial evaluation.
• Geometry: Generated models were validated, but geometric validity and requirement satisfaction are separate checks.
• Capability discovery: One case exposed a limitation in the available CAD operations, particularly around entity positioning. That is useful feedback for toolset design.
• Cost: The dashboard showed approximately $0.04 total for the evaluation. This is rounded; use unrounded provider usage for precise cost comparisons.

Recommended evaluation dimensions

Dimension	What to measure
Task success	Did the final model satisfy the request?
Tool-call correctness	Correct tool, arguments, sequence, and no unnecessary calls
Geometry validity	CAD-kernel validity and solid count
Specification correctness	Dimensions, feature count, positions, volume, and tolerances where applicable
Export success	Was the expected STEP/STL file created and readable?
Latency	Mean, P50, P95/P99 when the sample size supports interpretation, and maximum
Cost and tokens	Per-run input/output tokens and actual API cost
Failure classification	Model reasoning, invalid arguments, unsupported capability, tool error, or export failure

A model can be geometrically valid but still be the wrong shape. For engineering-oriented evaluation, geometry checks should be paired with specification-level assertions.

🔍 Why LLMOps matters for engineering agents

Tracing and evaluation answer different questions:

• Tracing: What did the agent do? Which model calls and tools ran, in what order, and where did time go?
• Evaluation: Did the agent achieve the intended result? Were the tools used correctly, and does the generated geometry meet the specification?
• Regression testing: Did a change to the prompt, model, tool schema, or CAD implementation improve results—or introduce new failures?

For engineering design, a useful evaluation should distinguish at least three outcomes:

1. Execution correctness — the agent called tools appropriately.
2. Geometric correctness — the CAD kernel produced valid geometry.
3. Design correctness — the final geometry meets the user's dimensions and feature requirements.

These outcomes should not be conflated into a single success score without explicit criteria.

🚧 Limitations and next steps

This is an early-stage baseline. The current results use only 10 experiments, and latency percentiles and success rates are therefore sensitive to individual runs.

Planned improvements:

• Expand the test set to 50+ diverse prompts, including edge cases and unsupported requests.
• Add or improve entity placement and translation support.
• Add automated checks for dimensions, bounding boxes, volume, solid count, and required features.
• Separate capability limitations from execution errors in evaluation reports.
• Inspect slow traces and optimize prompt size, repeated context, and workflow steps.
• Track per-run cost and token usage with unrounded provider data.
• Add regression tests for tool schemas and exported CAD artifacts.
• Explore more complex multi-step design and engineering validation workflows.

🚀 Long-term direction

The longer-term goal is to make CAD workflows more accessible through natural-language interaction while retaining the precision and verification expected in engineering. Potential extensions include richer parametric operations, assemblies, simulation integration, and iterative design optimization. These are future directions, not claims about the current implementation.

💻 Getting started

The exact commands depend on the repository's current environment and configuration. A typical setup is:

# Clone your repository
git clone <YOUR_REPOSITORY_URL>
cd <YOUR_REPOSITORY_DIRECTORY>

# Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate

# Install the project's declared dependencies
pip install -r requirements.txt

Configure the required model-provider credentials and project environment variables using your local .env or configuration approach. Do not commit API keys, tokens, or private tracing credentials.

Run the FastAPI application and MCP server using the entry points defined in your repository. For this project, use the actual module paths and launch commands from your current codebase rather than assuming a generic command.

🤝 Contributing

Ideas for improving the project include new CAD tools, stronger geometry assertions, more representative evaluation prompts, and better failure reporting. Contributions should include a clear description of the behavior and, where possible, a repeatable test.

⚠️ Disclaimer

This project is an experimental engineering automation prototype. Generated geometry and exported files should be independently reviewed and validated before being used for manufacturing, safety-critical applications, or other consequential engineering decisions.

---