Benchmark · 2026-10-24
Text to CAD, scored on 86 prompts
Every prompt below is a sentence someone could type — “model a 120x120x4mm VESA adapter plate…”, “model a 2mm mild-steel L-bracket…”. 86 of them, across 25 sectors, each paired with the tool calls a good agent would make and an acceptance block the finished solid has to satisfy: watertight, a volume band, a bounding box, a face-type census and a DFM verdict.
Pass rate by sector
| Sector | Prompts | Accepted | Rate | Mean calls | |
|---|---|---|---|---|---|
| aerospace | 4 | 3 | 75.0% | 2.75 | |
| appliances | 3 | 3 | 100.0% | 7.33 | |
| automotive | 3 | 3 | 100.0% | 4.67 | |
| bikes | 3 | 3 | 100.0% | 3.33 | |
| consumer | 3 | 2 | 66.7% | 7.33 | |
| desk-everyday | 4 | 4 | 100.0% | 3.25 | |
| desktop-robots | 3 | 3 | 100.0% | 5.67 | |
| electronics | 4 | 4 | 100.0% | 6 | |
| furniture | 4 | 4 | 100.0% | 3 | |
| garden-outdoor | 3 | 3 | 100.0% | 3.67 | |
| hardware | 2 | 2 | 100.0% | 2 | |
| jewellery-wearables | 4 | 4 | 100.0% | 1.5 | |
| kitchen-tableware | 3 | 3 | 100.0% | 3.67 | |
| maker-enclosures | 4 | 4 | 100.0% | 6.5 | |
| marine | 3 | 3 | 100.0% | 4 | |
| medical | 4 | 2 | 50.0% | 3.5 | |
| mounts-brackets | 6 | 6 | 100.0% | 3.83 | |
| peripherals | 4 | 3 | 75.0% | 4 | |
| piping | 4 | 4 | 100.0% | 3.5 | |
| power-transmission | 3 | 3 | 100.0% | 1 | |
| robotics | 4 | 4 | 100.0% | 3 | |
| sheet-metal | 4 | 4 | 100.0% | 1 | |
| tooling | 3 | 3 | 100.0% | 3.67 | |
| toys-games | 4 | 4 | 100.0% | 3.75 | |
| vehicles | 0 | 0 | 0.0% | 0 |
Refusal codes hit
Which coded refusals real prompts run into, and the tool that emitted each. A coded refusal with a hint is a good outcome for the server and a bad one for the part: the prompt did not produce geometry.
| Code | Prompts | Tools | Where |
|---|---|---|---|
E_LIMIT | 1 | cad_fillet_edges | aerospace-lightened-aluminium-bracket |
Acceptance checks that failed
For parts that DID build: which half of the acceptance block they missed.
| Check | Prompts | Where |
|---|---|---|
dfm | 4 | consumer-remote-control-shell, medical-handheld-device-shell, medical-sbs-microplate-nest, peripherals-mouse-shell |
Design loops
Real redesign requests, one turn at a time: "keep it under 115 g", "clear the injection-moulding gate", "open the bore until it clears the fit". Each loop is scored on the part it ends with, against the same acceptance blocks as the single-shot bank.
| Loop prompt | Loop | User turns | Replay | Iterations to pass | Refusals during redesign | Live | Turns to pass | Live refusals |
|---|---|---|---|---|---|---|---|---|
| peripherals-mouse-shell-mass-loop | mass_target | 3 | pass | 3 | 0 | not run | not run | not run |
| consumer-clamshell-moldability-loop | moldability | 2 | miss | — | 1 | not run | not run | not run |
| mechanical-shaft-bore-fit-loop | fit_target | 2 | pass | 3 | 0 | not run | not run | not run |
| machining-tray-floor-wall-loop | dfm_wall | 2 | pass | 3 | 0 | not run | not run | not run |
Live agent run
Not included in this build. node benchmarks/text-to-nativecad/run_live.js with ANTHROPIC_API_KEY set scores a real agent against the same acceptance blocks and this section fills in.
A/B matrix
Not included in this build. node benchmarks/text-to-nativecad/ab/run_ab.js --config <ab/configs/a1-refusal-recovery.json> --out out/ab/a1 followed by node benchmarks/text-to-nativecad/score.js --ab out/ab/a1/ab-a1-refusal-recovery.json inlines a cell here once it has run.
Every prompt
| Prompt id | Sector | Result | Calls | ms | Miss |
|---|---|---|---|---|---|
| aerospace-lightened-aluminium-bracket | aerospace | E_LIMIT | 5 | 5325 | build: refused at step 4 (cad_fillet_edges) |
| aerospace-lightening-hole-web | aerospace | pass | 1 | 3759 | |
| aerospace-payload-rail-tap | aerospace | pass | 2 | 12675 | |
| aerospace-quad-arm-motor-mount | aerospace | pass | 3 | 4491 | |
| appliances-control-knob-dshaft | appliances | pass | 8 | 52821 | |
| appliances-lampshade-e27-collar | appliances | pass | 4 | 23772 | |
| appliances-led-heatsink-puck | appliances | pass | 10 | 16396 | |
| automotive-alternator-bracket | automotive | pass | 6 | 32820 | |
| automotive-heat-shield-sheet-metal | automotive | pass | 3 | 33290 | |
| automotive-trim-clip-boss | automotive | pass | 5 | 4066 | |
| bikes-bottle-cage-mount-block | bikes | pass | 2 | 1593 | |
| bikes-seatpost-clamp-collar | bikes | pass | 5 | 35992 | |
| bikes-thru-axle-dropout-plate | bikes | pass | 3 | 4320 | |
| consumer-battery-door | consumer | pass | 5 | 1742 | |
| consumer-clamshell-base | consumer | pass | 6 | 2465 | |
| consumer-remote-control-shell | consumer | miss | 11 | 10740 | dfm: fail |
| desk-everyday-coaster | desk-everyday | pass | 5 | 19945 | |
| desk-everyday-pen-holder | desk-everyday | pass | 2 | 17055 | |
| desk-everyday-phone-stand | desk-everyday | pass | 5 | 3734 | |
| desk-everyday-riser-angled-slot | desk-everyday | pass | 1 | 1050 | |
| desktop-robots-round-robot-head-eye-cutout | desktop-robots | pass | 6 | 72848 | |
| desktop-robots-servo-bracket-sts3215 | desktop-robots | pass | 5 | 17206 | |
| desktop-robots-two-wheel-rover-chassis | desktop-robots | pass | 6 | 34867 | |
| electronics-din-rail-clip-profile | electronics | pass | 1 | 9240 | |
| electronics-finned-heatsink | electronics | pass | 4 | 8065 | |
| electronics-pcb-standoff-rail | electronics | pass | 1 | 5323 | |
| electronics-raspberry-pi-4-enclosure-base | electronics | pass | 18 | 38250 | |
| furniture-desk-top-leg-frame | furniture | pass | 6 | 16493 | |
| furniture-shelf-32mm-system | furniture | pass | 2 | 52136 | |
| furniture-shelf-cable-slot | furniture | pass | 1 | 19040 | |
| furniture-task-chair-seat-pan | furniture | pass | 3 | 7478 | |
| garden-outdoor-birdhouse-box | garden-outdoor | pass | 5 | 5422 | |
| garden-outdoor-post-cap-4x4 | garden-outdoor | pass | 3 | 4794 | |
| garden-outdoor-tapered-nursery-planter | garden-outdoor | pass | 3 | 9090 | |
| hardware-m6-spacer-plate | hardware | pass | 1 | 3894 | |
| hardware-shaft-bore-running-fit | hardware | pass | 3 | 3699 | |
| jewellery-wearables-pendant-tag-bail | jewellery-wearables | pass | 3 | 33201 | |
| jewellery-wearables-plain-band-ring | jewellery-wearables | pass | 1 | 2970 | |
| jewellery-wearables-round-wire-band | jewellery-wearables | pass | 1 | 725 | |
| jewellery-wearables-watch-bezel-ring | jewellery-wearables | pass | 1 | 275 | |
| kitchen-tableware-coffee-mug | kitchen-tableware | pass | 9 | 93458 | |
| kitchen-tableware-cookie-cutter | kitchen-tableware | pass | 1 | 4029 | |
| kitchen-tableware-mixing-bowl | kitchen-tableware | pass | 1 | 971 | |
| maker-enclosures-fan-grill-plate | maker-enclosures | pass | 3 | 25763 | |
| maker-enclosures-lid-display-window | maker-enclosures | pass | 1 | 7868 | |
| maker-enclosures-pi4-case-base | maker-enclosures | pass | 14 | 4398 | |
| maker-enclosures-pico-case | maker-enclosures | pass | 8 | 2747 | |
| marine-316-horn-cleat | marine | pass | 7 | 35259 | |
| marine-through-hull-flange | marine | pass | 3 | 21523 | |
| marine-transom-bracket | marine | pass | 2 | 4873 | |
| medical-handheld-device-shell | medical | miss | 3 | 2533 | dfm: fail |
| medical-manifold-disc-ports | medical | pass | 1 | 1229 | |
| medical-sbs-microplate-nest | medical | miss | 8 | 9544 | dfm: fail |
| medical-tube-rack-grid | medical | pass | 2 | 45924 | |
| mounts-brackets-gopro-3-finger-mount | mounts-brackets | pass | 10 | 14155 | |
| mounts-brackets-l-bracket-gusset | mounts-brackets | pass | 5 | 6501 | |
| mounts-brackets-l-bracket-sheet-fold-scs | mounts-brackets | pass | 1 | 2408 | |
| mounts-brackets-round-boss-tapped | mounts-brackets | pass | 4 | 7188 | |
| mounts-brackets-slotted-adjust-plate | mounts-brackets | pass | 1 | 3005 | |
| mounts-brackets-vesa-100-adapter-plate | mounts-brackets | pass | 2 | 6486 | |
| peripherals-60-keyboard-plate | peripherals | pass | 10 | 31490 | |
| peripherals-cable-puck | peripherals | pass | 1 | 2909 | |
| peripherals-controller-grip | peripherals | pass | 2 | 741 | |
| peripherals-mouse-shell | peripherals | miss | 3 | 1110 | dfm: fail |
| piping-blind-flange-blank | piping | pass | 1 | 831 | |
| piping-flange-nps2-class150 | piping | pass | 5 | 58642 | |
| piping-manifold-block-4port | piping | pass | 4 | 23922 | |
| piping-oring-faceseal-groove-flange | piping | pass | 4 | 31956 | |
| power-transmission-rack-segment-module2 | power-transmission | pass | 1 | 3858 | |
| power-transmission-sprocket-15t-25-chain | power-transmission | pass | 1 | 873 | |
| power-transmission-spur-gear-20t-module2 | power-transmission | pass | 1 | 2663 | |
| robotics-bearing-housing-608 | robotics | pass | 4 | 25964 | |
| robotics-nema17-mount-plate | robotics | pass | 3 | 6772 | |
| robotics-servo-horn-disc | robotics | pass | 1 | 1284 | |
| robotics-shaft-keyway-shoulder-circlip | robotics | pass | 4 | 2749 | |
| sheet-metal-1u-rack-ear | sheet-metal | pass | 1 | 796 | |
| sheet-metal-folded-enclosure-base | sheet-metal | pass | 1 | 2455 | |
| sheet-metal-switch-connector-panel | sheet-metal | pass | 1 | 1225 | |
| sheet-metal-u-bracket-two-bends | sheet-metal | pass | 1 | 1071 | |
| tooling-drill-jig | tooling | pass | 4 | 4477 | |
| tooling-fixture-plate-dowel-tap-grid | tooling | pass | 3 | 13176 | |
| tooling-soft-jaws | tooling | pass | 4 | 35091 | |
| toys-games-brick-2x4 | toys-games | pass | 10 | 7095 | |
| toys-games-chess-pawn-revolve | toys-games | pass | 1 | 1231 | |
| toys-games-die-16mm-pips | toys-games | pass | 3 | 5852 | |
| toys-games-fidget-spinner-body | toys-games | pass | 1 | 2873 |
How to reproduce it
The bank, the runners and this page's input are all in the repository:
node benchmarks/text-to-nativecad/run_replay.js --out benchmarks/text-to-nativecad/baseline.json
node benchmarks/text-to-nativecad/score.js
Prompt bank version 2026-10-08, Node v24.13.0. Recipes marked pack:… are copied from the manufacturing knowledge packs under data/dfm/packs; the rest are hand-authored, and every round feature in those uses the native circle/slot/rect profile primitives rather than a sampled polygon.
More demos
All demos →From one sentence to a checked bracket and its STEP file
Type what you need and an AI agent builds the bracket, checks it for manufacture and hands you the STEP file.
L-bracket drawingThe same bracket, as a dimensioned shop drawing
The bracket arrives with a dimensioned drawing sheet, ready to send with the STEP file.
Drawings to STEP
Manufacturing drawings in, STEP out, checked against the original
Real manufacturing drawings become exact 3D models, each matched against the official model.
6 of 6 NIST parts matched V12 engine
A V12 engine you can run, slice and take apart
Every crank, rod, cam and valve is a real solid, turning in your browser from one dial: the crank angle.
361 parts · 2,850 placementsBuild your own part with NativeCAD
Free for education and makers with a registered key · $200 per seat per year · Kernel SDK for products.