Methodology

Benchmarks

We cite public benchmarks for hardware-engineering AI and show a tool’s result only when it is published, linked to its source and verified.

Verified results

None of the tools listed here has a published, verified score on the benchmarks below yet. Results will appear once they are sourced and checked.

Simulation & Analysis

Measures: Whether a model can do real hardware-engineering work in CAE software. The first release has 77 tasks across combustion (21), battery (34), CFD (13) and packaging (9); each task is a container with a prompt, a reference solution and a programmatic verifier.
Scoring: Per task, the reported value either lands inside a tolerance band or it does not. The headline score (0–1) is the equal-weighted mean of the four per-domain means, shown with a 95% interval. All models run through the same agent scaffold (terminus-2), one trial per task.
Ranks: Models (run through a fixed agent scaffold), not end-user products

Electronics Design

Measures: Whether tool-calling agents on locally deployed open-source models can automate dependency-ordered hardware design workflows (creating components, adding ports, wiring connections) through an MCP server. Suites cover single-operation edits, multi-step dependency chains, invalid requests, misspelled prompts and multi-server tool contexts.
Scoring: Agent call sequences are compared with annotated expected calls. Expected Call Coverage (ECC, higher is better) is the share of expected calls executed correctly; Extraneous Valid Call Ratio and Tool Failure Rate track over-generation and failing calls. Seven open-source models are evaluated.
Ranks: Open-source models behind an MCP agent, not end-user products

We publish a number only when it appears on a public page we link to, and a person has checked it against that page.
Each benchmark gets its own table, so a column never mixes units or scoring methods.
Unverified, unsourced or placeholder numbers are never shown, not even as “pending”.
These benchmarks mostly score models and agents, while this directory lists products. A model’s score is not a product’s score unless the source says the product was tested.