Its weakest category beat their strongest.
In a blind evaluation run by SimulQ, on 100 questions graded on a 10-point rubric, SimulQ's adjudicated verdict scored 9.53 against 7.79 for the average individual model. Its lowest score, 9.19 on dissent engagement, is above the individual models' best of 8.08.
Results
| Dimension | Individual model average | SimulQ | Difference |
|---|---|---|---|
| Dissent engagement | 7.38 | 9.19 | +24.5% |
| Completeness | 8.08 | 9.68 | +19.8% |
| Actionability | 7.91 | 9.71 | +22.7% |
| Combined | 7.79 | 9.53 | +22.3% |
Scored 0–10. Higher is better.
Across every dimension measured, the adjudicated verdict scored higher than the average individual model, and the two sets of scores do not overlap: SimulQ's lowest dimension (9.19) sits above the individual models' highest (8.08).
How it was run
- Run by SimulQ as a blind evaluation.
- 100 questions, each answered by five AI systems: the four panel models, answering independently, and SimulQ's adjudicated verdict.
- The individual-model figure is the mean of the four panel models answering the same question independently.
- Blind grading. Verdicts and raw answers were anonymised and randomly ordered before grading.
- A 10-point rubric. The published dimensions are dissent engagement, completeness and actionability, with a combined score.
To see what the adjudicated verdict is, read how SimulQ works.