Crucible — computational invention and verification
A cheap, repeatable computational testbed for a question that has to be answered before an AI system can be trusted to run its own search: does the model's own confidence actually tell you anything about whether its idea will work?
The question
Does a model's own confidence carry information about whether its candidate will pass an objective verifier — beyond what simple base rates and external features of the task already predict?
Why it was not knowable in advance
An agent that decides when to keep searching, and when to stop, has to know when its own output is likely to fail. Whether that self-signal contains anything of its own, or merely re-encodes how hard the task is, could not be settled from prior work. It had to be measured against frozen baselines.
How it was tested
The testbed is LLM-guided combinatorial codeword search checked by a deterministic verifier — not source-code generation. Every candidate is judged mechanically, which makes the outcome label cheap, objective and repeatable, and lets the study concentrate on measurement instead of on human judgements of quality.
What we found
- We did not establish that the model's confidence carries independent predictive value. A predictor built only from external features of the task, with no access to the model's internal signal at all, did at least as well.
- That result is not strong enough to be called a refutation either. The comparison was not a pre-registered confirmatory test, so the honest reading is narrower: this study did not establish the effect in either direction.
- The pre-registered primary metric turned out to be undefined in practice — its denominator collapsed to near zero — and had to be revised and re-run. Finding that out early was most of the value.
- The useful output is a list of measurement problems that have to be solved before autonomous search control can be studied honestly at all.
What this does not show
- That an AI invented a new code, or exceeded human inventors.
- That confidence-guided search improves performance.
- That the agent has been shown to lack self-knowledge — a result this weak does not support that conclusion any more than the opposite one.
- That the results carry over to industrial design or physical manufacturing.
The study was run to answer a measurement question, not to demonstrate a capability. Its value is that it exposed problems that have to be solved before autonomous search control can be studied honestly — including how a primary metric can quietly degenerate, and how easily a signal that looks like self-knowledge turns out to be task difficulty wearing a disguise.
Research article
Qualifying Pre-Evaluation Confidence in LLM-Guided Combinatorial Codeword Search: An Evidence-Bound Measurement Audit
Read the full article →Questions, or think we got something wrong? Write to hello@1962lab.com .