Diaspore

All documents · source file docs/methodology/deal-v0.1-results.md · changed 2026-11-04 · sha256 a6c2f977a5e5

deal-v0.1 results (IDEA-014 M5-M7)

Run 2026-11-03 under the spec committed at 7d8d61d, after a pre-result audit with two re-checks (docs/audits/idea014-hindcast-preresult-audit.md). Files: data/processed/acquire/r2.json, r3.json, noise.json. All on past data; nothing here is a skill claim.

Short version

R2: which deals worked (not which deals to do). Result: cannot test

horizondealspositivescensoredminimum detectable AUC
36 months694550.920
60 months482761.000

With 4 positives, any AUC is noise, so they are in r2.json as description only and support no statement. Positives are probably undercounted, because acquired programmes can move to the acquirer's name in the registry; that was not measured. A usable label needs intervention names (R4, a free public fetch on the owner queue).

R3: does public evidence separate acquired from not-acquired? Descriptive; label noise unmeasured

Among trial-bearing companies listed at T, acquired by an SEC-registered acquirer in a parsed filing within 36 months. T = 2020-12-31 (33 of 406) and 2022-12-31 (52 of 490). This measures acquisition, not value: being acquired is not the same as being worth buying.

scorerwithin-date mean AUC95% CI, company-clusteredT=2020T=2022
deal-v0.1 rubric (identical without economics)0.5780.503 to 0.6550.5960.560
trial count (ruled citation-rank substitute)0.5830.505 to 0.6570.6090.557
stage only0.5590.488 to 0.6250.5960.521
random0.4890.422 to 0.5580.4430.535
blind LLM on anonymised packs0.586not computed0.6120.560

Rubric minus trial count: -0.005 (95% CI -0.079 to 0.069). No measurable edge over a trial count; the interval cannot exclude about 0.07 AUC either way. By the plan's rule this is the finding; deal-v0.1 is not tuned, and any change is deal-v0.2.

Secondaries look stronger and should not be read that way: trial-bearing any 0.714, all companies with a dated trace 0.715. Those pools hold private companies that the ledger cannot label as acquired. A plausible, untested explanation for the higher figures is that they partly measure "was public".

Limits that bind every number above

  1. Trials link to companies by 2026 sponsor name, not point in time. The direction of the bias is not guaranteed. Real fix: R4.
  2. Label noise was unmeasured at this run; **update 2026-11-04: D3 has since run, and an estimated 44 percent of true acquisitions are missing (95 percent interval 9 to 72), see deal-r4-d3-results.md. R3 stays descriptive.** At this run, the 30-company check (D3) had not been done, which alone kept R3 descriptive. Negatives with a delisted-acquired or delisted-unknown row in the window: 8 at 2020 and 31 at 2022, against 33 and 52 positives, so missed acquisitions could be a large share of the true events.
  3. exit_date is deal completion, so some positives were already announced at T.
  4. There are no declined deals in public data. Nothing here observes a deal someone chose not to do.
  5. Blind LLM, the row in the table is not a clean comparator. Route: path-mode subagent, blind by instruction, not by tool restriction. The answer key sat in its folder during the run; its tool log shows 16 reads of the pack file, and its 3 shell commands were read and neither list nor glob that folder. The model's training data postdates these deals. It reported 6 of 965 packs as recognisable; that is its own report, not a measured rate, and unrecognised packs may still be recalled. All 6 are in R2 (AUC 0.681 with them, 0.644 without; description only). The spec required a 20-pack check for sponsor or product strings; that was not done. An automated check ran over all packs for words of 5+ letters from the company's own name only (3 hits, all condition words), so the spec's condition was relaxed, not met: product and acquirer names were not tested. R2 and R3 packs were scored in one batch and 26 companies appear in both. It is reported because hard rule 6 asks for this baseline; its 0.586 is not evidence that it beats the rubric.
  6. Comparable values are filed lower bounds. Stage-cell floors span orders of magnitude, and the frozen bands call phase 2 plus one recent trial "strong", so "buy" fires for most typed prices in a populated cell. Treat the verdict as a framing device for now.

Deviations from the spec, all logged in decisions.md

Registry termination proxy not run. Holm not applied, since descriptive mode runs no test. All-companies secondary narrowed to a dated trace. Headline moved to a within-date mean before any number was computed. Leak check covered all packs by own-name words rather than a 20-pack sample.