Read the explanation
Battle AI system

How Battle AI regret drives bounded calibration

Turn and preview regret expose different decision errors, bounded candidate changes rerun the full visible evidence, and only a passing candidate may cross the sealed promotion gate.

Read the explanationReviewed · PNG download · Updated 21 October 2026
How Battle AI regret drives bounded calibration: stages and decision branches, explained in text below.
Two distinct regret rails · uncertainty-aware gates · one-shot sealed evidence · Read the equivalent explanation below, or open the PNG to inspect every label.

Scope and inputs

Intended reader: Competitive players and maintainers who need a precise map from visible AI mistakes to a qualified policy change.

Two distinct regret rails · uncertainty-aware gates · one-shot sealed evidence.

Follow the stages below. Branches name their destination; a loop returns to the named stage.

Stages and branches

  1. Freeze Evidence Rules

  2. Measure Turn Regret

    Selected-versus-best turn value gap.

  3. Measure Preview Regret

    Bring and lead gaps stay distinct.

  4. Challenge the Evidence

    Check RNG, cutoff, leakage, and evaluator.

  5. Is the Mistake Statistically Clear?

  6. Keep Counterexample

  7. Change One Policy Mechanism

  8. Rerun Every Visible Gate

  9. Do All Visible Gates Pass?

  10. Open Holdouts Once

    Frozen profile and schedule only.

  11. Does Sealed Evidence Pass?

  12. Promote One Frozen Profile

  13. Reject and Start Fresh

    Never tune on failed sealed evidence.

Outputs

  • Keep Counterexample
  • Promote One Frozen Profile
  • Reject and Start Fresh

Limits and uncertainty

  • Turn regret ≠ preview regret · Holdouts open once · Fitter suggestions ≠ qualification
  • RNG evidence is limited to declared paired seed schedules and orientations.
  • A six-root deep frontier identifies the best evaluated root, not a global best.
  • Terminal share and horizon stability expose bounded-cutoff artifacts.
  • Known-information hypotheses must remain acting-side scoped; leakage invalidates evidence.
  • Mirrored or paired orientations test side effects only on the declared schedule.
  • Regret is relative to the frozen terminal-or-cutoff value function.

What this does not prove

A bounded evaluation is not perfect-play proof or a guarantee of results outside its declared protocol.

Glossary

regret
non-negative value gap between the best evaluated and selected choice
separated
the best choice's lower 95% bound exceeds the selected choice's upper 95% bound
frontier
the declared bounded subset receiving deeper continuation
holdout
a separately named partition that remains unread until the promotion gate

Sources and review

Reviewed for publication . Source observations retain their original dates.

  • Turn regret · observed 2026-08-27
  • Preview regret · observed 2026-08-27
  • Candidate fitter · observed 2026-08-27
  • Production profile · observed 2026-08-27
  • Campaign ledger · observed 2026-07-31
  • Holdout record · observed 2026-07-31
Source references and revision

Source SHA-256: 5e3cec64a0ede8c4e4550dd5b1d691bd0793b8ef2b1c6311469dd4fb86eaea79

Originating chart: POK-100

  • Turn regret: scripts/benchmarkChampionsDecisionRegret.ts
  • Preview regret: scripts/benchmarkChampionsLeadRegret.ts
  • Candidate fitter: scripts/calibrateChampionsBattlePolicyVnext.ts
  • Production profile: src/server/battle/championsBattlePolicyProfile.ts
  • Campaign ledger: docs/archive/battle-analysis-simulation-plan-2026-07-20.md#f26
  • Holdout record: docs/archive/battle-ai-policy-vnext-plan-2026-07-27.md