vvs/sim-numerics
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Revision 1, step 1: one scoring criterion, the correct order, and the internal inconsistencies.

Browse Source 
Point 1. The base step's order is Muller's 1.839, not 2: the derivative of the parabola through
the last three points errs by f'''/6 (x_n - x_{n-1})(x_n - x_{n-2}) = O(e_{n-1} e_{n-2}).
order_high_precision.py (3000 digits, to 1e-2900): 1.840 on cos x - x, e^x - 2, x^3 - x - 2,
sin x - 1/2; 2.000 only on x^2 - 2; two-point 1.618. The double-precision medians (2.000, 1.94)
are removed from the text.

Point 3. Every protocol number is re-scored under the criterion of section 9 - a sign change
within 1e-6 or f(x) == 0. rescore_uniform.py re-runs all 2779 distinct configurations of the 25
protocol sweeps on APS and CHA (the runs did not keep x) -> sweep_uniform.json, same row format;
it reproduces the APS/CHA lines of Table 10 exactly (built-in check). verify_article.py,
audit_similarity_places.py and the gap diagnostics read the re-scored data; the multiplier and
endowment measurements score by the same criterion. Table 7 (two criteria side by side) is gone;
7.3 now explains why the criterion is a sign change. Consequences stated in the text: the base
rule 2634 / 2357 against Chandrupatla 2595 / 1505 (1.5% behind on APS, no longer "level"); A'
2259 / 1074 (12.9% and 28.6% below Chandrupatla); the defect reading 2351 is second to the
shape reading 2337, not the record; the remembered analogy is level with no regime (2633 vs
2634) under this criterion; Table 9a's range is 19% to 425%.

Point 8. report_sweep.head_to_head counts every problem, failures charged (it counted only the
problems both methods solved, which is why Table 8 rows summed to 153 and 40). Family-level
comparisons added (A' vs Chandrupatla 8/1/6 of 15 and 7/0/2 of 9 families, not significant), and
an independence paragraph in section 9.

Point 11. sweep_itp.py: ITP (recommended constants and per-problem oracle) on all six corpora ->
Table 10; behind A' on every corpus, 0 bound violations.

Points 7, 9, 10 and the internals: the 2x bound is ours, not the rivals' promise; the law
predicts the group order only; "outside the proven bracket" explained (narrowest bracket of all
observed signs); 7.4's fun1 paragraph corrected (five costs under safe; the unguarded numbers
belong to fun1.4, fun1.5 and to fun3.1, fun5.1 separately); Table 1's last cell; the corpus-4
composition and odd multiplicities stated, also in the abstract.

build_tex.py: pdflatex output decoded with errors="replace" (a UTF-8 character cut by TeX's log
wrap crashed the build). Table 10 column widths rebalanced.

Checks: verify_article on the server 500 OK / 0 BAD (467 existing with the new values, 33 new);
regression x2 on the server 1232 passed, 0 failed; PDF 28 pages, 0 errors, 0 undefined, 0 overfull.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
vvs
2026-09-22 07:01:36 +01:00
co-authored by Claude Opus 5
parent eeb9102d47
commit 862feaff86
29 changed files with 2529 additions and 889 deletions
Download Patch File Download Diff File Expand all files Collapse all files
verify_article.py
+304 -177
View File
@@ -37,8 +37,7 @@ def row(rows, setname, lbl):
def main():
with open("sweep_full.json", encoding="utf8") as fh:
rows = json.load(fh)
rows = _load("sweep_full.json")
# Guard against quoting places from a field that is missing a competitor.
# It has already happened once: before Chandrupatla's own method was
@@ -55,28 +54,28 @@ def main():
# ---- the proposed method: projection, one declared constant ----------
print("\nThe proposed method (projection, multiplier declared):")
for setname, pl, tot, sol in (("APS", 3, 2806, 153),
("CHA", 45, 3220, 40)):
for setname, pl, tot, sol in (("APS", 3, 2634, 154),
("CHA", 49, 2357, 45)):
d = row(rows, setname, PROPOSED)
check(f"{setname} place", place(rows, setname, PROPOSED), pl)
check(f"{setname} total", d["total"], tot)
check(f"{setname} solved", d["solved"], sol)
check("both sets", row(rows, "APS", PROPOSED)["total"]
+ row(rows, "CHA", PROPOSED)["total"], 6026)
+ row(rows, "CHA", PROPOSED)["total"], 4991)
print("\nHead to head of the proposed method:")
for setname, opp, want in [
("APS", "chandrupatla", (87, 24, 42)),
("APS", "toms748", (86, 16, 51)),
("APS", "brent", (102, 11, 40)),
("APS", "itp", (132, 3, 18)),
("APS", "itp_oracle", (121, 4, 28)),
("APS", "bisection", (152, 1, 0)),
("CHA", "chandrupatla", (9, 5, 26)),
("CHA", "toms748", (31, 1, 8)),
("CHA", "brent", (30, 4, 4)),
("CHA", "itp", (17, 0, 23)),
("CHA", "itp_oracle", (16, 0, 24))]:
("APS", "chandrupatla", (87, 24, 43)),
("APS", "toms748", (86, 16, 52)),
("APS", "brent", (102, 11, 41)),
("APS", "itp", (132, 3, 19)),
("APS", "itp_oracle", (121, 4, 29)),
("APS", "bisection", (152, 1, 1)),
("CHA", "chandrupatla", (11, 5, 29)),
("CHA", "toms748", (33, 1, 11)),
("CHA", "brent", (34, 4, 7)),
("CHA", "itp", (18, 1, 26)),
("CHA", "itp_oracle", (16, 1, 28))]:
w, t, l, _ = head_to_head(rows, PROPOSED, opp, setname)
check(f"{setname} vs {opp}", (w, t, l), want)
@@ -85,48 +84,48 @@ def main():
k, n = min(w, l), w + l
return min(1.0, 2.0 * sum(_m.comb(n, i) for i in range(k + 1)) / 2.0 ** n)
w, _t, l, _ = head_to_head(rows, PROPOSED, "chandrupatla", "APS")
check("APS vs Chandrupatla sign test p < 1e-4", sign_p(w, l) < 1e-4, True)
check("APS vs Chandrupatla sign test p (2 digits)", float(f"{sign_p(w, l):.2g}"), 0.00014)
w, _t, l, _ = head_to_head(rows, PROPOSED, "chandrupatla", "CHA")
check("CHA vs Chandrupatla sign test p (rounded)", round(sign_p(w, l), 3), 0.006)
# ---- the constant-free variant ---------------------------------------
print("\nThe constant-free variant (earned projection):")
for setname, pl, tot in (("APS", 29, 3568), ("CHA", 10, 3020)):
for setname, pl, tot in (("APS", 25, 3377), ("CHA", 9, 2073)):
check(f"credit {setname} place", place(rows, setname, CREDIT), pl)
check(f"credit {setname} total", row(rows, setname, CREDIT)["total"], tot)
check("credit both sets", row(rows, "APS", CREDIT)["total"]
+ row(rows, "CHA", CREDIT)["total"], 6588)
check("credit costs 762 more on the canonical set",
row(rows, "APS", CREDIT)["total"] - row(rows, "APS", PROPOSED)["total"], 762)
for setname, opp, want in [("APS", "chandrupatla", (79, 3, 71)),
("APS", "toms748", (77, 9, 67)),
("APS", "brent", (78, 5, 70))]:
+ row(rows, "CHA", CREDIT)["total"], 5450)
check("credit costs 743 more on the canonical set",
row(rows, "APS", CREDIT)["total"] - row(rows, "APS", PROPOSED)["total"], 743)
for setname, opp, want in [("APS", "chandrupatla", (79, 3, 72)),
("APS", "toms748", (78, 9, 67)),
("APS", "brent", (79, 5, 70))]:
w, t, l, _ = head_to_head(rows, CREDIT, opp, setname)
check(f"credit {setname} vs {opp}", (w, t, l), want)
w, _t, l, _ = head_to_head(rows, CREDIT, "chandrupatla", "APS")
check("credit APS vs Chandrupatla sign test p (rounded)", round(sign_p(w, l), 2), 0.57)
check("credit APS vs Chandrupatla sign test p (rounded)", round(sign_p(w, l), 2), 0.63)
# ---- what the earlier, order-destroying guards cost --------------------
print("\nThe guards that replace the step instead of projecting it:")
check("APS unguarded", row(rows, "APS", "pure.ratio.derived3.none")["total"],
2806)
2634)
cha_none = row(rows, "CHA", "pure.ratio.derived3.none")
check("CHA unguarded total", cha_none["total"], 4104)
check("CHA unguarded solved", cha_none["solved"], 35)
check("CHA unguarded total", cha_none["total"], 3391)
check("CHA unguarded solved", cha_none["solved"], 39)
check("CHA unguarded bound violations", cha_none["over"], 10)
check("APS bracket-guard", row(rows, "APS", "pure.ratio.derived3.bracket")["total"],
7260)
7068)
check("APS both-guard", row(rows, "APS", "pure.ratio.derived3.both")["total"],
7239)
7046)
# ---- the field --------------------------------------------------------
print("\nAlefeld-Potra-Shi:")
check("APS winner", rank(rows, "APS")[0]["label"], "chandrupatla")
for lbl, pl, tot in [("chandrupatla", 1, 2787), ("toms748", 4, 2820),
("brent", 5, 2889), ("dekker", 12, 3158),
("itp_oracle", 13, 3260), ("ridders", 20, 3313),
("itp", 34, 3957), ("illinois", 73, 7045),
("bisection", 79, 7530), ("regula_falsi", 244, 10560)]:
for lbl, pl, tot in [("chandrupatla", 1, 2595), ("toms748", 4, 2636),
("brent", 5, 2707), ("dekker", 232, 3158),
("itp_oracle", 11, 3068), ("ridders", 15, 3124),
("itp", 29, 3767), ("illinois", 237, 7045),
("bisection", 73, 7338), ("regula_falsi", 244, 10560)]:
check(f"APS {lbl} place", place(rows, "APS", lbl), pl)
check(f"APS {lbl} total", row(rows, "APS", lbl)["total"], tot)
check("APS regula falsi solved", row(rows, "APS", "regula_falsi")["solved"],
@@ -134,31 +133,31 @@ def main():
print("\nChandrupatla:")
top = rank(rows, "CHA")[0]
check("CHA rank 1 label", top["label"], "pure.sqrt.bracket.project")
check("CHA rank 1 solves one more than anyone else", top["CHA"]["solved"], 41)
for lbl, pl, tot in [("chandrupatla", 2, 2441), ("itp_oracle", 3, 2647),
("itp", 6, 2896), ("bisection", 16, 3090),
("toms748", 159, 4000), ("brent", 227, 4115)]:
check("CHA rank 1 label", top["label"], "chandrupatla")
check("CHA rank 1 solves all 45", top["CHA"]["solved"], 45)
for lbl, pl, tot in [("chandrupatla", 1, 1505), ("itp_oracle", 2, 1675),
("itp", 5, 1946), ("bisection", 15, 2136),
("toms748", 158, 3088), ("brent", 227, 3238)]:
check(f"CHA {lbl} place", place(rows, "CHA", lbl), pl)
check(f"CHA {lbl} total", row(rows, "CHA", lbl)["total"], tot)
check("CHA brent solved", row(rows, "CHA", "brent")["solved"], 38)
check("CHA brent solved", row(rows, "CHA", "brent")["solved"], 43)
ours_both = (row(rows, "APS", PROPOSED)["total"]
+ row(rows, "CHA", PROPOSED)["total"])
cha_both = (row(rows, "APS", "chandrupatla")["total"]
+ row(rows, "CHA", "chandrupatla")["total"])
check("chandrupatla, both sets", cha_both, 5228)
check("chandrupatla, both sets", cha_both, 4100)
check("Chandrupatla beats us on both sets", cha_both < ours_both, True)
# ---- the regime findings ---------------------------------------------
print("\nNegative results, the regime claims:")
check("CHA cbrt over the whole set",
row(rows, "CHA", "pure.cbrt.derived3.project")["total"], 4109)
row(rows, "CHA", "pure.cbrt.derived3.project")["total"], 3204)
switched = (row(rows, "APS", PROJ)["total"]
+ row(rows, "CHA", "pure.cbrt.derived3.project")["total"])
single = row(rows, "APS", PROJ)["total"] + row(rows, "CHA", PROJ)["total"]
check("single fixed choice, both sets", single, 6026)
check("switch by benchmark set, both sets", switched, 6915)
check("single fixed choice, both sets", single, 4991)
check("switch by benchmark set, both sets", switched, 5838)
check("switching is worse than not switching", switched > single, True)
for pid, ours_n, cb_n in (("fun4.1", 89, 32), ("fun4.2", 93, 32),
("fun4.3", 113, 40), ("fun4.4", 125, 47),
@@ -172,10 +171,10 @@ def main():
print("\nThe law across the grid:")
lin = ("ratio", "rational", "exp_abs")
quad = ("rat_sq", "exp_sq", "cosine")
want = {("derived3", "lin"): 6893, ("derived3", "quad"): 22315,
("derived", "lin"): 6981, ("derived", "quad"): 22206,
("bracket", "lin"): 10279, ("bracket", "quad"): 19127,
("start", "lin"): 16102, ("start", "quad"): 15345}
want = {("derived3", "lin"): 5764, ("derived3", "quad"): 21200,
("derived", "lin"): 5854, ("derived", "quad"): 21089,
("bracket", "lin"): 9149, ("bracket", "quad"): 18015,
("start", "lin"): 14983, ("start", "quad"): 14228}
for scale in ("derived3", "derived", "bracket", "start"):
for group, gname in ((lin, "lin"), (quad, "quad")):
tot = cnt = 0
@@ -204,15 +203,15 @@ def main():
[s for s, _ in sorted(best_by_sim.items(), key=lambda kv: kv[1])],
["ratio", "exp_abs", "rational", "ratio_soft", "sqrt", "cbrt",
"exp_sq", "rat_sq", "cosine"])
for sim, v in [("ratio", 6560), ("exp_abs", 6895), ("rational", 7118),
("ratio_soft", 9578), ("sqrt", 10546), ("cbrt", 12002),
("exp_sq", 14307), ("rat_sq", 15102), ("cosine", 16394)]:
for sim, v in [("ratio", 5422), ("exp_abs", 5772), ("rational", 5993),
("ratio_soft", 8455), ("sqrt", 9411), ("cbrt", 10874),
("exp_sq", 13187), ("rat_sq", 13990), ("cosine", 15272)]:
check(f"similarity {sim} best", best_by_sim[sim], v)
check("scale ranking",
[s for s, _ in sorted(best_by_scale.items(), key=lambda kv: kv[1])],
["derived3", "derived", "bracket", "start"])
for sc, v in [("derived3", 6560), ("derived", 6702), ("bracket", 9107),
("start", 14540)]:
for sc, v in [("derived3", 5422), ("derived", 5564), ("bracket", 7972),
("start", 13423)]:
check(f"scale {sc} best", best_by_scale[sc], v)
verify_placement_modes()
@@ -221,6 +220,7 @@ def main():
verify_gap_decomposition()
verify_safeguard_and_tables()
verify_flat_regime_and_oos()
verify_revision1()
print()
if FAILS:
@@ -239,32 +239,32 @@ def main():
# --------------------------------------------------------------------------
PLACEMENT = {
# file label APS CHA
"sweep_slope.json": ("slope.ratio.derived3.project", 2848, 3114),
"sweep_points.json": ("pure+shape+slope.ratio.derived3.safe", 2527, 3126),
"sweep_direction.json": ("pure+dissim+slope.ratio.derived3.project", 2726, 2799),
"sweep_combo.json": ("pure+shape+dissim+slope.ratio.derived3.safe", 2884, 2871),
"sweep_gate.json": ("progress.ratio.derived.project", 4449, 3047),
"sweep_log.json": ("log.ratio_soft.derived.safe", 6940, 3541),
"sweep_slope.json": ("slope.ratio.derived3.project", 2663, 2170),
"sweep_points.json": ("pure+shape+slope.ratio.derived3.safe", 2337, 2177),
"sweep_direction.json": ("pure+dissim+slope.ratio.derived3.project", 2539, 1864),
"sweep_combo.json": ("pure+shape+dissim+slope.ratio.derived3.safe", 2695, 1928),
"sweep_gate.json": ("progress.ratio.derived.project", 4258, 2093),
"sweep_log.json": ("log.ratio_soft.derived.safe", 6749, 2598),
}
# The per-place table of section "the similarity function belongs to the place":
# seven PAIRED comparisons in sweep_order.json. Checked as pairs, because a pair is
# what the pre-registration licensed and a single cell would prove nothing.
ORDER_PAIRS = {
# guard (APS m=1, APS measured, CHA m=1, CHA measured)
"both": (4522, 3675, 3406, 3241),
"bracket": (4606, 3542, 3602, 3359),
"credit": (5190, 4686, 3013, 2884),
"project": (6932, 6364, 4789, 4554),
"safe": (5987, 5229, 4134, 3954),
"step": (6435, 5551, 4287, 4068),
"none": (10676, 10043, 8265, 7794),
"both": (4331, 3485, 2469, 2306),
"bracket": (4418, 3352, 2671, 2428),
"credit": (4999, 4495, 2064, 1933),
"project": (6783, 6176, 4057, 3751),
"safe": (5803, 5040, 3218, 3032),
"step": (6251, 5362, 3371, 3146),
"none": (10676, 9855, 8265, 7424),
}
# the negative item: the order does not cure false position
ORDER_FALSEPOS = {("rational", "far", "none", "none"): (127, 10676),
("rational", "far", "none", "order"): (128, 10043)}
("rational", "far", "none", "order"): (129, 9855)}
# and the harmful direction, quoted in the same subsection
ORDER_SLOPE_HARM = {("ratio", "derived3", "none", "none"): 2806,
("ratio", "derived3", "none", "order"): 2854}
ORDER_SLOPE_HARM = {("ratio", "derived3", "none", "none"): 2634,
("ratio", "derived3", "none", "order"): 2703}
def verify_per_place():
@@ -273,8 +273,7 @@ def verify_per_place():
print(" !! sweep_order.json absent — section 'belongs to the place' unverifiable")
FAILS.append("missing sweep_order.json")
return
with open("sweep_order.json", encoding="utf8") as fh:
rows = json.load(fh)
rows = _load("sweep_order.json")
def cell(sim, sc, gu, ex):
for r in rows:
@@ -318,11 +317,11 @@ def verify_per_place():
DECISION_CELLS = {
# (similarity, prestep, guard): {set: total}
("ratio", "band", "project"): {"APS": 5432, "CHA": 3059},
("ratio", "band", "credit"): {"CHA": 2892},
("ratio", "none", "project"): {"APS": 2806},
("ratio", "defect", "project"): {"APS": 2523, "CHA": 3210},
("ratio_soft", "simpair", "project"): {"CHA": 2897},
("ratio", "band", "project"): {"APS": 5242, "CHA": 2101},
("ratio", "band", "credit"): {"CHA": 1942},
("ratio", "none", "project"): {"APS": 2634},
("ratio", "defect", "project"): {"APS": 2351, "CHA": 2346},
("ratio_soft", "simpair", "project"): {"CHA": 1941},
}
DECISION_RATES = {"band": 63.5, "defect": 6.6}
@@ -333,8 +332,7 @@ def verify_decision_place():
print(" !! sweep_decision.json absent — section 'decision place' unverifiable")
FAILS.append("missing sweep_decision.json")
return
with open("sweep_decision.json", encoding="utf8") as fh:
rows = json.load(fh)
rows = _load("sweep_decision.json")
by = {(r["spec"][1], r["spec"][5], r["spec"][3]): r for r in rows}
for k, sets in DECISION_CELLS.items():
r = by.get(k)
@@ -345,9 +343,9 @@ def verify_decision_place():
for setname, want in sets.items():
check(f"{'.'.join(k)} {setname}", r[setname]["total"], want)
d = by[("ratio", "defect", "project")]
check("defect APS solved", d["APS"]["solved"], 153)
check("defect both sets", d["APS"]["total"] + d["CHA"]["total"], 5733)
check("defect is the new canonical best (< 2527)", d["APS"]["total"] < 2527, True)
check("defect APS solved", d["APS"]["solved"], 154)
check("defect both sets", d["APS"]["total"] + d["CHA"]["total"], 4697)
check("defect second to the shape reading (2337 < defect)", 2337 < d["APS"]["total"], True)
fam = [r for r in rows if r["spec"][3] in ("project", "safe", "credit")]
check("decision grid guarantee violations", sum(r["APS"]["over"] + r["CHA"]["over"]
for r in fam), 0)
@@ -389,7 +387,7 @@ def verify_gap_decomposition():
ck = diag_model_order_checks.main()
fl = diag_flat_detectable.main()
per = mo["per_problem"]
check("problems where we lose", len(mo["lose"]), 33)
check("problems where we lose", len(mo["lose"]), 35)
check("fun1.4 our bracket halvings/eval", round(per["fun1.4"][2], 2), 0.62)
check("fun1.4 IQI from our points", round(per["fun1.4"][3], 2), 10.37)
flat = [p for p in per if p.split(".")[0] in ("fun3", "fun4", "fun5", "fun6")]
@@ -397,13 +395,13 @@ def verify_gap_decomposition():
check("flat: ours max", round(max(per[p][2] for p in flat), 2), 0.75)
check("flat: IQI min", round(min(per[p][3] for p in flat), 2), 0.01)
check("flat: IQI max", round(max(per[p][3] for p in flat), 2), 0.17)
check("non-degenerate states with one negative weight", mo["neg_ok"], 428)
check("bracketing states in total", mo["neg_all"], 431)
check("non-degenerate states with one negative weight", int(mo["neg_ok"]), 431)
check("bracketing states in total", mo["neg_all"], 434)
check("every exception is an underflowed zero weight",
all(any(w == 0.0 for w in ws) for _p, _f, ws in ck["exceptions"]), True)
fam = ck["fam"]
gap = {k: v[2] - v[3] for k, v in fam.items()}
check("total gap on the losing problems", sum(gap.values()), 256)
check("total gap on the losing problems", sum(gap.values()), 262)
check("flat families' gap", sum(gap[k] for k in ("fun3", "fun4", "fun5", "fun6")), 172)
check("fun1 gap", gap["fun1"], 58)
for k, want in (("fun3", 92), ("fun4", 96), ("fun5", 100), ("fun6", 100)):
@@ -411,11 +409,10 @@ def verify_gap_decomposition():
check(f"Chandrupatla bisect share {k} %", round(100.0 * b / (b + q)), want)
for k, want in (("fun4", 297), ("fun5", 233), ("fun6", 233)):
check(f"Chandrupatla total {k}", fam[k][3], want)
check("m>=2 on canonical states %", round(fl["APS (canonical)"][2], 1), 0.2)
check("m>=2 on canonical states %", float(round(fl["APS (canonical)"][2], 1)), 0.3)
check("m>=2 on flat states %", round(fl["CHA flat fun3-6"][2], 1), 0.0)
# the band per family, from the Linux sweep
with open("sweep_decision.json", encoding="utf8") as fh:
rows = json.load(fh)
rows = _load("sweep_decision.json")
by = {(r["spec"][1], r["spec"][5], r["spec"][3]): r for r in rows}
base = by[("ratio", "none", "project")]["CHA"]["per"]
band = by[("ratio", "band", "project")]["CHA"]["per"]
@@ -429,11 +426,10 @@ def verify_gap_decomposition():
for k, d in (("fun1", 164), ("fun2", 111), ("fun8", 170), ("fun9", 144)):
check(f"band cost on {k}", fam_total(band, k) - fam_total(base, k), d)
# negative item 11, corrected pairing: the same configuration in x and in log space
with open("sweep_full.json", encoding="utf8") as fh:
full = json.load(fh)
full = _load("sweep_full.json")
xcell = next(r for r in full if r["label"] == "pure.ratio_soft.derived.safe")
check("log item: x-space control APS", xcell["APS"]["total"], 5822)
check("log item: x-space control CHA", xcell["CHA"]["total"], 3695)
check("log item: x-space control APS", xcell["APS"]["total"], 5635)
check("log item: x-space control CHA", xcell["CHA"]["total"], 2759)
def verify_safeguard_and_tables():
@@ -448,51 +444,54 @@ def verify_safeguard_and_tables():
with contextlib.redirect_stdout(io.StringIO()):
bf = measure_budget_factor.main()
rm = measure_records_vs_multiplier.main()
for c, (aps, cha) in ((1.0, (4817, 2997)), (1.25, (2806, 2752)), (1.5, (2806, 2889)),
(2.0, (2806, 3220)), (3.0, (2806, 3756)), (4.0, (2806, 4006))):
for c, (aps, cha) in ((1.0, (4625, 2039)), (1.25, (2626, 1822)), (1.5, (2636, 1991)),
(2.0, (2634, 2357)), (3.0, (2634, 2936)), (4.0, (2634, 3229))):
check(f"multiplier {c} APS", bf[c][0], aps)
check(f"multiplier {c} CHA", bf[c][2], cha)
for c0, (aps, cha) in ((0.1, (5344, 3051)), (0.5, (4656, 3002)), (1.0, (3568, 3020)),
(2.0, (3692, 2712)), (5.0, (3006, 2615))):
for c0, (aps, cha) in ((0.1, (5150, 2097)), (0.5, (4462, 2037)), (1.0, (3377, 2073)),
(2.0, (3502, 1757)), (5.0, (2819, 1666))):
check(f"endowment {c0} APS", bf["endowment"][c0][0], aps)
check(f"endowment {c0} CHA", bf["endowment"][c0][1], cha)
within_one_step = {c0: sum(v) for c0, v in bf["endowment"].items() if c0 <= 1.0}
check("C0=1 cheapest within the one-step promise",
min(within_one_step, key=within_one_step.get), 1.0)
sh = rm["shape reading (2527)"]
check("2527 at multiplier 1.25", sh[1.25][0], 2541)
check("2527 flat from 1.5 to 4", {sh[f][0] for f in (1.5, 2.0, 3.0, 4.0)}, {2527})
check("shape reading at multiplier 1.25", sh[1.25][0], 2351)
check("shape reading flat from 1.5 to 4", {sh[f][0] for f in (1.5, 2.0, 3.0, 4.0)}, {2337})
de = rm["defect reading (2523)"]
check("2523 flat from 1.25 to 4", {de[f][0] for f in (1.25, 1.5, 2.0, 3.0, 4.0)}, {2523})
dv = [de[f][0] for f in (1.25, 1.5, 2.0, 3.0, 4.0)]
check("defect reading from 1.25 to 4: range", (min(dv), max(dv)), (2343, 2353))
print("\nTables 2-3 regenerated:")
with open("sweep_full.json", encoding="utf8") as fh:
rows = json.load(fh)
rows = _load("sweep_full.json")
by = {r["label"]: r for r in rows}
none_ = "pure.ratio.derived3.none"
for setname, lbl, pl in (("APS", PROJ, 3), ("APS", none_, 2), ("CHA", PROJ, 45),
("CHA", none_, 228), ("CHA", "pure.exp_abs.bracket.both", 4)):
for setname, lbl, pl in (("APS", PROJ, 3), ("APS", none_, 2), ("CHA", PROJ, 49),
("CHA", none_, 229), ("CHA", "pure.exp_abs.bracket.both", 3),
("CHA", "pure.exp_abs.bracket.bracket", 4),
("CHA", "pure.exp_abs.bracket.credit", 6),
("APS", CREDIT, 25), ("CHA", CREDIT, 9)):
check(f"{setname} {lbl} place", place(rows, setname, lbl), pl)
check("CHA exp_abs.bracket.both total", by["pure.exp_abs.bracket.both"]["CHA"]["total"],
2736)
1778)
check("CHA TOMS 748 over the 2x bound", by["toms748"]["CHA"]["over"], 15)
check("CHA Brent over the 2x bound", by["brent"]["CHA"]["over"], 18)
for lbl, setname, opp, want in (
(CREDIT, "APS", "itp_oracle", (86, 6, 61)),
(CREDIT, "CHA", "itp_oracle", (2, 2, 36)),
(CREDIT, "CHA", "bisection", (5, 0, 35)),
(PROJ, "APS", "chandrupatla", (87, 24, 42)),
(PROJ, "APS", "itp", (132, 3, 18)),
(PROJ, "APS", "itp_oracle", (121, 4, 28)),
(PROJ, "APS", "bisection", (152, 1, 0)),
(PROJ, "CHA", "toms748", (31, 1, 8)),
(PROJ, "CHA", "brent", (30, 4, 4)),
(PROJ, "CHA", "itp", (17, 0, 23)),
(PROJ, "CHA", "itp_oracle", (16, 0, 24))):
(CREDIT, "APS", "itp_oracle", (86, 6, 62)),
(CREDIT, "CHA", "itp_oracle", (2, 2, 41)),
(CREDIT, "CHA", "bisection", (5, 0, 40)),
(PROJ, "APS", "chandrupatla", (87, 24, 43)),
(PROJ, "APS", "itp", (132, 3, 19)),
(PROJ, "APS", "itp_oracle", (121, 4, 29)),
(PROJ, "APS", "bisection", (152, 1, 1)),
(PROJ, "CHA", "toms748", (33, 1, 11)),
(PROJ, "CHA", "brent", (34, 4, 7)),
(PROJ, "CHA", "itp", (18, 1, 26)),
(PROJ, "CHA", "itp_oracle", (16, 1, 28))):
w, t, l, _ = head_to_head(rows, lbl, opp, setname)
check(f"{lbl.split('.')[-1]} {setname} vs {opp}", (w, t, l), want)
for lbl, want in ((CREDIT, 6588), (PROJ, 6026), ("chandrupatla", 5228),
("itp_oracle", 5907), ("toms748", 6820), ("brent", 7004)):
for lbl, want in ((CREDIT, 5450), (PROJ, 4991), ("chandrupatla", 4100),
("itp_oracle", 4743), ("toms748", 5724), ("brent", 5945)):
check(f"both sets {lbl}", by[lbl]["APS"]["total"] + by[lbl]["CHA"]["total"], want)
print("\n§7 under the earned projection:")
@@ -501,36 +500,35 @@ def verify_safeguard_and_tables():
for p in glob.glob("sweep_*.json"):
if "smoke" in p:
continue
with open(p, encoding="utf8") as fh:
data = json.load(fh)
data = _load(p)
# only grid files (a list of labelled cells); the out-of-sample results
# (sweep_oos.json, sweep_fourth_*.json, ...) are dicts of a different shape
if not isinstance(data, list):
continue
for r in data:
cells.setdefault(r["label"], r)
for lbl, aps, cha in (("slope.ratio.derived3.credit", 3507, 2997),
("pure+shape+slope.ratio.derived3.credit", 3755, 3021),
("pure+dissim+slope.ratio.derived3.credit", 3857, 3002),
("pure+shape+dissim+slope.ratio.derived3.credit", 4354, 3037),
("defect.ratio.derived3.credit", 3545, 2949)):
for lbl, aps, cha in (("slope.ratio.derived3.credit", 3316, 2050),
("pure+shape+slope.ratio.derived3.credit", 3564, 2074),
("pure+dissim+slope.ratio.derived3.credit", 3666, 2052),
("pure+shape+dissim+slope.ratio.derived3.credit", 4163, 2087),
("defect.ratio.derived3.credit", 3354, 2002)):
check(f"{lbl} APS", cells[lbl]["APS"]["total"], aps)
check(f"{lbl} CHA", cells[lbl]["CHA"]["total"], cha)
free = [r for r in cells.values() if r.get("spec") and len(r["spec"]) > 3
and r["spec"][0] == "pure" and r["spec"][3] == "credit"
and r["APS"]["solved"] + r["CHA"]["solved"] == 193]
and r["APS"]["solved"] + r["CHA"]["solved"] == r["APS"]["n"] + r["CHA"]["n"]]
top = min(free, key=lambda r: r["APS"]["total"] + r["CHA"]["total"])
check("best constant-free configuration", top["label"],
"defect+flatanalogmempt+exp+half.ratio.derived3.credit")
check("its total over both sets", top["APS"]["total"] + top["CHA"]["total"], 6111)
check("its total over both sets", top["APS"]["total"] + top["CHA"]["total"], 4965)
d = cells["defect.ratio.derived3.credit"]
check("defect under the earned projection, both sets",
d["APS"]["total"] + d["CHA"]["total"], 6494)
d["APS"]["total"] + d["CHA"]["total"], 5356)
PLACEMENT_PER_SET = {
"sweep_direction.json": ("CHA", "pure+dissim+slope.ratio.derived3.safe", 2695),
"sweep_gate.json": ("CHA", "progress.ratio_soft.derived.safe", 2888),
"sweep_direction.json": ("CHA", "pure+dissim+slope.ratio.derived3.safe", 1752),
"sweep_gate.json": ("CHA", "progress.ratio_soft.derived.safe", 1929),
}
@@ -541,8 +539,7 @@ def verify_placement_modes():
print(f" !! {path} absent — the section quoting it cannot be verified")
FAILS.append(f"missing {path}")
continue
with open(path, encoding="utf8") as fh:
rows = json.load(fh)
rows = _load(path)
try:
d = next(r for r in rows if r["label"] == lbl)
except StopIteration:
@@ -551,13 +548,12 @@ def verify_placement_modes():
continue
check(f"{lbl} APS", d["APS"]["total"], aps)
check(f"{lbl} CHA", d["CHA"]["total"], cha)
check(f"{lbl} solved", d["APS"]["solved"] + d["CHA"]["solved"], 193)
check(f"{lbl} solved", d["APS"]["solved"] + d["CHA"]["solved"], 199)
check(f"{lbl} guarantee (over 2x)", d["APS"]["over"] + d["CHA"]["over"], 0)
for path, (setname, lbl, want) in PLACEMENT_PER_SET.items():
if not os.path.exists(path):
continue
with open(path, encoding="utf8") as fh:
rows = json.load(fh)
rows = _load(path)
d = next((r for r in rows if r["label"] == lbl), None)
if d is None:
FAILS.append(f"{lbl} missing from {path}")
@@ -566,8 +562,15 @@ def verify_placement_modes():
def _load(path):
"""A results file. A row-format protocol sweep (a list of labelled cells) is served from
sweep_uniform.json, its re-scoring under the paper's single criterion - a sign change
within 1e-6 or f(x) == 0 (revision point 3, 2026-09-22); every other file as stored."""
from rescore_uniform import load_uniform
with open(path, encoding="utf8") as fh:
return json.load(fh)
data = json.load(fh)
if isinstance(data, list) and data and "spec" in data[0]:
return load_uniform(path)
return data
def _cell(path, lbl):
@@ -591,21 +594,21 @@ def verify_flat_regime_and_oos():
print("\nSection 8 and negative items 12-16 (the development grids):")
P = "ratio.derived3.project"
for path, lbl, aps, cha in (
("sweep_flat.json", f"flat.{P}", 3029, 3198),
("sweep_flat2.json", f"flat2.{P}", 2803, 3341),
("sweep_flatcreep.json", f"flatcreep.{P}", 2878, 3354),
("sweep_flatstick.json", f"flatstickmid.{P}", 6897, 3019),
("sweep_flatexit.json", f"flatstickexit.{P}", 3125, 2503),
("sweep_flatjudge.json", f"flatjudge.{P}", 3009, 2486),
("sweep_orderpt.json", f"flatanalogmem.{P}", 2814, None),
("sweep_half.json", f"half.{P}", 2564, None)):
("sweep_flat.json", f"flat.{P}", 2839, 2243),
("sweep_flat2.json", f"flat2.{P}", 2613, 2388),
("sweep_flatcreep.json", f"flatcreep.{P}", 2688, 2399),
("sweep_flatstick.json", f"flatstickmid.{P}", 6707, 2063),
("sweep_flatexit.json", f"flatstickexit.{P}", 2935, 1546),
("sweep_flatjudge.json", f"flatjudge.{P}", 2820, 1532),
("sweep_orderpt.json", f"flatanalogmem.{P}", 2633, None),
("sweep_half.json", f"half.{P}", 2392, None)):
r = _cell(path, lbl)
check(f"{lbl} APS", r["APS"]["total"], aps)
if cha is not None:
check(f"{lbl} CHA", r["CHA"]["total"], cha)
check(f"{lbl} guarantee", r["APS"]["over"] + r["CHA"]["over"], 0)
check("sticky with order from the failed promise, APS",
_cell("sweep_flatstick.json", f"flatstick.{P}")["APS"]["total"], 6781)
_cell("sweep_flatstick.json", f"flatstick.{P}")["APS"]["total"], 6591)
check("sticky midpoint: flat families",
_flat_sum(_cell("sweep_flatstick.json", f"flatstickmid.{P}")), 1057)
full = {r["label"]: r for r in _load("sweep_full.json")}
@@ -617,9 +620,9 @@ def verify_flat_regime_and_oos():
check("split from the order: flat families, high end",
_flat_sum(op[f"pure+extP+flatanalogpt.{P}"]), 847)
check("item 16: remembered + split, with the step pattern, CHA",
op[f"pure+extP+flatanalogmempt.{P}"]["CHA"]["total"], 2117)
op[f"pure+extP+flatanalogmempt.{P}"]["CHA"]["total"], 1250)
check("item 16: one-state analogy + split, with the step pattern, CHA",
op[f"pure+extP+flatanalogpt.{P}"]["CHA"]["total"], 2221)
op[f"pure+extP+flatanalogpt.{P}"]["CHA"]["total"], 1388)
print("\nSection 8 diagnostics (fresh runs):")
import diag_analogy_decoupled
@@ -668,37 +671,37 @@ def verify_flat_regime_and_oos():
A = f"pure+extP+{ap}.ratio.derived3"
apr = {r["label"]: r for r in rows_ap}
a = apr[f"{A}.project"]
check("A' APS", a["APS"]["total"], 2438)
check("A' CHA", a["CHA"]["total"], 2005)
check("A' both", a["APS"]["total"] + a["CHA"]["total"], 4443)
check("A' solved", a["APS"]["solved"] + a["CHA"]["solved"], 193)
check("A' APS", a["APS"]["total"], 2259)
check("A' CHA", a["CHA"]["total"], 1074)
check("A' both", a["APS"]["total"] + a["CHA"]["total"], 3333)
check("A' solved", a["APS"]["solved"] + a["CHA"]["solved"], 199)
c = apr[f"{A}.credit"]
check("A' earned budget", (c["APS"]["total"], c["CHA"]["total"]), (3608, 2807))
check("A' earned budget", (c["APS"]["total"], c["CHA"]["total"]), (3417, 1852))
check("A' under every safeguard: violations",
sum(r["APS"]["over"] + r["CHA"]["over"] for lb, r in apr.items() if lb.startswith(A)), 0)
check("A' safeguards measured", sum(1 for lb in apr if lb.startswith(A + ".")), 7)
both = [r for r in _load("sweep_full.json")] + rows_ap
for setname, opp, want in (("APS", "chandrupatla", (103, 20, 30)),
("CHA", "chandrupatla", (29, 4, 7)),
("APS", "toms748", (127, 6, 20)), ("CHA", "toms748", (38, 2, 0)),
("APS", "brent", (118, 17, 18)), ("CHA", "brent", (33, 3, 2)),
("APS", "itp_oracle", (126, 7, 20)),
("CHA", "itp_oracle", (35, 1, 4))):
for setname, opp, want in (("APS", "chandrupatla", (103, 20, 31)),
("CHA", "chandrupatla", (31, 4, 10)),
("APS", "toms748", (127, 6, 21)), ("CHA", "toms748", (42, 2, 1)),
("APS", "brent", (118, 17, 19)), ("CHA", "brent", (39, 3, 3)),
("APS", "itp_oracle", (126, 7, 21)),
("CHA", "itp_oracle", (35, 1, 9))):
w, t, l, _ = head_to_head(both, f"{A}.project", opp, setname)
check(f"A' {setname} vs {opp}", (w, t, l), want)
def sign_p(w, l):
k, n = min(w, l), w + l
return min(1.0, 2.0 * sum(math.comb(n, i) for i in range(k + 1)) / 2.0 ** n)
check("A' APS vs Chandrupatla p (2 digits)", float(f"{sign_p(103, 30):.2g}"), 1.5e-10)
check("A' CHA vs Chandrupatla p (1 digit)", float(f"{sign_p(29, 7):.1g}"), 3e-4)
check("A' below Chandrupatla, APS %", round(100 * (1 - 2438 / 2787), 1), 12.5)
check("A' below Chandrupatla, CHA %", round(100 * (1 - 2005 / 2441), 1), 17.9)
check("A' APS vs Chandrupatla p (1 digit)", float(f"{sign_p(103, 31):.1g}"), 3e-10)
check("A' CHA vs Chandrupatla p (1 digit)", float(f"{sign_p(31, 10):.1g}"), 0.001)
check("A' below Chandrupatla, APS %", round(100 * (1 - 2259 / 2595), 1), 12.9)
check("A' below Chandrupatla, CHA %", round(100 * (1 - 1074 / 1505), 1), 28.6)
from where_we_lose import per_family
af, cf = per_family(a, "CHA"), per_family(full["chandrupatla"], "CHA")
check("A' fun9 vs Chandrupatla", (af["fun9"], cf["fun9"]), (65, 55))
check("A' ahead on wide-bracket families", sum(af[k] < cf[k] for k in cf), 7)
check("A' level on fun7", af["fun7"] == cf["fun7"], True)
check("A' fun7 vs Chandrupatla", (af["fun7"], cf["fun7"]), (69, 64))
print("\nTable 9 (ablation of A' on six corpora, sweep_ablation_all.json):")
ab = _load("sweep_ablation_all.json")
@@ -802,16 +805,16 @@ def verify_flat_regime_and_oos():
(1580, 1382))
print("\nModAB (sweep_modab.json, the authors' package):")
mb = _load("sweep_modab.json")
mp = mb["protocol"]["modab"]
check("ModAB protocol APS, CHA, both", (mp["APS"]["total"], mp["CHA"]["total"],
mp["APS"]["total"] + mp["CHA"]["total"]),
(2914, 2436, 5350))
for setname, want in (("APS", (118, 11, 25)), ("CHA", (32, 9, 4))):
ms = mb["seen"]["modab"]
check("Table 8 ModAB APS, CHA, both",
(sum(cost(v) for v in ms["APS"].values()), sum(cost(v) for v in ms["CHA"].values()),
sum(cost(v) for s_ in ("APS", "CHA") for v in ms[s_].values())), (2726, 1681, 4407))
for setname, want in (("APS", (118, 10, 26)), ("CHA", (34, 4, 7))):
am = {p: v for p, v in a[setname]["per"].items()}
mm = mp[setname]["per"]
mm = ms[setname]
w = sum(cost(am[p]) < cost(mm[p]) for p in am)
l = sum(cost(am[p]) > cost(mm[p]) for p in am)
check(f"A' vs ModAB protocol {setname}", (w, len(am) - w - l, l), want)
check(f"Table 8 A' vs ModAB {setname}", (w, len(am) - w - l, l), want)
md = dict(mb["seen"]["modab"])
md.update(mb["fourth"]["modab"])
worst_p = 0.0
@@ -843,20 +846,20 @@ def verify_flat_regime_and_oos():
print("\nTable 9a and section 7: the similarity in the step (fresh audit run):")
aps_ = {r["spec"][1]: r for r in _load("sweep_aprime_s.json")}
abs_ = _load("sweep_ablation_s.json")
t9a = {"ratio": (4443, 13032), "exp_abs": (4988, 14142), "rational": (5486, 14843),
"ratio_soft": (8360, 28500), "sqrt": (12718, 45552), "cbrt": (17207, 63031),
"exp_sq": (18410, 67701), "cosine": (18412, 67854), "rat_sq": (18422, 67394)}
t9a = {"ratio": (3333, 13032), "exp_abs": (3965, 14142), "rational": (4424, 14843),
"ratio_soft": (7249, 28500), "sqrt": (11613, 45552), "cbrt": (16094, 63031),
"exp_sq": (17429, 67701), "cosine": (17487, 67854), "rat_sq": (17452, 67394)}
for s_, (two, six) in t9a.items():
r = aps_[s_]
check(f"Table 9a {s_} APS+CHA", r["APS"]["total"] + r["CHA"]["total"], two)
check(f"Table 9a {s_} six corpora", sum(tot(abs_[f"S:{s_}"], s) for s in sets), six)
check(f"Table 9a {s_} guarantee", r["APS"]["over"] + r["CHA"]["over"]
+ sum(v[2] for s in sets for v in abs_[f"S:{s_}"][s].values()), 0)
check("nearest other S costs more, protocol %", round(100 * (4988 / 4443 - 1)), 12)
check("worst S costs more, protocol %", round(100 * (18422 / 4443 - 1)), 315)
check("nearest other S costs more, two sets %", round(100 * (3965 / 3333 - 1)), 19)
check("worst S costs more, two sets %", round(100 * (17487 / 3333 - 1)), 425)
check("nearest other S costs more, six corpora %", round(100 * (14142 / 13032 - 1)), 9)
check("quadratic S: four to five times",
(int(18410 / 4443), int(67854 / 13032)), (4, 5))
check("quadratic S: five times",
(int(17429 / 3333), int(67854 / 13032)), (5, 5))
import audit_similarity_places
buf = io.StringIO()
with contextlib.redirect_stdout(buf):
@@ -876,5 +879,129 @@ def verify_flat_regime_and_oos():
check(f"sign test p for {w}:{l}", f"{sign_p(w, l):.1g}", printed)
def verify_revision1():
"""Revision 1 (2026-09-22): the claims added or restated by the revision - the order
of the base step, section 7.3's solved counts, the families as the unit, ITP in Table 10,
negative items 5 and 9, and the one canonical problem the multiplier moves. Recomputed."""
import contextlib
import io
import math
import re
from collections import defaultdict
print("\nRevision 1:")
import order_high_precision
buf = io.StringIO()
with contextlib.redirect_stdout(buf):
order_high_precision.main()
out = buf.getvalue()
check("Muller order", re.search(r"Muller order .*: ([\d.]+)", out).group(1), "1.8393")
for name in ("cos x - x", "exp(x) - 2", "x^3 - x - 2", "sin x - 1/2"):
line = next(ln for ln in out.splitlines() if ln.startswith(name))
three = line[14:54].split()
check(f"three-point order settles, {name}", three[-2], "1.840")
line = next(ln for ln in out.splitlines() if ln.startswith("x^2 - 2"))
check("three-point order on x^2 - 2", line[14:54].split()[-1], "2.000")
check("two-point order, cos x - x", next(ln for ln in out.splitlines()
if ln.startswith("cos x - x"))[54:].split()[-1], "1.618")
rows = _load("sweep_full.json")
by = {r["label"]: r for r in rows}
solved = {k: by[k]["APS"]["solved"] + by[k]["CHA"]["solved"]
for k in ("chandrupatla", "toms748", "itp", "bisection", "brent", "dekker",
"illinois")}
check("section 7.3 solved counts", solved, {"chandrupatla": 199, "toms748": 199,
"itp": 199, "bisection": 199, "brent": 197,
"dekker": 178, "illinois": 177})
check("Chandrupatla fewer than the base rule, APS %",
round(100 * (by[PROJ]["APS"]["total"] / by["chandrupatla"]["APS"]["total"] - 1), 1), 1.5)
none_ = by["pure.ratio.derived3.none"]["CHA"]
check("guarantee on CHA: more solved, fewer evaluations",
(45 - none_["solved"], round(100 * (1 - by[PROJ]["CHA"]["total"] / none_["total"]))),
(6, 30))
check("earned costs 28% more on APS",
round(100 * (by[CREDIT]["APS"]["total"] / by[PROJ]["APS"]["total"] - 1)), 28)
def cost(v):
return v[0] if v[1] else 200
def fam_of(pid):
return ".".join(pid.split(".")[:2]) if pid.startswith("aps") else pid.split(".")[0]
def families(a, b, s):
ta, tb = defaultdict(int), defaultdict(int)
for p in a[s]["per"]:
ta[fam_of(p)] += cost(a[s]["per"][p])
tb[fam_of(p)] += cost(b[s]["per"][p])
w = sum(ta[k] < tb[k] for k in ta)
l = sum(ta[k] > tb[k] for k in ta)
return w, len(ta) - w - l, l
def sign_p(w, l):
k, n = min(w, l), w + l
return min(1.0, 2.0 * sum(math.comb(n, i) for i in range(k + 1)) / 2.0 ** n)
ap = {r["label"]: r for r in _load("sweep_aprime.json")}[
"pure+extP+contra+flatanalogmempt+exp+half+close+sub.ratio.derived3.project"]
ch = by["chandrupatla"]
check("base rule vs Chandrupatla by family, APS", families(by[PROJ], ch, "APS"), (6, 1, 8))
check("A' vs Chandrupatla by family, APS", families(ap, ch, "APS"), (8, 1, 6))
check("A' vs Chandrupatla by family, CHA", families(ap, ch, "CHA"), (7, 0, 2))
check("family-level p, APS and CHA (2 digits)",
(float(f"{sign_p(8, 6):.2g}"), float(f"{sign_p(7, 2):.2g}")), (0.79, 0.18))
order = _load("sweep_order.json")
cells = {}
for r in order:
s = r["spec"]
if s[2] == "far":
cells[(s[1], s[3], s[8] if len(s) > 8 else "none")] = r["APS"]["total"] + r["CHA"]["total"]
pairs = [(k[0], k[1]) for k in cells if k[2] == "none" and (k[0], k[1], "order") in cells]
check("far-end reading: measured order cheaper in cells",
(sum(cells[(s_, g, "order")] < cells[(s_, g, "none")] for s_, g in pairs), len(pairs)),
(33, 63))
cap, dire = _load("sweep_cap.json"), _load("sweep_direction.json")
for s, want in (("APS", (3101, 2539)), ("CHA", (1766, 1752))):
bc = min(r[s]["total"] for r in cap if r[s]["solved"] == r[s]["n"])
bd = min(r[s]["total"] for r in dire if r[s]["solved"] == r[s]["n"] and r["spec"][0] == "pure")
check(f"item 9: best capped vs best uncapped, {s}", (bc, bd), want)
itp = _load("sweep_itp.json")
t10 = {"APS": (3767, 3068), "CHA": (1946, 1675), "NEW": (5674, 4707), "RJL": (2161, 1629),
"APP": (3245, 2386), "D": (5721, 4832)}
for s, want in t10.items():
check(f"Table 10 ITP / oracle {s}", (sum(cost(v) for v in itp["itp"][s].values()),
sum(cost(v) for v in itp["itp_oracle"][s].values())), want)
check("Table 10 ITP / oracle over 855", tuple(sum(cost(v) for s in t10 for v in itp[k][s].values())
for k in ("itp", "itp_oracle")), (22514, 18297))
check("ITP bound violations over 855", sum(v[2] for k in ("itp", "itp_oracle")
for s in t10 for v in itp[k][s].values()), 0)
seen, four = _load("sweep_fourth_seen.json"), _load("sweep_fourth_fourth.json")
apx = dict(seen["Aprime"])
apx.update(four["Aprime"])
check("A' ahead of ITP and the oracle on every corpus",
all(sum(cost(v) for v in apx[s].values()) < sum(cost(v) for v in itp[k][s].values())
for s in t10 for k in ("itp", "itp_oracle")), True)
from simroot.pure import pure_root
from stage12_validation import load, per_problem
probs = load()
aps = [p for p in probs if p[0] == "APS"]
runs = {fac: per_problem(lambda f, a, b, xt, mx, _f=fac: pure_root(
f, a, b, xt, mx, similarity="ratio", scale="derived3", guard="project",
budget_factor=_f), aps, 200, criterion="sign") for fac in (1.25, 1.5, 2.0)}
moved = {p: tuple(runs[f][p][0] for f in (1.25, 1.5, 2.0)) for p in runs[2.0]
if len({runs[f][p][0] for f in (1.25, 1.5, 2.0)}) > 1}
check("Table 1a: the one canonical problem the multiplier moves", moved,
{"aps.13.00": (20, 30, 28)})
tot = {}
for sc in ("derived3", "derived3_m"):
pp = per_problem(lambda f, a, b, xt, mx, _s=sc: pure_root(
f, a, b, xt, mx, similarity="ratio", scale=_s, guard="project"), probs, 200,
criterion="sign")
tot[sc] = sum(v[0] for v in pp.values())
check("item 5: phase detector vs base rule, both sets", (tot["derived3_m"], tot["derived3"]),
(5336, 4991))
if __name__ == "__main__":
main()