vvs/sim-numerics
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

All-corpora wins/losses table: ModAB added (totals, violations, A' per problem and total difference against it on every corpus).

Browse Source 
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
vvs
2026-09-21 21:43:43 +01:00
co-authored by Claude Opus 5
parent f80aeb5d13
commit d2a142c39e
2 changed files with 21 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
results_table_all_corpora.txt
+14 -14
View File
@@ -1,18 +1,18 @@
TOTALS (violations of the 2x-bisection bound in brackets) TOTALS (violations of the 2x-bisection bound in brackets)
corpus A' A plain Chandrupatla Brent TOMS748 corpus A' A plain Chandrupatla ModAB Brent TOMS748
APS (154) 2259 (0) 2380 (0) 2634 (0) 2595 (0) 2707 (0) 2636 (0) APS (154) 2259 (0) 2380 (0) 2634 (0) 2595 (0) 2726 (0) 2707 (0) 2636 (0)
Chandrupatla (45) 1074 (0) 1109 (0) 2357 (0) 1505 (0) 3238 (18) 3088 (15) Chandrupatla (45) 1074 (0) 1109 (0) 2357 (0) 1505 (0) 1681 (0) 3238 (18) 3088 (15)
new corpus 1 (206) 3290 (0) 3380 (0) 5389 (0) 3716 (0) 6395 (38) 5746 (22) new corpus 1 (206) 3290 (0) 3380 (0) 5389 (0) 3716 (0) 3809 (1) 6395 (38) 5746 (22)
Roots.jl (78) 1371 (0) 1464 (0) 1884 (0) 1566 (0) 2455 (11) 2339 (9) Roots.jl (78) 1371 (0) 1464 (0) 1884 (0) 1566 (0) 1446 (0) 2455 (11) 2339 (9)
applied+GSL (132) 1382 (0) 1580 (0) 1590 (0) 1382 (0) 1503 (1) 1756 (1) applied+GSL (132) 1382 (0) 1580 (0) 1590 (0) 1382 (0) 1460 (0) 1503 (1) 1756 (1)
FOURTH (240) 3656 (0) 4547 (0) 7711 (0) 5717 (0) 8267 (50) 8084 (37) FOURTH (240) 3656 (0) 4547 (0) 7711 (0) 5717 (0) 5535 (8) 8267 (50) 8084 (37)
A' PER PROBLEM against each rival: wins/ties/losses (sign-test p); total difference A' PER PROBLEM against each rival: wins/ties/losses (sign-test p); total difference
APS (154) chand 103/20/31 p=3e-10 -13% brent 118/17/19 p=1e-18 -17% toms7 127/6/21 p=1e-19 -14% APS (154) chand 103/20/31 p=3e-10 -13% modab 118/10/26 p=3e-15 -17% brent 118/17/19 p=1e-18 -17% toms7 127/6/21 p=1e-19 -14%
Chandrupatla (45) chand 31/4/10 p=0.001 -29% brent 39/3/3 p=6e-09 -67% toms7 42/2/1 p=1e-11 -65% Chandrupatla (45) chand 31/4/10 p=0.001 -29% modab 34/4/7 p=3e-05 -36% brent 39/3/3 p=6e-09 -67% toms7 42/2/1 p=1e-11 -65%
new corpus 1 (206) chand 108/33/65 p=0.001 -11% brent 141/33/32 p=2e-17 -49% toms7 164/25/17 p=2e-31 -43% new corpus 1 (206) chand 108/33/65 p=0.001 -11% modab 125/26/55 p=2e-07 -14% brent 141/33/32 p=2e-17 -49% toms7 164/25/17 p=2e-31 -43%
Roots.jl (78) chand 44/14/20 p=0.004 -12% brent 58/7/13 p=6e-08 -44% toms7 65/3/10 p=5e-11 -41% Roots.jl (78) chand 44/14/20 p=0.004 -12% modab 47/11/20 p=0.001 -5% brent 58/7/13 p=6e-08 -44% toms7 65/3/10 p=5e-11 -41%
applied+GSL (132) chand 48/52/32 p=0.09 +0% brent 59/52/21 p=3e-05 -8% toms7 111/16/5 p=4e-27 -21% applied+GSL (132) chand 48/52/32 p=0.09 +0% modab 76/35/21 p=2e-08 -5% brent 59/52/21 p=3e-05 -8% toms7 111/16/5 p=4e-27 -21%
FOURTH (240) chand 177/35/28 p=1e-27 -36% brent 143/60/37 p=7e-16 -56% toms7 178/44/18 p=3e-34 -55% FOURTH (240) chand 177/35/28 p=1e-27 -36% modab 183/23/34 p=7e-26 -34% brent 143/60/37 p=7e-16 -56% toms7 178/44/18 p=3e-34 -55%
ALL SIX CORPORA (855 problems): Aprime 13032 A 14460 base 21565 chandrupatla 16481 brent 24565 toms748 23649 ALL SIX CORPORA (855 problems): Aprime 13032 A 14460 base 21565 chandrupatla 16481 modab 16657 brent 24565 toms748 23649
table_all_corpora.py
+7 -4
View File
@@ -25,19 +25,22 @@ def main():
F = json.load(open("sweep_fourth_fourth.json", encoding="utf8")) F = json.load(open("sweep_fourth_fourth.json", encoding="utf8"))
for k in R: for k in R:
R[k].update(F[k]) R[k].update(F[k])
M = json.load(open("sweep_modab.json", encoding="utf8"))
R["modab"] = dict(M["seen"]["modab"])
R["modab"].update(M["fourth"]["modab"])
print("TOTALS (violations of the 2x-bisection bound in brackets)") print("TOTALS (violations of the 2x-bisection bound in brackets)")
print(f" {'corpus':<20}" + "".join(f"{n:>16}" for n in print(f" {'corpus':<20}" + "".join(f"{n:>16}" for n in
("A'", "A", "plain", "Chandrupatla", "Brent", "TOMS748"))) ("A'", "A", "plain", "Chandrupatla", "ModAB", "Brent", "TOMS748")))
for s in NAMES: for s in NAMES:
cells = [] cells = []
for k in ("Aprime", "A", "base", "chandrupatla", "brent", "toms748"): for k in ("Aprime", "A", "base", "chandrupatla", "modab", "brent", "toms748"):
r = R[k][s] r = R[k][s]
cells.append(f"{sum(cost(v) for v in r.values())} ({sum(v[2] for v in r.values())})") cells.append(f"{sum(cost(v) for v in r.values())} ({sum(v[2] for v in r.values())})")
print(f" {NAMES[s]:<20}" + "".join(f"{c:>16}" for c in cells)) print(f" {NAMES[s]:<20}" + "".join(f"{c:>16}" for c in cells))
print("\nA' PER PROBLEM against each rival: wins/ties/losses (sign-test p); total difference") print("\nA' PER PROBLEM against each rival: wins/ties/losses (sign-test p); total difference")
for s in NAMES: for s in NAMES:
line = f" {NAMES[s]:<20}" line = f" {NAMES[s]:<20}"
for rv in ("chandrupatla", "brent", "toms748"): for rv in ("chandrupatla", "modab", "brent", "toms748"):
a, o = R["Aprime"][s], R[rv][s] a, o = R["Aprime"][s], R[rv][s]
w = sum(cost(a[p]) < cost(o[p]) for p in a) w = sum(cost(a[p]) < cost(o[p]) for p in a)
l = sum(cost(a[p]) > cost(o[p]) for p in a) l = sum(cost(a[p]) > cost(o[p]) for p in a)
@@ -47,7 +50,7 @@ def main():
tot = {k: sum(cost(v) for s in NAMES for v in R[k][s].values()) for k in R} tot = {k: sum(cost(v) for s in NAMES for v in R[k][s].values()) for k in R}
n_all = sum(len(R["Aprime"][s]) for s in NAMES) n_all = sum(len(R["Aprime"][s]) for s in NAMES)
print(f"\nALL SIX CORPORA ({n_all} problems):", print(f"\nALL SIX CORPORA ({n_all} problems):",
" ".join(f"{k} {tot[k]}" for k in ("Aprime", "A", "base", "chandrupatla", "brent", "toms748"))) " ".join(f"{k} {tot[k]}" for k in ("Aprime", "A", "base", "chandrupatla", "modab", "brent", "toms748")))
if __name__ == "__main__": if __name__ == "__main__":