vvs/sim-numerics
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Wins and losses of A' on all six corpora (1055 problems), server numbers only: the seen-corpora run is replaced by its Linux rerun (differences of 1-3 from Windows). Totals: A' 13032, Chandrupatla 16481, Brent 24565, TOMS748 23649. A' vs Chandrupatla per problem significant on 5 of 6 corpora; the applied+GSL corpus is parity (1382 = 1382, 48/52/32 p=0.09).

Browse Source 
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
vvs
2026-09-21 20:18:54 +01:00
co-authored by Claude Opus 5
parent 6d2531e6ea
commit 94d81f24d8
4 changed files with 77 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
results_fourth_seen.txt
+5 -5
View File
@@ -1,7 +1,7 @@
SEEN corpora (development, descriptive): totals; A' vs Chandrupatla w/t/l
APS base 2634 A 2380 Aprime 2259 Aprime-close 2389 Aprime-sub 2253 chandrupatla 2595 brent 2707 toms748 2635 A' vs Ch 103/20/31 p=3.2e-10
CHA base 2357 A 1109 Aprime 1074 Aprime-close 1109 Aprime-sub 1074 chandrupatla 1505 brent 3238 toms748 3111 A' vs Ch 31/4/10 p=0.0015
NEW base 5388 A 3380 Aprime 3289 Aprime-close 3387 Aprime-sub 3282 chandrupatla 3717 brent 6391 toms748 5727 A' vs Ch 109/32/65 p=0.0011
RJL base 1888 A 1464 Aprime 1371 Aprime-close 1449 Aprime-sub 1387 chandrupatla 1566 brent 2455 toms748 2338 A' vs Ch 44/14/20 p=0.0037
APP base 1588 A 1577 Aprime 1382 Aprime-close 1512 Aprime-sub 1447 chandrupatla 1380 brent 1502 toms748 1757 A' vs Ch 48/52/32 p=0.093
APS base 2634 A 2380 Aprime 2259 Aprime-close 2389 Aprime-sub 2253 chandrupatla 2595 brent 2707 toms748 2636 A' vs Ch 103/20/31 p=3.2e-10
CHA base 2357 A 1109 Aprime 1074 Aprime-close 1109 Aprime-sub 1074 chandrupatla 1505 brent 3238 toms748 3088 A' vs Ch 31/4/10 p=0.0015
NEW base 5389 A 3380 Aprime 3290 Aprime-close 3387 Aprime-sub 3283 chandrupatla 3716 brent 6395 toms748 5746 A' vs Ch 108/33/65 p=0.0013
RJL base 1884 A 1464 Aprime 1371 Aprime-close 1449 Aprime-sub 1387 chandrupatla 1566 brent 2455 toms748 2339 A' vs Ch 44/14/20 p=0.0037
APP base 1590 A 1580 Aprime 1382 Aprime-close 1517 Aprime-sub 1445 chandrupatla 1382 brent 1503 toms748 1756 A' vs Ch 48/52/32 p=0.093
fourth done
results_table_all_corpora.txt
+18
View File
@@ -0,0 +1,18 @@
TOTALS (violations of the 2x-bisection bound in brackets)
corpus A' A plain Chandrupatla Brent TOMS748
APS (154) 2259 (0) 2380 (0) 2634 (0) 2595 (0) 2707 (0) 2636 (0)
Chandrupatla (45) 1074 (0) 1109 (0) 2357 (0) 1505 (0) 3238 (18) 3088 (15)
new corpus 1 (206) 3290 (0) 3380 (0) 5389 (0) 3716 (0) 6395 (38) 5746 (22)
Roots.jl (78) 1371 (0) 1464 (0) 1884 (0) 1566 (0) 2455 (11) 2339 (9)
applied+GSL (132) 1382 (0) 1580 (0) 1590 (0) 1382 (0) 1503 (1) 1756 (1)
FOURTH (240) 3656 (0) 4547 (0) 7711 (0) 5717 (0) 8267 (50) 8084 (37)
A' PER PROBLEM against each rival: wins/ties/losses (sign-test p); total difference
APS (154) chand 103/20/31 p=3e-10 -13% brent 118/17/19 p=1e-18 -17% toms7 127/6/21 p=1e-19 -14%
Chandrupatla (45) chand 31/4/10 p=0.001 -29% brent 39/3/3 p=6e-09 -67% toms7 42/2/1 p=1e-11 -65%
new corpus 1 (206) chand 108/33/65 p=0.001 -11% brent 141/33/32 p=2e-17 -49% toms7 164/25/17 p=2e-31 -43%
Roots.jl (78) chand 44/14/20 p=0.004 -12% brent 58/7/13 p=6e-08 -44% toms7 65/3/10 p=5e-11 -41%
applied+GSL (132) chand 48/52/32 p=0.09 +0% brent 59/52/21 p=3e-05 -8% toms7 111/16/5 p=4e-27 -21%
FOURTH (240) chand 177/35/28 p=1e-27 -36% brent 143/60/37 p=7e-16 -56% toms7 178/44/18 p=3e-34 -55%
ALL SIX CORPORA (1055 problems): Aprime 13032 A 14460 base 21565 chandrupatla 16481 brent 24565 toms748 23649
sweep_fourth_seen.json
+1 -1
View File
File diff suppressed because one or more lines are too long
table_all_corpora.py
+53
View File
@@ -0,0 +1,53 @@
"""Wins and losses of A' (and A, the plain rule) against the library Chandrupatla, Brent and
TOMS748 on all six corpora, from the saved server results only (no new run):
sweep_fourth_seen.json (APS, CHA, new corpus 1, Roots.jl, applied - uniform sign-change
criterion) and sweep_fourth_fourth.json (the fourth corpus).
python -u table_all_corpora.py
"""
from __future__ import annotations
import json
from sweep_oos import sign_p
MAXFEV = 200
NAMES = {"APS": "APS (154)", "CHA": "Chandrupatla (45)", "NEW": "new corpus 1 (206)",
"RJL": "Roots.jl (78)", "APP": "applied+GSL (132)", "D": "FOURTH (240)"}
def cost(v):
return v[0] if v[1] else MAXFEV
def main():
R = json.load(open("sweep_fourth_seen.json", encoding="utf8"))
F = json.load(open("sweep_fourth_fourth.json", encoding="utf8"))
for k in R:
R[k].update(F[k])
print("TOTALS (violations of the 2x-bisection bound in brackets)")
print(f" {'corpus':<20}" + "".join(f"{n:>16}" for n in
("A'", "A", "plain", "Chandrupatla", "Brent", "TOMS748")))
for s in NAMES:
cells = []
for k in ("Aprime", "A", "base", "chandrupatla", "brent", "toms748"):
r = R[k][s]
cells.append(f"{sum(cost(v) for v in r.values())} ({sum(v[2] for v in r.values())})")
print(f" {NAMES[s]:<20}" + "".join(f"{c:>16}" for c in cells))
print("\nA' PER PROBLEM against each rival: wins/ties/losses (sign-test p); total difference")
for s in NAMES:
line = f" {NAMES[s]:<20}"
for rv in ("chandrupatla", "brent", "toms748"):
a, o = R["Aprime"][s], R[rv][s]
w = sum(cost(a[p]) < cost(o[p]) for p in a)
l = sum(cost(a[p]) > cost(o[p]) for p in a)
ta, to = sum(cost(v) for v in a.values()), sum(cost(v) for v in o.values())
line += f" {rv[:5]} {w}/{len(a) - w - l}/{l} p={sign_p(w, l):.1g} {100 * (ta - to) / to:+.0f}%"
print(line)
tot = {k: sum(cost(v) for s in NAMES for v in R[k][s].values()) for k in R}
print("\nALL SIX CORPORA (1055 problems):",
" ".join(f"{k} {tot[k]}" for k in ("Aprime", "A", "base", "chandrupatla", "brent", "toms748")))
if __name__ == "__main__":
main()