vvs/sim-numerics
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
b7271a1ce4e4efeb1f7464bcd59a8f3021ea4933
sim-numerics/results_audit_head_to_head.txt
T
vvsandClaude Opus 5 de90c7a089 Audit on Valeri's challenge: our Chandrupatla implementation stops below 2*xtol (tl = tol/span > 1/2) while every other method, SciPy's included, stops at xtol; the protocol corrected that factor for ITP and not here, and best_of let the looser run win. Corrected it costs Chandrupatla 2 evaluations on APS (2785 -> 2787) and 18 on CHA (2423 -> 2441): real, in the competitor's favour, and not what changes the picture. The public scipy.optimize.elementwise.find_root, the private _chandrupatla and ours corrected agree on all 199 problems exactly. 
What the tables' totals hid: problem by problem the scheduled projection beats Chandrupatla on APS 87/24/42, exact sign test p = 9e-5, while trailing by 0.7% in total; on CHA Chandrupatla is genuinely better (26 to 9, p = 0.006). The earned projection ties him on APS problems (79/3/71, p = 0.57). Benchmark not yet switched to the library baseline - awaiting Valeri's decision together with the framing of the proposed method.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 10:14:42 +01:00

5 lines
466 B
Plaintext
Raw Blame History

scheduled projection (c=2) APS: wins/ties/losses 87/24/42 sign test p = 9.2e-05 totals 2806 vs Chandrupatla 2787
scheduled projection (c=2) CHA: wins/ties/losses 9/5/26 sign test p = 0.006 totals 3220 vs Chandrupatla 2441
earned projection (credit) APS: wins/ties/losses 79/3/71 sign test p = 0.57 totals 3568 vs Chandrupatla 2787
earned projection (credit) CHA: wins/ties/losses 5/0/35 sign test p = 1.4e-06 totals 3020 vs Chandrupatla 2441
Reference in New Issue View Git Blame Copy Permalink