vvs/sim-numerics
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Valeri's decisions (1) competitors from the library, (2) variant a. (1) Chandrupatla is scipy.optimize.elementwise.find_root, Brent is brentq, no more per-problem best_of. Our Chandrupatla stopped on a bracket twice as wide as every other method (the factor the protocol corrects for ITP); fixed (+0.5*xtol) in baselines.py and in the two copies found by grep (stage16_blend, stage19_modern) and the traced mirror, and pinned: ours == library on every problem. Regenerating sweep_full.json changed exactly three rows - chandrupatla APS 2785->2787, CHA 2423->2441, brent CHA 4099->4115 - and nothing else (diff_sweeps.py: 0 unexpected).

Browse Source 
(2) The proposed method is again the scheduled projection (2806/3220/6026), its budget multiplier DECLARED as the one free constant: the canonical total is the same for every multiplier 1.25-4, the wide-bracket one is not. The earned projection is the constant-free variant (3568/3020/6588). What the old tables hid: problem by problem the proposed rule beats Chandrupatla on the canonical set 87/24/42, exact sign test p = 9e-5, while trailing him by 0.7% in total; on the wide-bracket set he is genuinely stronger (26 to 9, p = 0.006). Gap decomposition recomputed with the library: 244 (was 262), 160 of it on the flat families (two thirds), fun1 58 (a quarter, not a third as the abstract said).

Also corrected on the way: the 'subsumed by Chandrupatla' pin claimed he is better on the worst single problem - true only with the looser tolerance (84 against the credit rule's 83 once fixed); §8.3 still called zero slack degenerate, which Table 1a refutes; §5.2's Brent figure is our pure-Python implementation and now says so. verify_article.py 286/286, 802 tests pass twice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
vvs
2026-09-21 10:53:00 +01:00
co-authored by Claude Opus 5
parent de90c7a089
commit 1b6c626f67
32 changed files with 950 additions and 186 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_article_numbers.py
+41 -26
View File
@@ -16,8 +16,8 @@ import pytest
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# the other end of the trade the paper states: better average, one constant
PROJ_LABEL = "pure.ratio.derived3.project"
# the constant-free variant (the proposed rule is the projection, variant a)
CREDIT_LABEL = "pure.ratio.derived3.credit"
@pytest.mark.skipif(not os.path.exists(os.path.join(ROOT, "sweep_full.json")),
@@ -40,16 +40,21 @@ def test_the_headline_claim_is_what_the_paper_says_it_is():
It caught exactly that. Until Chandrupatla's own method was added - a
reader asked why the owner of half the benchmark was missing - this test
asserted that a configuration of ours came first, and it passed only
because the method that beats us was absent from the field. The claim it
now guards is the corrected one: we are not first, Chandrupatla is, and
our margin behind it on the canonical set is under one percent."""
because the method that beats us was absent from the field. The claims it
now guards (variant a, Valeri 2026-09-21): by TOTAL Chandrupatla is first
on the canonical set and we are within one percent of it; problem by
problem we beat it with significance; on the wide-bracket set and over
both sets it beats us. And the constant-free variant may not borrow the
projection's canonical numbers."""
import json
import math
from report_sweep import PROPOSED, rank
from report_sweep import PROPOSED, head_to_head, rank
with open(os.path.join(ROOT, "sweep_full.json"), encoding="utf8") as fh:
rows = json.load(fh)
by = {r["label"]: r for r in rows}
assert "chandrupatla" in by, "the sweep predates the missing baselines"
assert PROPOSED == "pure.ratio.derived3.project"
aps = rank(rows, "APS")
assert aps[0]["label"] == "chandrupatla", aps[0]["label"]
@@ -57,33 +62,43 @@ def test_the_headline_claim_is_what_the_paper_says_it_is():
def both(lbl):
return by[lbl]["APS"]["total"] + by[lbl]["CHA"]["total"]
# What the paper may claim for the credit method, and nothing more.
# It is NOT within a percent of the best on Alefeld-Potra-Shi - that was
# true of the projection variant, which carries a constant. Buying
# constant-freedom and a near-bisection worst case costs average
# performance, and the test pins the direction of that trade so the
# abstract cannot quietly reclaim the projection variant's numbers.
def tot(lbl, s):
return by[lbl][s]["total"]
# canonical set: parity by total with the best, a tie with TOMS 748
assert tot(PROPOSED, "APS") > tot("chandrupatla", "APS")
assert tot(PROPOSED, "APS") < 1.01 * tot("chandrupatla", "APS")
assert tot(PROPOSED, "APS") < tot("toms748", "APS") < 1.01 * tot(PROPOSED, "APS")
# ... and an edge problem by problem, with significance
w, _t, l, _ = head_to_head(rows, PROPOSED, "chandrupatla", "APS")
p = min(1.0, 2.0 * sum(math.comb(w + l, i) for i in range(min(w, l) + 1))
/ 2.0 ** (w + l))
assert w > 2 * l - 1 and p < 1e-3, (w, l, p)
# wide brackets and both sets: Chandrupatla is stronger, and the paper says so
assert tot(PROPOSED, "CHA") > tot("chandrupatla", "CHA")
assert both(PROPOSED) > both("chandrupatla"), "behind Chandrupatla"
for weaker in ("toms748", "brent", "itp", "ridders", "bisection"):
assert both(PROPOSED) < both(weaker), weaker
# On the canonical set alone it is behind the classical methods; saying
# otherwise would be quoting the projection variant.
assert by[PROPOSED]["APS"]["total"] > by["toms748"]["APS"]["total"]
assert by[PROPOSED]["APS"]["total"] > by["brent"]["APS"]["total"]
# the constant-free variant may not borrow the projection's canonical numbers
assert tot(CREDIT_LABEL, "APS") > tot("toms748", "APS")
assert tot(CREDIT_LABEL, "APS") > tot("brent", "APS")
assert tot(CREDIT_LABEL, "CHA") < tot(PROPOSED, "CHA")
# The claim it does own: the worst single problem, over the problems it
# solves, is close to bisection's and far below the classical methods'.
# the claim the constant-free variant does own: the worst single problem,
# over the problems it solves, is close to bisection's and far below the
# classical methods'
def worst(lbl):
w = 0
wst = 0
for setname in ("APS", "CHA"):
for n, ok in by[lbl][setname]["per"].values():
if ok:
w = max(w, n)
return w
wst = max(wst, n)
return wst
assert worst(PROPOSED) < 100, worst(PROPOSED)
assert worst(PROPOSED) < worst("brent")
assert worst(PROPOSED) < worst("toms748")
assert worst(PROPOSED) < worst(PROJ_LABEL), (worst(PROPOSED),
worst(PROJ_LABEL))
assert worst(CREDIT_LABEL) < 100, worst(CREDIT_LABEL)
assert worst(CREDIT_LABEL) < worst("brent")
assert worst(CREDIT_LABEL) < worst("toms748")
assert worst(CREDIT_LABEL) < worst(PROPOSED), (worst(CREDIT_LABEL), worst(PROPOSED))
tests/test_baselines_library.py
+58
View File
@@ -0,0 +1,58 @@
"""Pins for the 2026-09-21 switch of the competitors to the library.
REGRESSION: our Chandrupatla stopped on a bracket twice as wide as every other method
(tl = tol/span > 1/2 with tol = xtol), and best_of let that looser run into the
reported numbers. The benchmark now uses the library alone, and our implementation -
kept for the timing section and the traced mirror - must agree with it exactly.
"""
from __future__ import annotations
import math
import pytest
from scipy.optimize._tstutils import _APS_TESTS_DICTS, _CHANDRUPATLA_TESTS_DICTS
from simroot.baselines import (BRENT_BASELINE, CHANDRUPATLA_BASELINE, brent_scipy,
chandrupatla, chandrupatla_lib)
XTOL, MAXFEV = 1e-12, 200
def _problems():
out = []
for d in _APS_TESTS_DICTS[::7]:
f = (lambda ff, aa: lambda t: ff(t, *aa))(d["f"], d.get("args", ()))
out.append((d["ID"], f, float(d["bracket"][0]), float(d["bracket"][1])))
for d in _CHANDRUPATLA_TESTS_DICTS[::3]:
out.append((d["ID"], d["f"], float(d["bracket"][0]), float(d["bracket"][1])))
return out
def test_the_benchmark_competitors_are_the_library():
assert CHANDRUPATLA_BASELINE is chandrupatla_lib
assert BRENT_BASELINE is brent_scipy
def test_the_sweep_uses_them():
import sweep_full
assert sweep_full.BASELINES["chandrupatla"] is chandrupatla_lib
assert sweep_full.BASELINES["brent"] is brent_scipy
@pytest.mark.parametrize("pid,f,a,b", _problems())
def test_our_chandrupatla_spends_exactly_what_the_library_spends(pid, f, a, b):
"""The tolerance fix, pinned: same evaluation count and same verdict."""
x1, n1, c1 = chandrupatla(f, a, b, XTOL, MAXFEV)
x2, n2, c2 = chandrupatla_lib(f, a, b, XTOL, MAXFEV)
assert (n1, bool(c1)) == (n2, bool(c2)), (pid, n1, n2)
if c1 and c2 and math.isfinite(x1) and math.isfinite(x2):
assert abs(x1 - x2) <= 2 * XTOL + 1e-15 * max(1.0, abs(x1))
def test_our_chandrupatla_no_longer_stops_on_a_double_width_bracket():
"""On a smooth problem the looser rule saved an evaluation; with the fix our count
must equal the library's rather than undercut it."""
f = lambda t: math.exp(t) - 2.0 # noqa: E731
_x, n_ours, _c = chandrupatla(f, 0.0, 3.0, XTOL, MAXFEV)
_x, n_lib, _c = chandrupatla_lib(f, 0.0, 3.0, XTOL, MAXFEV)
assert n_ours == n_lib
tests/test_credit_guard.py
+2 -1
View File
@@ -147,7 +147,8 @@ def test_one_bit_is_the_cheapest_endowment_that_promises_one_step():
def test_the_scheduled_multiplier_has_no_such_determination():
"""The contrast that decides which safeguard the paper should propose.
"""The contrast behind the paper's declaring the multiplier a free constant
(and offering the earned projection as the constant-free variant).
The multiplicative family has no smallest non-trivial member: 1 is
degenerate and everything above it is admissible with a different cost, so
the multiplier is genuinely free."""
tests/test_reusability.py
+11 -3
View File
@@ -52,8 +52,16 @@ def test_the_guarded_worst_case_is_at_bisection_level(name, rule):
def test_the_similarity_method_is_subsumed_by_chandrupatla():
"""Stated as a test so the paper cannot drift back to claiming otherwise.
Chandrupatla is better on total, on the worst case, and on stability,
with the same number of constants (none) and the same 193 solved."""
Chandrupatla is better on total, with the same number of constants (none)
and the same 193 solved.
CORRECTED 2026-09-21. This test used to assert that he is also better on
the worst single problem. That held only while our implementation of his
method stopped on a bracket twice as wide as every other method's; with
the tolerance corrected (it now matches scipy's elementwise.find_root on
all 199 problems) his worst problem costs 84 evaluations against the
credit rule's 83. The worst cases are level, to one evaluation, and the
test now says exactly that."""
from simroot.pure import pure_root
XTOL, MAXFEV, ACC = 1e-12, 200, 1e-6
ours_t = ours_w = ours_s = 0
@@ -79,4 +87,4 @@ def test_the_similarity_method_is_subsumed_by_chandrupatla():
cha_w = max(cha_w, n) if ok else cha_w
assert cha_s == ours_s == 193
assert cha_t < ours_t, (cha_t, ours_t)
assert cha_w <= ours_w, (cha_w, ours_w)
assert abs(cha_w - ours_w) <= 1, (cha_w, ours_w)
tests/test_revision_points.py
+7 -4
View File
@@ -168,10 +168,13 @@ def test_charged_totals_are_what_section_5_reports():
root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
with open(os.path.join(root, "sweep_full.json"), encoding="utf8") as fh:
rows = {r["label"]: r for r in json.load(fh)}
# brent is now run at the better of ours and scipy.brentq per problem,
# which is why it is 6988 rather than the 6991 of our implementation alone
# since 2026-09-21 the competitors are the library alone: brent is
# scipy.brentq (7004; our pure-Python implementation, timed in section 5.2,
# is 6991) and chandrupatla is scipy's elementwise.find_root (5228; the
# earlier 5208 came from a per-problem best of ours and the library, with
# ours stopping on a bracket twice as wide)
for lbl, want in (("pure.ratio.derived3.project", 6026),
("brent", 6988), ("bisection", 10620),
("chandrupatla", 5208)):
("brent", 7004), ("bisection", 10620),
("chandrupatla", 5228)):
got = rows[lbl]["APS"]["total"] + rows[lbl]["CHA"]["total"]
assert got == want, (lbl, got, want)