Files
sim-numerics/tests/test_baselines_library.py
T
vvsandClaude Opus 5 1b6c626f67 Valeri's decisions (1) competitors from the library, (2) variant a. (1) Chandrupatla is scipy.optimize.elementwise.find_root, Brent is brentq, no more per-problem best_of. Our Chandrupatla stopped on a bracket twice as wide as every other method (the factor the protocol corrects for ITP); fixed (+0.5*xtol) in baselines.py and in the two copies found by grep (stage16_blend, stage19_modern) and the traced mirror, and pinned: ours == library on every problem. Regenerating sweep_full.json changed exactly three rows - chandrupatla APS 2785->2787, CHA 2423->2441, brent CHA 4099->4115 - and nothing else (diff_sweeps.py: 0 unexpected).
(2) The proposed method is again the scheduled projection (2806/3220/6026), its budget multiplier DECLARED as the one free constant: the canonical total is the same for every multiplier 1.25-4, the wide-bracket one is not. The earned projection is the constant-free variant (3568/3020/6588). What the old tables hid: problem by problem the proposed rule beats Chandrupatla on the canonical set 87/24/42, exact sign test p = 9e-5, while trailing him by 0.7% in total; on the wide-bracket set he is genuinely stronger (26 to 9, p = 0.006). Gap decomposition recomputed with the library: 244 (was 262), 160 of it on the flat families (two thirds), fun1 58 (a quarter, not a third as the abstract said).

Also corrected on the way: the 'subsumed by Chandrupatla' pin claimed he is better on the worst single problem - true only with the looser tolerance (84 against the credit rule's 83 once fixed); §8.3 still called zero slack degenerate, which Table 1a refutes; §5.2's Brent figure is our pure-Python implementation and now says so. verify_article.py 286/286, 802 tests pass twice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-21 10:53:00 +01:00

59 lines
2.3 KiB
Python

"""Pins for the 2026-09-21 switch of the competitors to the library.
REGRESSION: our Chandrupatla stopped on a bracket twice as wide as every other method
(tl = tol/span > 1/2 with tol = xtol), and best_of let that looser run into the
reported numbers. The benchmark now uses the library alone, and our implementation -
kept for the timing section and the traced mirror - must agree with it exactly.
"""
from __future__ import annotations
import math
import pytest
from scipy.optimize._tstutils import _APS_TESTS_DICTS, _CHANDRUPATLA_TESTS_DICTS
from simroot.baselines import (BRENT_BASELINE, CHANDRUPATLA_BASELINE, brent_scipy,
chandrupatla, chandrupatla_lib)
XTOL, MAXFEV = 1e-12, 200
def _problems():
out = []
for d in _APS_TESTS_DICTS[::7]:
f = (lambda ff, aa: lambda t: ff(t, *aa))(d["f"], d.get("args", ()))
out.append((d["ID"], f, float(d["bracket"][0]), float(d["bracket"][1])))
for d in _CHANDRUPATLA_TESTS_DICTS[::3]:
out.append((d["ID"], d["f"], float(d["bracket"][0]), float(d["bracket"][1])))
return out
def test_the_benchmark_competitors_are_the_library():
assert CHANDRUPATLA_BASELINE is chandrupatla_lib
assert BRENT_BASELINE is brent_scipy
def test_the_sweep_uses_them():
import sweep_full
assert sweep_full.BASELINES["chandrupatla"] is chandrupatla_lib
assert sweep_full.BASELINES["brent"] is brent_scipy
@pytest.mark.parametrize("pid,f,a,b", _problems())
def test_our_chandrupatla_spends_exactly_what_the_library_spends(pid, f, a, b):
"""The tolerance fix, pinned: same evaluation count and same verdict."""
x1, n1, c1 = chandrupatla(f, a, b, XTOL, MAXFEV)
x2, n2, c2 = chandrupatla_lib(f, a, b, XTOL, MAXFEV)
assert (n1, bool(c1)) == (n2, bool(c2)), (pid, n1, n2)
if c1 and c2 and math.isfinite(x1) and math.isfinite(x2):
assert abs(x1 - x2) <= 2 * XTOL + 1e-15 * max(1.0, abs(x1))
def test_our_chandrupatla_no_longer_stops_on_a_double_width_bracket():
"""On a smooth problem the looser rule saved an evaluation; with the fix our count
must equal the library's rather than undercut it."""
f = lambda t: math.exp(t) - 2.0 # noqa: E731
_x, n_ours, _c = chandrupatla(f, 0.0, 3.0, XTOL, MAXFEV)
_x, n_lib, _c = chandrupatla_lib(f, 0.0, 3.0, XTOL, MAXFEV)
assert n_ours == n_lib