Sign tests by product underflowed: FIXED everywhere (branch, not master - the paper is not yet updated). FA * fnew < 0.0 with FA ~ -1e-200, fnew ~ 1e-150 is -0.0 and '-0.0 < 0.0' is False, so on flat functions (x^19) the bracket update moved the wrong end, the root left the bracket and the run still reported convergence - the proposed rule returned 4.9e-9, bisection 3.5e-9, ITP 1.6e-8 with a claimed 1e-12 bracket; the 1e-6 harness scored them solved. Same defect in our bisection, false-position family, Ridders, Brent, ITP and twelve copies in stage scripts, simroot/sim.py and one test; all replaced by _opposite / _same_sign, pinned by test_sign_underflow.
Impact, every sweep regenerated: the paper's key configurations are unchanged to the evaluation (proposed 2806/3220, credit 3568/3020, Chandrupatla, TOMS 748, Brent, bisection); 46 stated numbers move (ITP CHA 2857 -> 2896, ITP oracle 2611 -> 2647, record CHA figures by 14-18, the grid means by ~1%) and one finding weakens: the measured-order exponent of section 7.1 was cheaper in 29 of 60 far cells and 7 of 7 safeguards; with the fix 12 of 61 and 6 of 7, and on the wide-bracket set it now loses in every safeguard and drops 5 problems. PREREG G1-G3 outcome on the fixed code: G2 (repeating step pattern, period from the data) PASSES Y1-Y4 and is ACCEPTED by the stop rule - proposed rule APS 2806 -> 2782 (below Chandrupatla's 2787), CHA 3220 -> 3169, both 6026 -> 5951, per problem vs Chandrupatla on APS 95/20/38 p = 8e-7. G1 fails Y1/Y2 (its Y3 miss was the sign bug), G3 fails Y2, A fails Y2. Regression 887 pass; the two red tests are the paper-number check (awaiting the paper update) and test_blend (fixed since, copy in stage16). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,227 @@
|
||||
OK number of configurations got 264 want 264
|
||||
|
||||
The proposed method (projection, multiplier declared):
|
||||
OK APS place got 3 want 3
|
||||
OK APS total got 2806 want 2806
|
||||
OK APS solved got 153 want 153
|
||||
BAD CHA place got 45 want 47
|
||||
OK CHA total got 3220 want 3220
|
||||
OK CHA solved got 40 want 40
|
||||
OK both sets got 6026 want 6026
|
||||
|
||||
Head to head of the proposed method:
|
||||
OK APS vs chandrupatla got (87, 24, 42) want (87, 24, 42)
|
||||
OK APS vs toms748 got (86, 16, 51) want (86, 16, 51)
|
||||
OK APS vs brent got (102, 11, 40) want (102, 11, 40)
|
||||
OK APS vs itp got (132, 3, 18) want (132, 3, 18)
|
||||
OK APS vs itp_oracle got (121, 4, 28) want (121, 4, 28)
|
||||
OK APS vs bisection got (152, 1, 0) want (152, 1, 0)
|
||||
OK CHA vs chandrupatla got (9, 5, 26) want (9, 5, 26)
|
||||
OK CHA vs toms748 got (31, 1, 8) want (31, 1, 8)
|
||||
OK CHA vs brent got (30, 4, 4) want (30, 4, 4)
|
||||
OK CHA vs itp got (17, 0, 23) want (17, 0, 23)
|
||||
OK CHA vs itp_oracle got (16, 0, 24) want (16, 0, 24)
|
||||
OK APS vs Chandrupatla sign test p < 1e-4 got True want True
|
||||
OK CHA vs Chandrupatla sign test p (rounded) got 0.006 want 0.006
|
||||
|
||||
The constant-free variant (earned projection):
|
||||
OK credit APS place got 29 want 29
|
||||
OK credit APS total got 3568 want 3568
|
||||
OK credit CHA place got 10 want 10
|
||||
OK credit CHA total got 3020 want 3020
|
||||
OK credit both sets got 6588 want 6588
|
||||
OK credit costs 762 more on the canonical set got 762 want 762
|
||||
OK credit APS vs chandrupatla got (79, 3, 71) want (79, 3, 71)
|
||||
OK credit APS vs toms748 got (77, 9, 67) want (77, 9, 67)
|
||||
OK credit APS vs brent got (78, 5, 70) want (78, 5, 70)
|
||||
OK credit APS vs Chandrupatla sign test p (rounded) got 0.57 want 0.57
|
||||
|
||||
The guards that replace the step instead of projecting it:
|
||||
OK APS unguarded got 2806 want 2806
|
||||
OK CHA unguarded total got 4104 want 4104
|
||||
OK CHA unguarded solved got 35 want 35
|
||||
OK CHA unguarded bound violations got 10 want 10
|
||||
OK APS bracket-guard got 7260 want 7260
|
||||
OK APS both-guard got 7239 want 7239
|
||||
|
||||
Alefeld-Potra-Shi:
|
||||
OK APS winner got 'chandrupatla' want 'chandrupatla'
|
||||
OK APS chandrupatla place got 1 want 1
|
||||
OK APS chandrupatla total got 2787 want 2787
|
||||
OK APS toms748 place got 4 want 4
|
||||
OK APS toms748 total got 2820 want 2820
|
||||
OK APS brent place got 5 want 5
|
||||
OK APS brent total got 2889 want 2889
|
||||
OK APS dekker place got 12 want 12
|
||||
OK APS dekker total got 3158 want 3158
|
||||
OK APS itp_oracle place got 13 want 13
|
||||
OK APS itp_oracle total got 3260 want 3260
|
||||
OK APS ridders place got 20 want 20
|
||||
OK APS ridders total got 3313 want 3313
|
||||
OK APS itp place got 34 want 34
|
||||
OK APS itp total got 3957 want 3957
|
||||
OK APS illinois place got 73 want 73
|
||||
OK APS illinois total got 7045 want 7045
|
||||
OK APS bisection place got 79 want 79
|
||||
OK APS bisection total got 7530 want 7530
|
||||
OK APS regula_falsi place got 244 want 244
|
||||
OK APS regula_falsi total got 10560 want 10560
|
||||
OK APS regula falsi solved got 127 want 127
|
||||
|
||||
Chandrupatla:
|
||||
OK CHA rank 1 label got 'pure.sqrt.bracket.project' want 'pure.sqrt.bracket.project'
|
||||
OK CHA rank 1 solves one more than anyone else got 41 want 41
|
||||
OK CHA chandrupatla place got 2 want 2
|
||||
OK CHA chandrupatla total got 2441 want 2441
|
||||
OK CHA itp_oracle place got 3 want 3
|
||||
BAD CHA itp_oracle total got 2647 want 2611
|
||||
OK CHA itp place got 6 want 6
|
||||
BAD CHA itp total got 2896 want 2857
|
||||
OK CHA bisection place got 16 want 16
|
||||
OK CHA bisection total got 3090 want 3090
|
||||
BAD CHA toms748 place got 159 want 164
|
||||
OK CHA toms748 total got 4000 want 4000
|
||||
BAD CHA brent place got 227 want 226
|
||||
OK CHA brent total got 4115 want 4115
|
||||
OK CHA brent solved got 38 want 38
|
||||
OK chandrupatla, both sets got 5228 want 5228
|
||||
OK Chandrupatla beats us on both sets got True want True
|
||||
|
||||
Negative results, the regime claims:
|
||||
BAD CHA cbrt over the whole set got 4109 want 4099
|
||||
OK single fixed choice, both sets got 6026 want 6026
|
||||
BAD switch by benchmark set, both sets got 6915 want 6905
|
||||
OK switching is worse than not switching got True want True
|
||||
OK fun4.1 projection variant got 89 want 89
|
||||
OK fun4.1 cbrt got 32 want 32
|
||||
OK fun4.2 projection variant got 93 want 93
|
||||
OK fun4.2 cbrt got 32 want 32
|
||||
OK fun4.3 projection variant got 113 want 113
|
||||
OK fun4.3 cbrt got 40 want 40
|
||||
OK fun4.4 projection variant got 125 want 125
|
||||
OK fun4.4 cbrt got 47 want 47
|
||||
OK fun4.5 projection variant got 153 want 153
|
||||
OK fun4.5 cbrt got 59 want 59
|
||||
|
||||
The law across the grid:
|
||||
BAD mean total, scale=derived3, lin got 6893 want 6809
|
||||
BAD mean total, scale=derived3, quad got 22315 want 22294
|
||||
BAD mean total, scale=derived, lin got 6981 want 6906
|
||||
BAD mean total, scale=derived, quad got 22206 want 22167
|
||||
BAD mean total, scale=bracket, lin got 10279 want 10263
|
||||
BAD mean total, scale=bracket, quad got 19127 want 19137
|
||||
OK mean total, scale=start, lin got 16102 want 16102
|
||||
OK mean total, scale=start, quad got 15345 want 15345
|
||||
OK every c=1 similarity beats every c=0 similarity got True want True
|
||||
OK similarity ranking got ['ratio', 'exp_abs', 'rational', 'ratio_soft', 'sqrt', 'cbrt', 'exp_sq', 'rat_sq', 'cosine'] want ['ratio', 'exp_abs', 'rational', 'ratio_soft', 'sqrt', 'cbrt', 'exp_sq', 'rat_sq', 'cosine']
|
||||
BAD similarity ratio best got 6560 want 6496
|
||||
BAD similarity exp_abs best got 6895 want 6822
|
||||
BAD similarity rational best got 7118 want 7038
|
||||
BAD similarity ratio_soft best got 9578 want 9502
|
||||
BAD similarity sqrt best got 10546 want 10497
|
||||
BAD similarity cbrt best got 12002 want 11941
|
||||
BAD similarity exp_sq best got 14307 want 14314
|
||||
OK similarity rat_sq best got 15102 want 15102
|
||||
OK similarity cosine best got 16394 want 16394
|
||||
OK scale ranking got ['derived3', 'derived', 'bracket', 'start'] want ['derived3', 'derived', 'bracket', 'start']
|
||||
BAD scale derived3 best got 6560 want 6496
|
||||
BAD scale derived best got 6702 want 6628
|
||||
BAD scale bracket best got 9107 want 9081
|
||||
OK scale start best got 14540 want 14540
|
||||
|
||||
The placement modes (each in its own sweep file):
|
||||
OK slope.ratio.derived3.project APS got 2848 want 2848
|
||||
BAD slope.ratio.derived3.project CHA got 3114 want 3074
|
||||
OK slope.ratio.derived3.project solved got 193 want 193
|
||||
OK slope.ratio.derived3.project guarantee (over 2x) got 0 want 0
|
||||
OK pure+shape+slope.ratio.derived3.safe APS got 2527 want 2527
|
||||
BAD pure+shape+slope.ratio.derived3.safe CHA got 3126 want 3108
|
||||
OK pure+shape+slope.ratio.derived3.safe solved got 193 want 193
|
||||
OK pure+shape+slope.ratio.derived3.safe guarantee (over 2x) got 0 want 0
|
||||
OK pure+dissim+slope.ratio.derived3.project APS got 2726 want 2726
|
||||
BAD pure+dissim+slope.ratio.derived3.project CHA got 2799 want 2785
|
||||
OK pure+dissim+slope.ratio.derived3.project solved got 193 want 193
|
||||
OK pure+dissim+slope.ratio.derived3.project guarantee (over 2x) got 0 want 0
|
||||
OK pure+shape+dissim+slope.ratio.derived3.safe APS got 2884 want 2884
|
||||
BAD pure+shape+dissim+slope.ratio.derived3.safe CHA got 2871 want 2859
|
||||
OK pure+shape+dissim+slope.ratio.derived3.safe solved got 193 want 193
|
||||
OK pure+shape+dissim+slope.ratio.derived3.safe guarantee (over 2x) got 0 want 0
|
||||
!! gate.ratio.derived.project not in sweep_gate.json — stale sweep
|
||||
OK log.ratio_soft.derived.safe APS got 6940 want 6940
|
||||
BAD log.ratio_soft.derived.safe CHA got 3541 want 3474
|
||||
OK log.ratio_soft.derived.safe solved got 193 want 193
|
||||
OK log.ratio_soft.derived.safe guarantee (over 2x) got 0 want 0
|
||||
BAD pure+dissim+slope.ratio.derived3.safe CHA per-set got 2695 want 2683
|
||||
|
||||
The per-place reading (sweep_order.json):
|
||||
OK far+rational both APS m=1 got 4522 want 4522
|
||||
OK far+rational both APS order got 3675 want 3675
|
||||
BAD far+rational both CHA m=1 got 3406 want 3387
|
||||
BAD far+rational both CHA order got 3906 want 3216
|
||||
OK far+rational bracket APS m=1 got 4606 want 4606
|
||||
OK far+rational bracket APS order got 3542 want 3542
|
||||
BAD far+rational bracket CHA m=1 got 3602 want 3571
|
||||
BAD far+rational bracket CHA order got 3950 want 3303
|
||||
OK far+rational credit APS m=1 got 5190 want 5190
|
||||
OK far+rational credit APS order got 4686 want 4686
|
||||
OK far+rational credit CHA m=1 got 3013 want 3013
|
||||
BAD far+rational credit CHA order got 3646 want 2884
|
||||
OK far+rational project APS m=1 got 6932 want 6932
|
||||
OK far+rational project APS order got 6364 want 6364
|
||||
BAD far+rational project CHA m=1 got 4789 want 4794
|
||||
BAD far+rational project CHA order got 5098 want 4559
|
||||
OK far+rational safe APS m=1 got 5987 want 5987
|
||||
OK far+rational safe APS order got 5229 want 5229
|
||||
BAD far+rational safe CHA m=1 got 4134 want 4108
|
||||
BAD far+rational safe CHA order got 4503 want 3923
|
||||
OK far+rational step APS m=1 got 6435 want 6435
|
||||
OK far+rational step APS order got 5551 want 5551
|
||||
BAD far+rational step CHA m=1 got 4287 want 4257
|
||||
BAD far+rational step CHA order got 4602 want 4022
|
||||
OK far+rational none APS m=1 got 10676 want 10676
|
||||
OK far+rational none APS order got 10043 want 10043
|
||||
OK far+rational none CHA m=1 got 8265 want 8265
|
||||
OK far+rational none CHA order got 7794 want 7794
|
||||
BAD cheaper in 7 of 7 safeguards got 6 want 7
|
||||
OK false position none solved got 127 want 127
|
||||
OK false position none APS got 10676 want 10676
|
||||
OK false position order solved got 128 want 128
|
||||
OK false position order APS got 10043 want 10043
|
||||
OK slope reading none APS got 2806 want 2806
|
||||
OK slope reading order APS got 2854 want 2854
|
||||
OK guarantee family violations got 0 want 0
|
||||
|
||||
The decision place (sweep_decision.json):
|
||||
OK ratio.band.project APS got 5432 want 5432
|
||||
BAD ratio.band.project CHA got 3059 want 3066
|
||||
BAD ratio.band.credit CHA got 2892 want 2899
|
||||
OK ratio.none.project APS got 2806 want 2806
|
||||
OK ratio.defect.project APS got 2523 want 2523
|
||||
OK ratio.defect.project CHA got 3210 want 3210
|
||||
BAD ratio_soft.simpair.project CHA got 2897 want 2868
|
||||
OK defect APS solved got 153 want 153
|
||||
OK defect both sets got 5733 want 5733
|
||||
OK defect is the new canonical best (< 2527) got True want True
|
||||
OK decision grid guarantee violations got 0 want 0
|
||||
BAD band refusal rate % got 63.5 want 63.6
|
||||
BAD defect refusal rate % got 6.6 want 7.0
|
||||
|
||||
The gap decomposition (fresh runs of the diagnosis scripts):
|
||||
Traceback (most recent call last):
|
||||
File "/home/valeri/sim-numerics/verify_article.py", line 558, in <module>
|
||||
main()
|
||||
File "/home/valeri/sim-numerics/verify_article.py", line 220, in main
|
||||
verify_gap_decomposition()
|
||||
File "/home/valeri/sim-numerics/verify_article.py", line 388, in verify_gap_decomposition
|
||||
fl = diag_flat_detectable.main()
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/home/valeri/sim-numerics/diag_flat_detectable.py", line 54, in main
|
||||
groups[key] += states(d["f"], a, b, float(d["root"]))
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/home/valeri/sim-numerics/diag_flat_detectable.py", line 40, in states
|
||||
out.append(_local_order(s["x"], s["fx"], s["xp"], s["fp"], s["xp2"], s["fp2"]))
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/home/valeri/sim-numerics/simroot/pure.py", line 592, in _local_order
|
||||
theta = f2 * fpp / (fp * fp)
|
||||
~~~~~~~~~^~~~~~~~~~~
|
||||
ZeroDivisionError: float division by zero
|
||||
Reference in New Issue
Block a user