Valeri's decisions (1) competitors from the library, (2) variant a. (1) Chandrupatla is scipy.optimize.elementwise.find_root, Brent is brentq, no more per-problem best_of. Our Chandrupatla stopped on a bracket twice as wide as every other method (the factor the protocol corrects for ITP); fixed (+0.5*xtol) in baselines.py and in the two copies found by grep (stage16_blend, stage19_modern) and the traced mirror, and pinned: ours == library on every problem. Regenerating sweep_full.json changed exactly three rows - chandrupatla APS 2785->2787, CHA 2423->2441, brent CHA 4099->4115 - and nothing else (diff_sweeps.py: 0 unexpected).
(2) The proposed method is again the scheduled projection (2806/3220/6026), its budget multiplier DECLARED as the one free constant: the canonical total is the same for every multiplier 1.25-4, the wide-bracket one is not. The earned projection is the constant-free variant (3568/3020/6588). What the old tables hid: problem by problem the proposed rule beats Chandrupatla on the canonical set 87/24/42, exact sign test p = 9e-5, while trailing him by 0.7% in total; on the wide-bracket set he is genuinely stronger (26 to 9, p = 0.006). Gap decomposition recomputed with the library: 244 (was 262), 160 of it on the flat families (two thirds), fun1 58 (a quarter, not a third as the abstract said). Also corrected on the way: the 'subsumed by Chandrupatla' pin claimed he is better on the worst single problem - true only with the looser tolerance (84 against the credit rule's 83 once fixed); §8.3 still called zero slack degenerate, which Table 1a refutes; §5.2's Brent figure is our pure-Python implementation and now says so. verify_article.py 286/286, 802 tests pass twice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,318 @@
|
||||
OK number of configurations got 264 want 264
|
||||
|
||||
The proposed method (projection, multiplier declared):
|
||||
OK APS place got 3 want 3
|
||||
OK APS total got 2806 want 2806
|
||||
OK APS solved got 153 want 153
|
||||
OK CHA place got 47 want 47
|
||||
OK CHA total got 3220 want 3220
|
||||
OK CHA solved got 40 want 40
|
||||
OK both sets got 6026 want 6026
|
||||
|
||||
Head to head of the proposed method:
|
||||
OK APS vs chandrupatla got (87, 24, 42) want (87, 24, 42)
|
||||
OK APS vs toms748 got (86, 16, 51) want (86, 16, 51)
|
||||
OK APS vs brent got (102, 11, 40) want (102, 11, 40)
|
||||
OK APS vs itp got (132, 3, 18) want (132, 3, 18)
|
||||
OK APS vs itp_oracle got (121, 4, 28) want (121, 4, 28)
|
||||
OK APS vs bisection got (152, 1, 0) want (152, 1, 0)
|
||||
OK CHA vs chandrupatla got (9, 5, 26) want (9, 5, 26)
|
||||
OK CHA vs toms748 got (31, 1, 8) want (31, 1, 8)
|
||||
OK CHA vs brent got (30, 4, 4) want (30, 4, 4)
|
||||
OK CHA vs itp got (17, 0, 23) want (17, 0, 23)
|
||||
OK CHA vs itp_oracle got (16, 0, 24) want (16, 0, 24)
|
||||
OK APS vs Chandrupatla sign test p < 1e-4 got True want True
|
||||
OK CHA vs Chandrupatla sign test p (rounded) got 0.006 want 0.006
|
||||
|
||||
The constant-free variant (earned projection):
|
||||
OK credit APS place got 29 want 29
|
||||
OK credit APS total got 3568 want 3568
|
||||
OK credit CHA place got 10 want 10
|
||||
OK credit CHA total got 3020 want 3020
|
||||
OK credit both sets got 6588 want 6588
|
||||
OK credit costs 762 more on the canonical set got 762 want 762
|
||||
OK credit APS vs chandrupatla got (79, 3, 71) want (79, 3, 71)
|
||||
OK credit APS vs toms748 got (77, 9, 67) want (77, 9, 67)
|
||||
OK credit APS vs brent got (78, 5, 70) want (78, 5, 70)
|
||||
OK credit APS vs Chandrupatla sign test p (rounded) got 0.57 want 0.57
|
||||
|
||||
The guards that replace the step instead of projecting it:
|
||||
OK APS unguarded got 2806 want 2806
|
||||
OK CHA unguarded total got 4104 want 4104
|
||||
OK CHA unguarded solved got 35 want 35
|
||||
OK CHA unguarded bound violations got 10 want 10
|
||||
OK APS bracket-guard got 7260 want 7260
|
||||
OK APS both-guard got 7239 want 7239
|
||||
|
||||
Alefeld-Potra-Shi:
|
||||
OK APS winner got 'chandrupatla' want 'chandrupatla'
|
||||
OK APS chandrupatla place got 1 want 1
|
||||
OK APS chandrupatla total got 2787 want 2787
|
||||
OK APS toms748 place got 4 want 4
|
||||
OK APS toms748 total got 2820 want 2820
|
||||
OK APS brent place got 5 want 5
|
||||
OK APS brent total got 2889 want 2889
|
||||
OK APS dekker place got 12 want 12
|
||||
OK APS dekker total got 3158 want 3158
|
||||
OK APS itp_oracle place got 13 want 13
|
||||
OK APS itp_oracle total got 3260 want 3260
|
||||
OK APS ridders place got 20 want 20
|
||||
OK APS ridders total got 3313 want 3313
|
||||
OK APS itp place got 34 want 34
|
||||
OK APS itp total got 3957 want 3957
|
||||
OK APS illinois place got 73 want 73
|
||||
OK APS illinois total got 7045 want 7045
|
||||
OK APS bisection place got 79 want 79
|
||||
OK APS bisection total got 7530 want 7530
|
||||
OK APS regula_falsi place got 244 want 244
|
||||
OK APS regula_falsi total got 10560 want 10560
|
||||
OK APS regula falsi solved got 127 want 127
|
||||
|
||||
Chandrupatla:
|
||||
OK CHA rank 1 label got 'pure.sqrt.bracket.project' want 'pure.sqrt.bracket.project'
|
||||
OK CHA rank 1 solves one more than anyone else got 41 want 41
|
||||
OK CHA chandrupatla place got 2 want 2
|
||||
OK CHA chandrupatla total got 2441 want 2441
|
||||
OK CHA itp_oracle place got 3 want 3
|
||||
OK CHA itp_oracle total got 2611 want 2611
|
||||
OK CHA itp place got 6 want 6
|
||||
OK CHA itp total got 2857 want 2857
|
||||
OK CHA bisection place got 16 want 16
|
||||
OK CHA bisection total got 3090 want 3090
|
||||
OK CHA toms748 place got 164 want 164
|
||||
OK CHA toms748 total got 4000 want 4000
|
||||
OK CHA brent place got 226 want 226
|
||||
OK CHA brent total got 4115 want 4115
|
||||
OK CHA brent solved got 38 want 38
|
||||
OK chandrupatla, both sets got 5228 want 5228
|
||||
OK Chandrupatla beats us on both sets got True want True
|
||||
|
||||
Negative results, the regime claims:
|
||||
OK CHA cbrt over the whole set got 4099 want 4099
|
||||
OK single fixed choice, both sets got 6026 want 6026
|
||||
OK switch by benchmark set, both sets got 6905 want 6905
|
||||
OK switching is worse than not switching got True want True
|
||||
OK fun4.1 projection variant got 89 want 89
|
||||
OK fun4.1 cbrt got 32 want 32
|
||||
OK fun4.2 projection variant got 93 want 93
|
||||
OK fun4.2 cbrt got 32 want 32
|
||||
OK fun4.3 projection variant got 113 want 113
|
||||
OK fun4.3 cbrt got 40 want 40
|
||||
OK fun4.4 projection variant got 125 want 125
|
||||
OK fun4.4 cbrt got 47 want 47
|
||||
OK fun4.5 projection variant got 153 want 153
|
||||
OK fun4.5 cbrt got 59 want 59
|
||||
|
||||
The law across the grid:
|
||||
OK mean total, scale=derived3, lin got 6809 want 6809
|
||||
OK mean total, scale=derived3, quad got 22294 want 22294
|
||||
OK mean total, scale=derived, lin got 6906 want 6906
|
||||
OK mean total, scale=derived, quad got 22167 want 22167
|
||||
OK mean total, scale=bracket, lin got 10263 want 10263
|
||||
OK mean total, scale=bracket, quad got 19137 want 19137
|
||||
OK mean total, scale=start, lin got 16102 want 16102
|
||||
OK mean total, scale=start, quad got 15345 want 15345
|
||||
OK every c=1 similarity beats every c=0 similarity got True want True
|
||||
OK similarity ranking got ['ratio', 'exp_abs', 'rational', 'ratio_soft', 'sqrt', 'cbrt', 'exp_sq', 'rat_sq', 'cosine'] want ['ratio', 'exp_abs', 'rational', 'ratio_soft', 'sqrt', 'cbrt', 'exp_sq', 'rat_sq', 'cosine']
|
||||
OK similarity ratio best got 6496 want 6496
|
||||
OK similarity exp_abs best got 6822 want 6822
|
||||
OK similarity rational best got 7038 want 7038
|
||||
OK similarity ratio_soft best got 9502 want 9502
|
||||
OK similarity sqrt best got 10497 want 10497
|
||||
OK similarity cbrt best got 11941 want 11941
|
||||
OK similarity exp_sq best got 14314 want 14314
|
||||
OK similarity rat_sq best got 15102 want 15102
|
||||
OK similarity cosine best got 16394 want 16394
|
||||
OK scale ranking got ['derived3', 'derived', 'bracket', 'start'] want ['derived3', 'derived', 'bracket', 'start']
|
||||
OK scale derived3 best got 6496 want 6496
|
||||
OK scale derived best got 6628 want 6628
|
||||
OK scale bracket best got 9081 want 9081
|
||||
OK scale start best got 14540 want 14540
|
||||
|
||||
The placement modes (each in its own sweep file):
|
||||
OK slope.ratio.derived3.project APS got 2848 want 2848
|
||||
OK slope.ratio.derived3.project CHA got 3074 want 3074
|
||||
OK slope.ratio.derived3.project solved got 193 want 193
|
||||
OK slope.ratio.derived3.project guarantee (over 2x) got 0 want 0
|
||||
OK pure+shape+slope.ratio.derived3.safe APS got 2527 want 2527
|
||||
OK pure+shape+slope.ratio.derived3.safe CHA got 3108 want 3108
|
||||
OK pure+shape+slope.ratio.derived3.safe solved got 193 want 193
|
||||
OK pure+shape+slope.ratio.derived3.safe guarantee (over 2x) got 0 want 0
|
||||
OK pure+dissim+slope.ratio.derived3.project APS got 2726 want 2726
|
||||
OK pure+dissim+slope.ratio.derived3.project CHA got 2785 want 2785
|
||||
OK pure+dissim+slope.ratio.derived3.project solved got 193 want 193
|
||||
OK pure+dissim+slope.ratio.derived3.project guarantee (over 2x) got 0 want 0
|
||||
OK pure+shape+dissim+slope.ratio.derived3.safe APS got 2884 want 2884
|
||||
OK pure+shape+dissim+slope.ratio.derived3.safe CHA got 2859 want 2859
|
||||
OK pure+shape+dissim+slope.ratio.derived3.safe solved got 193 want 193
|
||||
OK pure+shape+dissim+slope.ratio.derived3.safe guarantee (over 2x) got 0 want 0
|
||||
OK gate.ratio.derived.project APS got 4449 want 4449
|
||||
OK gate.ratio.derived.project CHA got 2991 want 2991
|
||||
OK gate.ratio.derived.project solved got 193 want 193
|
||||
OK gate.ratio.derived.project guarantee (over 2x) got 0 want 0
|
||||
OK log.ratio_soft.derived.safe APS got 6940 want 6940
|
||||
OK log.ratio_soft.derived.safe CHA got 3474 want 3474
|
||||
OK log.ratio_soft.derived.safe solved got 193 want 193
|
||||
OK log.ratio_soft.derived.safe guarantee (over 2x) got 0 want 0
|
||||
OK pure+dissim+slope.ratio.derived3.safe CHA per-set got 2683 want 2683
|
||||
OK gate.ratio_soft.derived.safe CHA per-set got 2851 want 2851
|
||||
|
||||
The per-place reading (sweep_order.json):
|
||||
OK far+rational both APS m=1 got 4522 want 4522
|
||||
OK far+rational both APS order got 3675 want 3675
|
||||
OK far+rational both CHA m=1 got 3387 want 3387
|
||||
OK far+rational both CHA order got 3216 want 3216
|
||||
OK far+rational bracket APS m=1 got 4606 want 4606
|
||||
OK far+rational bracket APS order got 3542 want 3542
|
||||
OK far+rational bracket CHA m=1 got 3571 want 3571
|
||||
OK far+rational bracket CHA order got 3303 want 3303
|
||||
OK far+rational credit APS m=1 got 5190 want 5190
|
||||
OK far+rational credit APS order got 4686 want 4686
|
||||
OK far+rational credit CHA m=1 got 3013 want 3013
|
||||
OK far+rational credit CHA order got 2884 want 2884
|
||||
OK far+rational project APS m=1 got 6932 want 6932
|
||||
OK far+rational project APS order got 6364 want 6364
|
||||
OK far+rational project CHA m=1 got 4794 want 4794
|
||||
OK far+rational project CHA order got 4559 want 4559
|
||||
OK far+rational safe APS m=1 got 5987 want 5987
|
||||
OK far+rational safe APS order got 5229 want 5229
|
||||
OK far+rational safe CHA m=1 got 4108 want 4108
|
||||
OK far+rational safe CHA order got 3923 want 3923
|
||||
OK far+rational step APS m=1 got 6435 want 6435
|
||||
OK far+rational step APS order got 5551 want 5551
|
||||
OK far+rational step CHA m=1 got 4257 want 4257
|
||||
OK far+rational step CHA order got 4022 want 4022
|
||||
OK far+rational none APS m=1 got 10676 want 10676
|
||||
OK far+rational none APS order got 10043 want 10043
|
||||
OK far+rational none CHA m=1 got 8265 want 8265
|
||||
OK far+rational none CHA order got 7794 want 7794
|
||||
OK cheaper in 7 of 7 safeguards got 7 want 7
|
||||
OK false position none solved got 127 want 127
|
||||
OK false position none APS got 10676 want 10676
|
||||
OK false position order solved got 128 want 128
|
||||
OK false position order APS got 10043 want 10043
|
||||
OK slope reading none APS got 2806 want 2806
|
||||
OK slope reading order APS got 2854 want 2854
|
||||
OK guarantee family violations got 0 want 0
|
||||
|
||||
The decision place (sweep_decision.json):
|
||||
OK ratio.band.project APS got 5432 want 5432
|
||||
OK ratio.band.project CHA got 3066 want 3066
|
||||
OK ratio.band.credit CHA got 2899 want 2899
|
||||
OK ratio.none.project APS got 2806 want 2806
|
||||
OK ratio.defect.project APS got 2523 want 2523
|
||||
OK ratio.defect.project CHA got 3210 want 3210
|
||||
OK ratio_soft.simpair.project CHA got 2868 want 2868
|
||||
OK defect APS solved got 153 want 153
|
||||
OK defect both sets got 5733 want 5733
|
||||
OK defect is the new canonical best (< 2527) got True want True
|
||||
OK decision grid guarantee violations got 0 want 0
|
||||
OK band refusal rate % got 63.6 want 63.6
|
||||
OK defect refusal rate % got 7.0 want 7.0
|
||||
|
||||
The gap decomposition (fresh runs of the diagnosis scripts):
|
||||
OK problems where we lose got 33 want 33
|
||||
OK fun1.4 our bracket halvings/eval got 0.62 want 0.62
|
||||
OK fun1.4 IQI from our points got 10.37 want 10.37
|
||||
OK flat: ours min got 0.41 want 0.41
|
||||
OK flat: ours max got 0.81 want 0.81
|
||||
OK flat: IQI min got 0.01 want 0.01
|
||||
OK flat: IQI max got 0.17 want 0.17
|
||||
OK non-degenerate states with one negative weight got np.int64(428) want 428
|
||||
OK bracketing states in total got 433 want 433
|
||||
OK every exception is an underflowed zero weight got True want True
|
||||
OK total gap on the losing problems got 244 want 244
|
||||
OK flat families' gap got 160 want 160
|
||||
OK fun1 gap got 58 want 58
|
||||
OK Chandrupatla bisect share fun3 % got 92 want 92
|
||||
OK Chandrupatla bisect share fun4 % got 96 want 96
|
||||
OK Chandrupatla bisect share fun5 % got 100 want 100
|
||||
OK Chandrupatla bisect share fun6 % got 100 want 100
|
||||
OK Chandrupatla total fun4 got 297 want 297
|
||||
OK Chandrupatla total fun5 got 233 want 233
|
||||
OK Chandrupatla total fun6 got 233 want 233
|
||||
OK m>=2 on canonical states % got np.float64(0.2) want 0.2
|
||||
OK m>=2 on flat states % got 5.8 want 5.8
|
||||
OK band fun4 no-gate got 573 want 573
|
||||
OK band fun4 with band got 296 want 296
|
||||
OK band fun5 no-gate got 456 want 456
|
||||
OK band fun5 with band got 239 want 239
|
||||
OK band fun6 no-gate got 456 want 456
|
||||
OK band fun6 with band got 235 want 235
|
||||
OK band cost on fun1 got 164 want 164
|
||||
OK band cost on fun2 got 111 want 111
|
||||
OK band cost on fun8 got 170 want 170
|
||||
OK band cost on fun9 got 144 want 144
|
||||
OK log item: x-space control APS got 5822 want 5822
|
||||
OK log item: x-space control CHA got 3643 want 3643
|
||||
|
||||
§3.1: the multiplier and the endowment (fresh runs):
|
||||
OK multiplier 1.0 APS got 4817 want 4817
|
||||
OK multiplier 1.0 CHA got 2997 want 2997
|
||||
OK multiplier 1.25 APS got 2806 want 2806
|
||||
OK multiplier 1.25 CHA got 2752 want 2752
|
||||
OK multiplier 1.5 APS got 2806 want 2806
|
||||
OK multiplier 1.5 CHA got 2889 want 2889
|
||||
OK multiplier 2.0 APS got 2806 want 2806
|
||||
OK multiplier 2.0 CHA got 3220 want 3220
|
||||
OK multiplier 3.0 APS got 2806 want 2806
|
||||
OK multiplier 3.0 CHA got 3756 want 3756
|
||||
OK multiplier 4.0 APS got 2806 want 2806
|
||||
OK multiplier 4.0 CHA got 4006 want 4006
|
||||
OK endowment 0.1 APS got 5344 want 5344
|
||||
OK endowment 0.1 CHA got 3051 want 3051
|
||||
OK endowment 0.5 APS got 4656 want 4656
|
||||
OK endowment 0.5 CHA got 3002 want 3002
|
||||
OK endowment 1.0 APS got 3568 want 3568
|
||||
OK endowment 1.0 CHA got 3020 want 3020
|
||||
OK endowment 2.0 APS got 3692 want 3692
|
||||
OK endowment 2.0 CHA got 2712 want 2712
|
||||
OK endowment 5.0 APS got 3006 want 3006
|
||||
OK endowment 5.0 CHA got 2615 want 2615
|
||||
OK C0=1 cheapest within the one-step promise got 1.0 want 1.0
|
||||
OK 2527 at multiplier 1.25 got 2541 want 2541
|
||||
OK 2527 flat from 1.5 to 4 got {2527} want {2527}
|
||||
OK 2523 flat from 1.25 to 4 got {2523} want {2523}
|
||||
|
||||
Tables 2-3 regenerated:
|
||||
OK APS pure.ratio.derived3.project place got 3 want 3
|
||||
OK APS pure.ratio.derived3.none place got 2 want 2
|
||||
OK CHA pure.ratio.derived3.project place got 47 want 47
|
||||
OK CHA pure.ratio.derived3.none place got 228 want 228
|
||||
OK CHA pure.exp_abs.bracket.both place got 4 want 4
|
||||
OK CHA exp_abs.bracket.both total got 2733 want 2733
|
||||
OK CHA TOMS 748 over the 2x bound got 15 want 15
|
||||
OK CHA Brent over the 2x bound got 18 want 18
|
||||
OK credit APS vs itp_oracle got (86, 6, 61) want (86, 6, 61)
|
||||
OK credit CHA vs itp_oracle got (2, 3, 35) want (2, 3, 35)
|
||||
OK credit CHA vs bisection got (5, 0, 35) want (5, 0, 35)
|
||||
OK project APS vs chandrupatla got (87, 24, 42) want (87, 24, 42)
|
||||
OK project APS vs itp got (132, 3, 18) want (132, 3, 18)
|
||||
OK project APS vs itp_oracle got (121, 4, 28) want (121, 4, 28)
|
||||
OK project APS vs bisection got (152, 1, 0) want (152, 1, 0)
|
||||
OK project CHA vs toms748 got (31, 1, 8) want (31, 1, 8)
|
||||
OK project CHA vs brent got (30, 4, 4) want (30, 4, 4)
|
||||
OK project CHA vs itp got (17, 0, 23) want (17, 0, 23)
|
||||
OK project CHA vs itp_oracle got (16, 0, 24) want (16, 0, 24)
|
||||
OK both sets pure.ratio.derived3.credit got 6588 want 6588
|
||||
OK both sets pure.ratio.derived3.project got 6026 want 6026
|
||||
OK both sets chandrupatla got 5228 want 5228
|
||||
OK both sets itp_oracle got 5871 want 5871
|
||||
OK both sets toms748 got 6820 want 6820
|
||||
OK both sets brent got 7004 want 7004
|
||||
|
||||
§7 under the earned projection:
|
||||
OK slope.ratio.derived3.credit APS got 3507 want 3507
|
||||
OK slope.ratio.derived3.credit CHA got 2997 want 2997
|
||||
OK pure+shape+slope.ratio.derived3.credit APS got 3755 want 3755
|
||||
OK pure+shape+slope.ratio.derived3.credit CHA got 3021 want 3021
|
||||
OK pure+dissim+slope.ratio.derived3.credit APS got 3857 want 3857
|
||||
OK pure+dissim+slope.ratio.derived3.credit CHA got 3002 want 3002
|
||||
OK pure+shape+dissim+slope.ratio.derived3.credit APS got 4354 want 4354
|
||||
OK pure+shape+dissim+slope.ratio.derived3.credit CHA got 3037 want 3037
|
||||
OK defect.ratio.derived3.credit APS got 3545 want 3545
|
||||
OK defect.ratio.derived3.credit CHA got 2949 want 2949
|
||||
OK best constant-free configuration got 'defect.ratio.derived3.credit' want 'defect.ratio.derived3.credit'
|
||||
OK its total over both sets got 6494 want 6494
|
||||
|
||||
every number stated in the paper matches the sweep data.
|
||||
Reference in New Issue
Block a user