444 lines
40 KiB
Plaintext
444 lines
40 KiB
Plaintext
OK number of configurations got 264 want 264
|
|
|
|
The proposed method (projection, multiplier declared):
|
|
OK APS place got 3 want 3
|
|
OK APS total got 2806 want 2806
|
|
OK APS solved got 153 want 153
|
|
OK CHA place got 45 want 45
|
|
OK CHA total got 3220 want 3220
|
|
OK CHA solved got 40 want 40
|
|
OK both sets got 6026 want 6026
|
|
|
|
Head to head of the proposed method:
|
|
OK APS vs chandrupatla got (87, 24, 42) want (87, 24, 42)
|
|
OK APS vs toms748 got (86, 16, 51) want (86, 16, 51)
|
|
OK APS vs brent got (102, 11, 40) want (102, 11, 40)
|
|
OK APS vs itp got (132, 3, 18) want (132, 3, 18)
|
|
OK APS vs itp_oracle got (121, 4, 28) want (121, 4, 28)
|
|
OK APS vs bisection got (152, 1, 0) want (152, 1, 0)
|
|
OK CHA vs chandrupatla got (9, 5, 26) want (9, 5, 26)
|
|
OK CHA vs toms748 got (31, 1, 8) want (31, 1, 8)
|
|
OK CHA vs brent got (30, 4, 4) want (30, 4, 4)
|
|
OK CHA vs itp got (17, 0, 23) want (17, 0, 23)
|
|
OK CHA vs itp_oracle got (16, 0, 24) want (16, 0, 24)
|
|
OK APS vs Chandrupatla sign test p < 1e-4 got True want True
|
|
OK CHA vs Chandrupatla sign test p (rounded) got 0.006 want 0.006
|
|
|
|
The constant-free variant (earned projection):
|
|
OK credit APS place got 29 want 29
|
|
OK credit APS total got 3568 want 3568
|
|
OK credit CHA place got 10 want 10
|
|
OK credit CHA total got 3020 want 3020
|
|
OK credit both sets got 6588 want 6588
|
|
OK credit costs 762 more on the canonical set got 762 want 762
|
|
OK credit APS vs chandrupatla got (79, 3, 71) want (79, 3, 71)
|
|
OK credit APS vs toms748 got (77, 9, 67) want (77, 9, 67)
|
|
OK credit APS vs brent got (78, 5, 70) want (78, 5, 70)
|
|
OK credit APS vs Chandrupatla sign test p (rounded) got 0.57 want 0.57
|
|
|
|
The guards that replace the step instead of projecting it:
|
|
OK APS unguarded got 2806 want 2806
|
|
OK CHA unguarded total got 4104 want 4104
|
|
OK CHA unguarded solved got 35 want 35
|
|
OK CHA unguarded bound violations got 10 want 10
|
|
OK APS bracket-guard got 7260 want 7260
|
|
OK APS both-guard got 7239 want 7239
|
|
|
|
Alefeld-Potra-Shi:
|
|
OK APS winner got 'chandrupatla' want 'chandrupatla'
|
|
OK APS chandrupatla place got 1 want 1
|
|
OK APS chandrupatla total got 2787 want 2787
|
|
OK APS toms748 place got 4 want 4
|
|
OK APS toms748 total got 2820 want 2820
|
|
OK APS brent place got 5 want 5
|
|
OK APS brent total got 2889 want 2889
|
|
OK APS dekker place got 12 want 12
|
|
OK APS dekker total got 3158 want 3158
|
|
OK APS itp_oracle place got 13 want 13
|
|
OK APS itp_oracle total got 3260 want 3260
|
|
OK APS ridders place got 20 want 20
|
|
OK APS ridders total got 3313 want 3313
|
|
OK APS itp place got 34 want 34
|
|
OK APS itp total got 3957 want 3957
|
|
OK APS illinois place got 73 want 73
|
|
OK APS illinois total got 7045 want 7045
|
|
OK APS bisection place got 79 want 79
|
|
OK APS bisection total got 7530 want 7530
|
|
OK APS regula_falsi place got 244 want 244
|
|
OK APS regula_falsi total got 10560 want 10560
|
|
OK APS regula falsi solved got 127 want 127
|
|
|
|
Chandrupatla:
|
|
OK CHA rank 1 label got 'pure.sqrt.bracket.project' want 'pure.sqrt.bracket.project'
|
|
OK CHA rank 1 solves one more than anyone else got 41 want 41
|
|
OK CHA chandrupatla place got 2 want 2
|
|
OK CHA chandrupatla total got 2441 want 2441
|
|
OK CHA itp_oracle place got 3 want 3
|
|
OK CHA itp_oracle total got 2647 want 2647
|
|
OK CHA itp place got 6 want 6
|
|
OK CHA itp total got 2896 want 2896
|
|
OK CHA bisection place got 16 want 16
|
|
OK CHA bisection total got 3090 want 3090
|
|
OK CHA toms748 place got 159 want 159
|
|
OK CHA toms748 total got 4000 want 4000
|
|
OK CHA brent place got 227 want 227
|
|
OK CHA brent total got 4115 want 4115
|
|
OK CHA brent solved got 38 want 38
|
|
OK chandrupatla, both sets got 5228 want 5228
|
|
OK Chandrupatla beats us on both sets got True want True
|
|
|
|
Negative results, the regime claims:
|
|
OK CHA cbrt over the whole set got 4109 want 4109
|
|
OK single fixed choice, both sets got 6026 want 6026
|
|
OK switch by benchmark set, both sets got 6915 want 6915
|
|
OK switching is worse than not switching got True want True
|
|
OK fun4.1 projection variant got 89 want 89
|
|
OK fun4.1 cbrt got 32 want 32
|
|
OK fun4.2 projection variant got 93 want 93
|
|
OK fun4.2 cbrt got 32 want 32
|
|
OK fun4.3 projection variant got 113 want 113
|
|
OK fun4.3 cbrt got 40 want 40
|
|
OK fun4.4 projection variant got 125 want 125
|
|
OK fun4.4 cbrt got 47 want 47
|
|
OK fun4.5 projection variant got 153 want 153
|
|
OK fun4.5 cbrt got 59 want 59
|
|
|
|
The law across the grid:
|
|
OK mean total, scale=derived3, lin got 6893 want 6893
|
|
OK mean total, scale=derived3, quad got 22315 want 22315
|
|
OK mean total, scale=derived, lin got 6981 want 6981
|
|
OK mean total, scale=derived, quad got 22206 want 22206
|
|
OK mean total, scale=bracket, lin got 10279 want 10279
|
|
OK mean total, scale=bracket, quad got 19127 want 19127
|
|
OK mean total, scale=start, lin got 16102 want 16102
|
|
OK mean total, scale=start, quad got 15345 want 15345
|
|
OK every c=1 similarity beats every c=0 similarity got True want True
|
|
OK similarity ranking got ['ratio', 'exp_abs', 'rational', 'ratio_soft', 'sqrt', 'cbrt', 'exp_sq', 'rat_sq', 'cosine'] want ['ratio', 'exp_abs', 'rational', 'ratio_soft', 'sqrt', 'cbrt', 'exp_sq', 'rat_sq', 'cosine']
|
|
OK similarity ratio best got 6560 want 6560
|
|
OK similarity exp_abs best got 6895 want 6895
|
|
OK similarity rational best got 7118 want 7118
|
|
OK similarity ratio_soft best got 9578 want 9578
|
|
OK similarity sqrt best got 10546 want 10546
|
|
OK similarity cbrt best got 12002 want 12002
|
|
OK similarity exp_sq best got 14307 want 14307
|
|
OK similarity rat_sq best got 15102 want 15102
|
|
OK similarity cosine best got 16394 want 16394
|
|
OK scale ranking got ['derived3', 'derived', 'bracket', 'start'] want ['derived3', 'derived', 'bracket', 'start']
|
|
OK scale derived3 best got 6560 want 6560
|
|
OK scale derived best got 6702 want 6702
|
|
OK scale bracket best got 9107 want 9107
|
|
OK scale start best got 14540 want 14540
|
|
|
|
The placement modes (each in its own sweep file):
|
|
OK slope.ratio.derived3.project APS got 2848 want 2848
|
|
OK slope.ratio.derived3.project CHA got 3114 want 3114
|
|
OK slope.ratio.derived3.project solved got 193 want 193
|
|
OK slope.ratio.derived3.project guarantee (over 2x) got 0 want 0
|
|
OK pure+shape+slope.ratio.derived3.safe APS got 2527 want 2527
|
|
OK pure+shape+slope.ratio.derived3.safe CHA got 3126 want 3126
|
|
OK pure+shape+slope.ratio.derived3.safe solved got 193 want 193
|
|
OK pure+shape+slope.ratio.derived3.safe guarantee (over 2x) got 0 want 0
|
|
OK pure+dissim+slope.ratio.derived3.project APS got 2726 want 2726
|
|
OK pure+dissim+slope.ratio.derived3.project CHA got 2799 want 2799
|
|
OK pure+dissim+slope.ratio.derived3.project solved got 193 want 193
|
|
OK pure+dissim+slope.ratio.derived3.project guarantee (over 2x) got 0 want 0
|
|
OK pure+shape+dissim+slope.ratio.derived3.safe APS got 2884 want 2884
|
|
OK pure+shape+dissim+slope.ratio.derived3.safe CHA got 2871 want 2871
|
|
OK pure+shape+dissim+slope.ratio.derived3.safe solved got 193 want 193
|
|
OK pure+shape+dissim+slope.ratio.derived3.safe guarantee (over 2x) got 0 want 0
|
|
OK progress.ratio.derived.project APS got 4449 want 4449
|
|
OK progress.ratio.derived.project CHA got 3047 want 3047
|
|
OK progress.ratio.derived.project solved got 193 want 193
|
|
OK progress.ratio.derived.project guarantee (over 2x) got 0 want 0
|
|
OK log.ratio_soft.derived.safe APS got 6940 want 6940
|
|
OK log.ratio_soft.derived.safe CHA got 3541 want 3541
|
|
OK log.ratio_soft.derived.safe solved got 193 want 193
|
|
OK log.ratio_soft.derived.safe guarantee (over 2x) got 0 want 0
|
|
OK pure+dissim+slope.ratio.derived3.safe CHA per-set got 2695 want 2695
|
|
OK progress.ratio_soft.derived.safe CHA per-set got 2888 want 2888
|
|
|
|
The per-place reading (sweep_order.json):
|
|
OK far+rational both APS m=1 got 4522 want 4522
|
|
OK far+rational both APS order got 3675 want 3675
|
|
OK far+rational both CHA m=1 got 3406 want 3406
|
|
OK far+rational both CHA order got 3241 want 3241
|
|
OK far+rational bracket APS m=1 got 4606 want 4606
|
|
OK far+rational bracket APS order got 3542 want 3542
|
|
OK far+rational bracket CHA m=1 got 3602 want 3602
|
|
OK far+rational bracket CHA order got 3359 want 3359
|
|
OK far+rational credit APS m=1 got 5190 want 5190
|
|
OK far+rational credit APS order got 4686 want 4686
|
|
OK far+rational credit CHA m=1 got 3013 want 3013
|
|
OK far+rational credit CHA order got 2884 want 2884
|
|
OK far+rational project APS m=1 got 6932 want 6932
|
|
OK far+rational project APS order got 6364 want 6364
|
|
OK far+rational project CHA m=1 got 4789 want 4789
|
|
OK far+rational project CHA order got 4554 want 4554
|
|
OK far+rational safe APS m=1 got 5987 want 5987
|
|
OK far+rational safe APS order got 5229 want 5229
|
|
OK far+rational safe CHA m=1 got 4134 want 4134
|
|
OK far+rational safe CHA order got 3954 want 3954
|
|
OK far+rational step APS m=1 got 6435 want 6435
|
|
OK far+rational step APS order got 5551 want 5551
|
|
OK far+rational step CHA m=1 got 4287 want 4287
|
|
OK far+rational step CHA order got 4068 want 4068
|
|
OK far+rational none APS m=1 got 10676 want 10676
|
|
OK far+rational none APS order got 10043 want 10043
|
|
OK far+rational none CHA m=1 got 8265 want 8265
|
|
OK far+rational none CHA order got 7794 want 7794
|
|
OK cheaper in 7 of 7 safeguards got 7 want 7
|
|
OK false position none solved got 127 want 127
|
|
OK false position none APS got 10676 want 10676
|
|
OK false position order solved got 128 want 128
|
|
OK false position order APS got 10043 want 10043
|
|
OK slope reading none APS got 2806 want 2806
|
|
OK slope reading order APS got 2854 want 2854
|
|
OK guarantee family violations got 0 want 0
|
|
|
|
The decision place (sweep_decision.json):
|
|
OK ratio.band.project APS got 5432 want 5432
|
|
OK ratio.band.project CHA got 3059 want 3059
|
|
OK ratio.band.credit CHA got 2892 want 2892
|
|
OK ratio.none.project APS got 2806 want 2806
|
|
OK ratio.defect.project APS got 2523 want 2523
|
|
OK ratio.defect.project CHA got 3210 want 3210
|
|
OK ratio_soft.simpair.project CHA got 2897 want 2897
|
|
OK defect APS solved got 153 want 153
|
|
OK defect both sets got 5733 want 5733
|
|
OK defect is the new canonical best (< 2527) got True want True
|
|
OK decision grid guarantee violations got 0 want 0
|
|
OK band refusal rate % got 63.5 want 63.5
|
|
OK defect refusal rate % got 6.6 want 6.6
|
|
|
|
The gap decomposition (fresh runs of the diagnosis scripts):
|
|
OK problems where we lose got 33 want 33
|
|
OK fun1.4 our bracket halvings/eval got 0.62 want 0.62
|
|
OK fun1.4 IQI from our points got 10.37 want 10.37
|
|
OK flat: ours min got 0.41 want 0.41
|
|
OK flat: ours max got 0.75 want 0.75
|
|
OK flat: IQI min got 0.01 want 0.01
|
|
OK flat: IQI max got 0.17 want 0.17
|
|
OK non-degenerate states with one negative weight got np.int64(428) want 428
|
|
OK bracketing states in total got 431 want 431
|
|
OK every exception is an underflowed zero weight got True want True
|
|
OK total gap on the losing problems got 256 want 256
|
|
OK flat families' gap got 172 want 172
|
|
OK fun1 gap got 58 want 58
|
|
OK Chandrupatla bisect share fun3 % got 92 want 92
|
|
OK Chandrupatla bisect share fun4 % got 96 want 96
|
|
OK Chandrupatla bisect share fun5 % got 100 want 100
|
|
OK Chandrupatla bisect share fun6 % got 100 want 100
|
|
OK Chandrupatla total fun4 got 297 want 297
|
|
OK Chandrupatla total fun5 got 233 want 233
|
|
OK Chandrupatla total fun6 got 233 want 233
|
|
OK m>=2 on canonical states % got np.float64(0.2) want 0.2
|
|
OK m>=2 on flat states % got 0.0 want 0.0
|
|
OK band fun4 no-gate got 573 want 573
|
|
OK band fun4 with band got 296 want 296
|
|
OK band fun5 no-gate got 456 want 456
|
|
OK band fun5 with band got 239 want 239
|
|
OK band fun6 no-gate got 456 want 456
|
|
OK band fun6 with band got 228 want 228
|
|
OK band cost on fun1 got 164 want 164
|
|
OK band cost on fun2 got 111 want 111
|
|
OK band cost on fun8 got 170 want 170
|
|
OK band cost on fun9 got 144 want 144
|
|
OK log item: x-space control APS got 5822 want 5822
|
|
OK log item: x-space control CHA got 3695 want 3695
|
|
|
|
§3.1: the multiplier and the endowment (fresh runs):
|
|
OK multiplier 1.0 APS got 4817 want 4817
|
|
OK multiplier 1.0 CHA got 2997 want 2997
|
|
OK multiplier 1.25 APS got 2806 want 2806
|
|
OK multiplier 1.25 CHA got 2752 want 2752
|
|
OK multiplier 1.5 APS got 2806 want 2806
|
|
OK multiplier 1.5 CHA got 2889 want 2889
|
|
OK multiplier 2.0 APS got 2806 want 2806
|
|
OK multiplier 2.0 CHA got 3220 want 3220
|
|
OK multiplier 3.0 APS got 2806 want 2806
|
|
OK multiplier 3.0 CHA got 3756 want 3756
|
|
OK multiplier 4.0 APS got 2806 want 2806
|
|
OK multiplier 4.0 CHA got 4006 want 4006
|
|
OK endowment 0.1 APS got 5344 want 5344
|
|
OK endowment 0.1 CHA got 3051 want 3051
|
|
OK endowment 0.5 APS got 4656 want 4656
|
|
OK endowment 0.5 CHA got 3002 want 3002
|
|
OK endowment 1.0 APS got 3568 want 3568
|
|
OK endowment 1.0 CHA got 3020 want 3020
|
|
OK endowment 2.0 APS got 3692 want 3692
|
|
OK endowment 2.0 CHA got 2712 want 2712
|
|
OK endowment 5.0 APS got 3006 want 3006
|
|
OK endowment 5.0 CHA got 2615 want 2615
|
|
OK C0=1 cheapest within the one-step promise got 1.0 want 1.0
|
|
OK 2527 at multiplier 1.25 got 2541 want 2541
|
|
OK 2527 flat from 1.5 to 4 got {2527} want {2527}
|
|
OK 2523 flat from 1.25 to 4 got {2523} want {2523}
|
|
|
|
Tables 2-3 regenerated:
|
|
OK APS pure.ratio.derived3.project place got 3 want 3
|
|
OK APS pure.ratio.derived3.none place got 2 want 2
|
|
OK CHA pure.ratio.derived3.project place got 45 want 45
|
|
OK CHA pure.ratio.derived3.none place got 228 want 228
|
|
OK CHA pure.exp_abs.bracket.both place got 4 want 4
|
|
OK CHA exp_abs.bracket.both total got 2736 want 2736
|
|
OK CHA TOMS 748 over the 2x bound got 15 want 15
|
|
OK CHA Brent over the 2x bound got 18 want 18
|
|
OK credit APS vs itp_oracle got (86, 6, 61) want (86, 6, 61)
|
|
OK credit CHA vs itp_oracle got (2, 2, 36) want (2, 2, 36)
|
|
OK credit CHA vs bisection got (5, 0, 35) want (5, 0, 35)
|
|
OK project APS vs chandrupatla got (87, 24, 42) want (87, 24, 42)
|
|
OK project APS vs itp got (132, 3, 18) want (132, 3, 18)
|
|
OK project APS vs itp_oracle got (121, 4, 28) want (121, 4, 28)
|
|
OK project APS vs bisection got (152, 1, 0) want (152, 1, 0)
|
|
OK project CHA vs toms748 got (31, 1, 8) want (31, 1, 8)
|
|
OK project CHA vs brent got (30, 4, 4) want (30, 4, 4)
|
|
OK project CHA vs itp got (17, 0, 23) want (17, 0, 23)
|
|
OK project CHA vs itp_oracle got (16, 0, 24) want (16, 0, 24)
|
|
OK both sets pure.ratio.derived3.credit got 6588 want 6588
|
|
OK both sets pure.ratio.derived3.project got 6026 want 6026
|
|
OK both sets chandrupatla got 5228 want 5228
|
|
OK both sets itp_oracle got 5907 want 5907
|
|
OK both sets toms748 got 6820 want 6820
|
|
OK both sets brent got 7004 want 7004
|
|
|
|
§7 under the earned projection:
|
|
OK slope.ratio.derived3.credit APS got 3507 want 3507
|
|
OK slope.ratio.derived3.credit CHA got 2997 want 2997
|
|
OK pure+shape+slope.ratio.derived3.credit APS got 3755 want 3755
|
|
OK pure+shape+slope.ratio.derived3.credit CHA got 3021 want 3021
|
|
OK pure+dissim+slope.ratio.derived3.credit APS got 3857 want 3857
|
|
OK pure+dissim+slope.ratio.derived3.credit CHA got 3002 want 3002
|
|
OK pure+shape+dissim+slope.ratio.derived3.credit APS got 4354 want 4354
|
|
OK pure+shape+dissim+slope.ratio.derived3.credit CHA got 3037 want 3037
|
|
OK defect.ratio.derived3.credit APS got 3545 want 3545
|
|
OK defect.ratio.derived3.credit CHA got 2949 want 2949
|
|
OK best constant-free configuration got 'defect+flatanalogmempt+exp+half.ratio.derived3.credit' want 'defect+flatanalogmempt+exp+half.ratio.derived3.credit'
|
|
OK its total over both sets got 6111 want 6111
|
|
OK defect under the earned projection, both sets got 6494 want 6494
|
|
|
|
Section 8 and negative items 12-16 (the development grids):
|
|
OK flat.ratio.derived3.project APS got 3029 want 3029
|
|
OK flat.ratio.derived3.project CHA got 3198 want 3198
|
|
OK flat.ratio.derived3.project guarantee got 0 want 0
|
|
OK flat2.ratio.derived3.project APS got 2803 want 2803
|
|
OK flat2.ratio.derived3.project CHA got 3341 want 3341
|
|
OK flat2.ratio.derived3.project guarantee got 0 want 0
|
|
OK flatcreep.ratio.derived3.project APS got 2878 want 2878
|
|
OK flatcreep.ratio.derived3.project CHA got 3354 want 3354
|
|
OK flatcreep.ratio.derived3.project guarantee got 0 want 0
|
|
OK flatstickmid.ratio.derived3.project APS got 6897 want 6897
|
|
OK flatstickmid.ratio.derived3.project CHA got 3019 want 3019
|
|
OK flatstickmid.ratio.derived3.project guarantee got 0 want 0
|
|
OK flatstickexit.ratio.derived3.project APS got 3125 want 3125
|
|
OK flatstickexit.ratio.derived3.project CHA got 2503 want 2503
|
|
OK flatstickexit.ratio.derived3.project guarantee got 0 want 0
|
|
OK flatjudge.ratio.derived3.project APS got 3009 want 3009
|
|
OK flatjudge.ratio.derived3.project CHA got 2486 want 2486
|
|
OK flatjudge.ratio.derived3.project guarantee got 0 want 0
|
|
OK flatanalogmem.ratio.derived3.project APS got 2814 want 2814
|
|
OK flatanalogmem.ratio.derived3.project guarantee got 0 want 0
|
|
OK half.ratio.derived3.project APS got 2564 want 2564
|
|
OK half.ratio.derived3.project guarantee got 0 want 0
|
|
OK sticky with order from the failed promise, APS got 6781 want 6781
|
|
OK sticky midpoint: flat families got 1057 want 1057
|
|
OK bisection: flat families got 1044 want 1044
|
|
OK Chandrupatla: flat families got 1068 want 1068
|
|
OK split from the order: flat families, low end got 743 want 743
|
|
OK split from the order: flat families, high end got 847 want 847
|
|
OK item 16: remembered + split, with the step pattern, CHA got 2117 want 2117
|
|
OK item 16: one-state analogy + split, with the step pattern, CHA got 2221 want 2221
|
|
|
|
Section 8 diagnostics (fresh runs):
|
|
OK flag rises on canonical problems got 142 want 142
|
|
OK flag + analogy + m>=2 on canonical problems got 3 want 3
|
|
OK flag + analogy + m>=2 on flat problems got 20 want 20
|
|
OK flag + analogy + m>=2 on fun2 got 0 want 0
|
|
OK flag + analogy + m>=2 on fun8 got 0 want 0
|
|
OK flag + analogy + m>=2 on fun9 got 0 want 0
|
|
OK mean entry state, one state got '12.9' want '12.9'
|
|
OK mean entry state, remembered got '8.2' want '8.2'
|
|
OK order from the failed promise is inf got (163, 174) want (163, 174)
|
|
OK sqrt|x| sgn x, base rule got 84 want 84
|
|
OK sin on [3,4], base rule got 24 want 24
|
|
OK sin on [3,4], A' got 7 want 7
|
|
|
|
Table 8 (A' on the protocol of section 4, sweep_aprime.json):
|
|
OK A' APS got 2438 want 2438
|
|
OK A' CHA got 2005 want 2005
|
|
OK A' both got 4443 want 4443
|
|
OK A' solved got 193 want 193
|
|
OK A' earned budget got (3608, 2807) want (3608, 2807)
|
|
OK A' under every safeguard: violations got 0 want 0
|
|
OK A' safeguards measured got 7 want 7
|
|
OK A' APS vs chandrupatla got (103, 20, 30) want (103, 20, 30)
|
|
OK A' CHA vs chandrupatla got (29, 4, 7) want (29, 4, 7)
|
|
OK A' APS vs toms748 got (127, 6, 20) want (127, 6, 20)
|
|
OK A' CHA vs toms748 got (38, 2, 0) want (38, 2, 0)
|
|
OK A' APS vs brent got (118, 17, 18) want (118, 17, 18)
|
|
OK A' CHA vs brent got (33, 3, 2) want (33, 3, 2)
|
|
OK A' APS vs itp_oracle got (126, 7, 20) want (126, 7, 20)
|
|
OK A' CHA vs itp_oracle got (35, 1, 4) want (35, 1, 4)
|
|
OK A' APS vs Chandrupatla p (2 digits) got 1.5e-10 want 1.5e-10
|
|
OK A' CHA vs Chandrupatla p (1 digit) got 0.0003 want 0.0003
|
|
OK A' below Chandrupatla, APS % got 12.5 want 12.5
|
|
OK A' below Chandrupatla, CHA % got 17.9 want 17.9
|
|
OK A' fun9 vs Chandrupatla got (65, 55) want (65, 55)
|
|
OK A' ahead on wide-bracket families got 7 want 7
|
|
OK A' level on fun7 got True want True
|
|
|
|
Table 9 (ablation of A' on six corpora, sweep_ablation_all.json):
|
|
OK ablation -regime got (-7, 1013, 1660, 324, 77, 2269) want (-7, 1013, 1660, 324, 77, 2269)
|
|
OK ablation -point got (-5, 479, 667, 272, 30, 1672) want (-5, 479, 667, 272, 30, 1672)
|
|
OK ablation -sub got (-6, 0, -7, 16, 63, 893) want (-6, 0, -7, 16, 63, 893)
|
|
OK ablation -close got (130, 35, 97, 78, 135, -2) want (130, 35, 97, 78, 135, -2)
|
|
OK ablation -half got (249, 65, 82, -16, -5, 1) want (249, 65, 82, -16, -5, 1)
|
|
OK ablation -exp got (31, 143, 100, -21, -2, 87) want (31, 143, 100, -21, -2, 87)
|
|
OK ablation -G2 got (-22, -6, 123, -30, -10, 34) want (-22, -6, 123, -30, -10, 34)
|
|
OK ablation -contra got (11, -12, 29, 0, 1, -1) want (11, -12, 29, 0, 1, -1)
|
|
|
|
Section 9 (out-of-sample corpora):
|
|
OK corpus APS size got 154 want 154
|
|
OK corpus CHA size got 45 want 45
|
|
OK corpus NEW size got 206 want 206
|
|
OK corpus RJL size got 78 want 78
|
|
OK corpus APP size got 132 want 132
|
|
OK corpus D size got 240 want 240
|
|
OK Table 10 APS totals got (2259, 2595, 2707, 2636) want (2259, 2595, 2707, 2636)
|
|
OK Table 10 APS vs Chandrupatla got (103, 20, 31) want (103, 20, 31)
|
|
OK Table 10 APS A' violations got 0 want 0
|
|
OK Table 10 CHA totals got (1074, 1505, 3238, 3088) want (1074, 1505, 3238, 3088)
|
|
OK Table 10 CHA vs Chandrupatla got (31, 4, 10) want (31, 4, 10)
|
|
OK Table 10 CHA A' violations got 0 want 0
|
|
OK Table 10 NEW totals got (3290, 3716, 6395, 5746) want (3290, 3716, 6395, 5746)
|
|
OK Table 10 NEW vs Chandrupatla got (108, 33, 65) want (108, 33, 65)
|
|
OK Table 10 NEW A' violations got 0 want 0
|
|
OK Table 10 RJL totals got (1371, 1566, 2455, 2339) want (1371, 1566, 2455, 2339)
|
|
OK Table 10 RJL vs Chandrupatla got (44, 14, 20) want (44, 14, 20)
|
|
OK Table 10 RJL A' violations got 0 want 0
|
|
OK Table 10 APP totals got (1382, 1382, 1503, 1756) want (1382, 1382, 1503, 1756)
|
|
OK Table 10 APP vs Chandrupatla got (48, 52, 32) want (48, 52, 32)
|
|
OK Table 10 APP A' violations got 0 want 0
|
|
OK Table 10 D totals got (3656, 5717, 8267, 8084) want (3656, 5717, 8267, 8084)
|
|
OK Table 10 D vs Chandrupatla got (177, 35, 28) want (177, 35, 28)
|
|
OK Table 10 D A' violations got 0 want 0
|
|
OK Brent / TOMS 748 bound violations over 855 got (118, 84) want (118, 84)
|
|
OK all 855 totals got (13032, 16481, 24565, 23649) want (13032, 16481, 24565, 23649)
|
|
OK corpus 4 without targeted got (2403, 3569, (107, 31, 22)) want (2403, 3569, (107, 31, 22))
|
|
OK corpus 4 simple roots got (737, 747, (43, 22, 15)) want (737, 747, (43, 22, 15))
|
|
OK Table 11 D1 got (433, 419, (20, 8, 12)) want (433, 419, (20, 8, 12))
|
|
OK Table 11 D2 got (1163, 1805, (39, 1, 0)) want (1163, 1805, (39, 1, 0))
|
|
OK Table 11 D3 got (924, 1784, (37, 1, 2)) want (924, 1784, (37, 1, 2))
|
|
OK Table 11 D4 got (304, 328, (23, 14, 3)) want (304, 328, (23, 14, 3))
|
|
OK Table 11 D5 got (503, 1017, (25, 8, 7)) want (503, 1017, (25, 8, 7))
|
|
OK Table 11 D6 got (329, 364, (33, 3, 4)) want (329, 364, (33, 3, 4))
|
|
OK near-cancelling class of corpus 1 got (491, 391) want (491, 391)
|
|
OK corpus 1, variant carried: total vs Chandrupatla got (3492, 3899) want (3492, 3899)
|
|
OK corpus 1, variant carried: per problem got (105, 34, 67) want (105, 34, 67)
|
|
OK corpus 1, with defect: per problem got (95, 33, 78) want (95, 33, 78)
|
|
OK corpus 1, with defect: exponential class got (9, 5, 26) want (9, 5, 26)
|
|
OK Roots.jl, variant carried: totals got (1464, 1566) want (1464, 1566)
|
|
OK Roots.jl, variant carried: per problem got (38, 10, 30) want (38, 10, 30)
|
|
OK applied, variant carried: totals got (1580, 1382) want (1580, 1382)
|
|
|
|
every number stated in the paper matches the sweep data.
|