vvs/sim-numerics
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Article (canonical .md), step 2: abstract, introduction, a note in section 3, the close of 7.4, negative results 12-16, conclusion, future work and references 8-12 rewritten for the proposed method A'; two statements that could not be recomputed rephrased. verify_article: expected values updated to the regenerated sweeps, the gate cells renamed (progress.*; their CHA values had moved unnoticed, 2991 -> 3047, 2851 -> 2888, not quoted in the paper), and verify_flat_regime_and_oos() added - Sections 8-9, Tables 8-11, items 12-16 and the abstract, recomputed from server files and fresh diagnosis runs. Verifier on the server: 401 OK, 0 BAD, 'every number stated in the paper matches the sweep data'. Server regression x2: 1226 passed, 0 failed.

Browse Source 
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
vvs
2026-09-21 20:56:58 +01:00
co-authored by Claude Opus 5
parent 8bf595d9be
commit 16b9ab87f1
6 changed files with 772 additions and 172 deletions
Download Patch File Download Diff File Expand all files Collapse all files
results_verify_article.txt
+188 -118
View File
@@ -4,7 +4,7 @@ The proposed method (projection, multiplier declared):
OK APS place got 3 want 3
OK APS total got 2806 want 2806
OK APS solved got 153 want 153
BAD CHA place got 45 want 47
OK CHA place got 45 want 45
OK CHA total got 3220 want 3220
OK CHA solved got 40 want 40
OK both sets got 6026 want 6026
@@ -74,23 +74,23 @@ Chandrupatla:
OK CHA chandrupatla place got 2 want 2
OK CHA chandrupatla total got 2441 want 2441
OK CHA itp_oracle place got 3 want 3
BAD CHA itp_oracle total got 2647 want 2611
OK CHA itp_oracle total got 2647 want 2647
OK CHA itp place got 6 want 6
BAD CHA itp total got 2896 want 2857
OK CHA itp total got 2896 want 2896
OK CHA bisection place got 16 want 16
OK CHA bisection total got 3090 want 3090
BAD CHA toms748 place got 159 want 164
OK CHA toms748 place got 159 want 159
OK CHA toms748 total got 4000 want 4000
BAD CHA brent place got 227 want 226
OK CHA brent place got 227 want 227
OK CHA brent total got 4115 want 4115
OK CHA brent solved got 38 want 38
OK chandrupatla, both sets got 5228 want 5228
OK Chandrupatla beats us on both sets got True want True
Negative results, the regime claims:
BAD CHA cbrt over the whole set got 4109 want 4099
OK CHA cbrt over the whole set got 4109 want 4109
OK single fixed choice, both sets got 6026 want 6026
BAD switch by benchmark set, both sets got 6915 want 6905
OK switch by benchmark set, both sets got 6915 want 6915
OK switching is worse than not switching got True want True
OK fun4.1 projection variant got 89 want 89
OK fun4.1 cbrt got 32 want 32
@@ -104,80 +104,84 @@ Negative results, the regime claims:
OK fun4.5 cbrt got 59 want 59
The law across the grid:
BAD mean total, scale=derived3, lin got 6893 want 6809
BAD mean total, scale=derived3, quad got 22315 want 22294
BAD mean total, scale=derived, lin got 6981 want 6906
BAD mean total, scale=derived, quad got 22206 want 22167
BAD mean total, scale=bracket, lin got 10279 want 10263
BAD mean total, scale=bracket, quad got 19127 want 19137
OK mean total, scale=derived3, lin got 6893 want 6893
OK mean total, scale=derived3, quad got 22315 want 22315
OK mean total, scale=derived, lin got 6981 want 6981
OK mean total, scale=derived, quad got 22206 want 22206
OK mean total, scale=bracket, lin got 10279 want 10279
OK mean total, scale=bracket, quad got 19127 want 19127
OK mean total, scale=start, lin got 16102 want 16102
OK mean total, scale=start, quad got 15345 want 15345
OK every c=1 similarity beats every c=0 similarity got True want True
OK similarity ranking got ['ratio', 'exp_abs', 'rational', 'ratio_soft', 'sqrt', 'cbrt', 'exp_sq', 'rat_sq', 'cosine'] want ['ratio', 'exp_abs', 'rational', 'ratio_soft', 'sqrt', 'cbrt', 'exp_sq', 'rat_sq', 'cosine']
BAD similarity ratio best got 6560 want 6496
BAD similarity exp_abs best got 6895 want 6822
BAD similarity rational best got 7118 want 7038
BAD similarity ratio_soft best got 9578 want 9502
BAD similarity sqrt best got 10546 want 10497
BAD similarity cbrt best got 12002 want 11941
BAD similarity exp_sq best got 14307 want 14314
OK similarity ratio best got 6560 want 6560
OK similarity exp_abs best got 6895 want 6895
OK similarity rational best got 7118 want 7118
OK similarity ratio_soft best got 9578 want 9578
OK similarity sqrt best got 10546 want 10546
OK similarity cbrt best got 12002 want 12002
OK similarity exp_sq best got 14307 want 14307
OK similarity rat_sq best got 15102 want 15102
OK similarity cosine best got 16394 want 16394
OK scale ranking got ['derived3', 'derived', 'bracket', 'start'] want ['derived3', 'derived', 'bracket', 'start']
BAD scale derived3 best got 6560 want 6496
BAD scale derived best got 6702 want 6628
BAD scale bracket best got 9107 want 9081
OK scale derived3 best got 6560 want 6560
OK scale derived best got 6702 want 6702
OK scale bracket best got 9107 want 9107
OK scale start best got 14540 want 14540
The placement modes (each in its own sweep file):
OK slope.ratio.derived3.project APS got 2848 want 2848
BAD slope.ratio.derived3.project CHA got 3114 want 3074
OK slope.ratio.derived3.project CHA got 3114 want 3114
OK slope.ratio.derived3.project solved got 193 want 193
OK slope.ratio.derived3.project guarantee (over 2x) got 0 want 0
OK pure+shape+slope.ratio.derived3.safe APS got 2527 want 2527
BAD pure+shape+slope.ratio.derived3.safe CHA got 3126 want 3108
OK pure+shape+slope.ratio.derived3.safe CHA got 3126 want 3126
OK pure+shape+slope.ratio.derived3.safe solved got 193 want 193
OK pure+shape+slope.ratio.derived3.safe guarantee (over 2x) got 0 want 0
OK pure+dissim+slope.ratio.derived3.project APS got 2726 want 2726
BAD pure+dissim+slope.ratio.derived3.project CHA got 2799 want 2785
OK pure+dissim+slope.ratio.derived3.project CHA got 2799 want 2799
OK pure+dissim+slope.ratio.derived3.project solved got 193 want 193
OK pure+dissim+slope.ratio.derived3.project guarantee (over 2x) got 0 want 0
OK pure+shape+dissim+slope.ratio.derived3.safe APS got 2884 want 2884
BAD pure+shape+dissim+slope.ratio.derived3.safe CHA got 2871 want 2859
OK pure+shape+dissim+slope.ratio.derived3.safe CHA got 2871 want 2871
OK pure+shape+dissim+slope.ratio.derived3.safe solved got 193 want 193
OK pure+shape+dissim+slope.ratio.derived3.safe guarantee (over 2x) got 0 want 0
!! gate.ratio.derived.project not in sweep_gate.json — stale sweep
OK progress.ratio.derived.project APS got 4449 want 4449
OK progress.ratio.derived.project CHA got 3047 want 3047
OK progress.ratio.derived.project solved got 193 want 193
OK progress.ratio.derived.project guarantee (over 2x) got 0 want 0
OK log.ratio_soft.derived.safe APS got 6940 want 6940
BAD log.ratio_soft.derived.safe CHA got 3541 want 3474
OK log.ratio_soft.derived.safe CHA got 3541 want 3541
OK log.ratio_soft.derived.safe solved got 193 want 193
OK log.ratio_soft.derived.safe guarantee (over 2x) got 0 want 0
BAD pure+dissim+slope.ratio.derived3.safe CHA per-set got 2695 want 2683
OK pure+dissim+slope.ratio.derived3.safe CHA per-set got 2695 want 2695
OK progress.ratio_soft.derived.safe CHA per-set got 2888 want 2888
The per-place reading (sweep_order.json):
OK far+rational both APS m=1 got 4522 want 4522
OK far+rational both APS order got 3675 want 3675
BAD far+rational both CHA m=1 got 3406 want 3387
BAD far+rational both CHA order got 3241 want 3216
OK far+rational both CHA m=1 got 3406 want 3406
OK far+rational both CHA order got 3241 want 3241
OK far+rational bracket APS m=1 got 4606 want 4606
OK far+rational bracket APS order got 3542 want 3542
BAD far+rational bracket CHA m=1 got 3602 want 3571
BAD far+rational bracket CHA order got 3359 want 3303
OK far+rational bracket CHA m=1 got 3602 want 3602
OK far+rational bracket CHA order got 3359 want 3359
OK far+rational credit APS m=1 got 5190 want 5190
OK far+rational credit APS order got 4686 want 4686
OK far+rational credit CHA m=1 got 3013 want 3013
OK far+rational credit CHA order got 2884 want 2884
OK far+rational project APS m=1 got 6932 want 6932
OK far+rational project APS order got 6364 want 6364
BAD far+rational project CHA m=1 got 4789 want 4794
BAD far+rational project CHA order got 4554 want 4559
OK far+rational project CHA m=1 got 4789 want 4789
OK far+rational project CHA order got 4554 want 4554
OK far+rational safe APS m=1 got 5987 want 5987
OK far+rational safe APS order got 5229 want 5229
BAD far+rational safe CHA m=1 got 4134 want 4108
BAD far+rational safe CHA order got 3954 want 3923
OK far+rational safe CHA m=1 got 4134 want 4134
OK far+rational safe CHA order got 3954 want 3954
OK far+rational step APS m=1 got 6435 want 6435
OK far+rational step APS order got 5551 want 5551
BAD far+rational step CHA m=1 got 4287 want 4257
BAD far+rational step CHA order got 4068 want 4022
OK far+rational step CHA m=1 got 4287 want 4287
OK far+rational step CHA order got 4068 want 4068
OK far+rational none APS m=1 got 10676 want 10676
OK far+rational none APS order got 10043 want 10043
OK far+rational none CHA m=1 got 8265 want 8265
@@ -193,32 +197,32 @@ The per-place reading (sweep_order.json):
The decision place (sweep_decision.json):
OK ratio.band.project APS got 5432 want 5432
BAD ratio.band.project CHA got 3059 want 3066
BAD ratio.band.credit CHA got 2892 want 2899
OK ratio.band.project CHA got 3059 want 3059
OK ratio.band.credit CHA got 2892 want 2892
OK ratio.none.project APS got 2806 want 2806
OK ratio.defect.project APS got 2523 want 2523
OK ratio.defect.project CHA got 3210 want 3210
BAD ratio_soft.simpair.project CHA got 2897 want 2868
OK ratio_soft.simpair.project CHA got 2897 want 2897
OK defect APS solved got 153 want 153
OK defect both sets got 5733 want 5733
OK defect is the new canonical best (< 2527) got True want True
OK decision grid guarantee violations got 0 want 0
BAD band refusal rate % got 63.5 want 63.6
BAD defect refusal rate % got 6.6 want 7.0
OK band refusal rate % got 63.5 want 63.5
OK defect refusal rate % got 6.6 want 6.6
The gap decomposition (fresh runs of the diagnosis scripts):
OK problems where we lose got 33 want 33
OK fun1.4 our bracket halvings/eval got 0.62 want 0.62
OK fun1.4 IQI from our points got 10.37 want 10.37
OK flat: ours min got 0.41 want 0.41
BAD flat: ours max got 0.75 want 0.81
OK flat: ours max got 0.75 want 0.75
OK flat: IQI min got 0.01 want 0.01
OK flat: IQI max got 0.17 want 0.17
OK non-degenerate states with one negative weight got np.int64(428) want 428
BAD bracketing states in total got 431 want 433
OK bracketing states in total got 431 want 431
OK every exception is an underflowed zero weight got True want True
BAD total gap on the losing problems got 256 want 244
BAD flat families' gap got 172 want 160
OK total gap on the losing problems got 256 want 256
OK flat families' gap got 172 want 172
OK fun1 gap got 58 want 58
OK Chandrupatla bisect share fun3 % got 92 want 92
OK Chandrupatla bisect share fun4 % got 96 want 96
@@ -228,19 +232,19 @@ The gap decomposition (fresh runs of the diagnosis scripts):
OK Chandrupatla total fun5 got 233 want 233
OK Chandrupatla total fun6 got 233 want 233
OK m>=2 on canonical states % got np.float64(0.2) want 0.2
BAD m>=2 on flat states % got 0.0 want 5.8
OK m>=2 on flat states % got 0.0 want 0.0
OK band fun4 no-gate got 573 want 573
OK band fun4 with band got 296 want 296
OK band fun5 no-gate got 456 want 456
OK band fun5 with band got 239 want 239
OK band fun6 no-gate got 456 want 456
BAD band fun6 with band got 228 want 235
OK band fun6 with band got 228 want 228
OK band cost on fun1 got 164 want 164
OK band cost on fun2 got 111 want 111
OK band cost on fun8 got 170 want 170
OK band cost on fun9 got 144 want 144
OK log item: x-space control APS got 5822 want 5822
BAD log item: x-space control CHA got 3695 want 3643
OK log item: x-space control CHA got 3695 want 3695
§3.1: the multiplier and the endowment (fresh runs):
OK multiplier 1.0 APS got 4817 want 4817
@@ -273,14 +277,14 @@ The gap decomposition (fresh runs of the diagnosis scripts):
Tables 2-3 regenerated:
OK APS pure.ratio.derived3.project place got 3 want 3
OK APS pure.ratio.derived3.none place got 2 want 2
BAD CHA pure.ratio.derived3.project place got 45 want 47
OK CHA pure.ratio.derived3.project place got 45 want 45
OK CHA pure.ratio.derived3.none place got 228 want 228
OK CHA pure.exp_abs.bracket.both place got 4 want 4
BAD CHA exp_abs.bracket.both total got 2736 want 2733
OK CHA exp_abs.bracket.both total got 2736 want 2736
OK CHA TOMS 748 over the 2x bound got 15 want 15
OK CHA Brent over the 2x bound got 18 want 18
OK credit APS vs itp_oracle got (86, 6, 61) want (86, 6, 61)
BAD credit CHA vs itp_oracle got (2, 2, 36) want (2, 3, 35)
OK credit CHA vs itp_oracle got (2, 2, 36) want (2, 2, 36)
OK credit CHA vs bisection got (5, 0, 35) want (5, 0, 35)
OK project APS vs chandrupatla got (87, 24, 42) want (87, 24, 42)
OK project APS vs itp got (132, 3, 18) want (132, 3, 18)
@@ -293,7 +297,7 @@ Tables 2-3 regenerated:
OK both sets pure.ratio.derived3.credit got 6588 want 6588
OK both sets pure.ratio.derived3.project got 6026 want 6026
OK both sets chandrupatla got 5228 want 5228
BAD both sets itp_oracle got 5907 want 5871
OK both sets itp_oracle got 5907 want 5907
OK both sets toms748 got 6820 want 6820
OK both sets brent got 7004 want 7004
@@ -308,66 +312,132 @@ Tables 2-3 regenerated:
OK pure+shape+dissim+slope.ratio.derived3.credit CHA got 3037 want 3037
OK defect.ratio.derived3.credit APS got 3545 want 3545
OK defect.ratio.derived3.credit CHA got 2949 want 2949
BAD best constant-free configuration got 'defect+flatanalogmempt+exp+half.ratio.derived3.credit' want 'defect.ratio.derived3.credit'
BAD its total over both sets got 6111 want 6494
OK best constant-free configuration got 'defect+flatanalogmempt+exp+half.ratio.derived3.credit' want 'defect+flatanalogmempt+exp+half.ratio.derived3.credit'
OK its total over both sets got 6111 want 6111
OK defect under the earned projection, both sets got 6494 want 6494
59 STATED NUMBERS DO NOT MATCH THE DATA:
CHA place
CHA itp_oracle total
CHA itp total
CHA toms748 place
CHA brent place
CHA cbrt over the whole set
switch by benchmark set, both sets
mean total, scale=derived3, lin
mean total, scale=derived3, quad
mean total, scale=derived, lin
mean total, scale=derived, quad
mean total, scale=bracket, lin
mean total, scale=bracket, quad
similarity ratio best
similarity exp_abs best
similarity rational best
similarity ratio_soft best
similarity sqrt best
similarity cbrt best
similarity exp_sq best
scale derived3 best
scale derived best
scale bracket best
slope.ratio.derived3.project CHA
pure+shape+slope.ratio.derived3.safe CHA
pure+dissim+slope.ratio.derived3.project CHA
pure+shape+dissim+slope.ratio.derived3.safe CHA
gate.ratio.derived.project missing from sweep_gate.json
log.ratio_soft.derived.safe CHA
pure+dissim+slope.ratio.derived3.safe CHA per-set
gate.ratio_soft.derived.safe missing from sweep_gate.json
far+rational both CHA m=1
far+rational both CHA order
far+rational bracket CHA m=1
far+rational bracket CHA order
far+rational project CHA m=1
far+rational project CHA order
far+rational safe CHA m=1
far+rational safe CHA order
far+rational step CHA m=1
far+rational step CHA order
ratio.band.project CHA
ratio.band.credit CHA
ratio_soft.simpair.project CHA
band refusal rate %
defect refusal rate %
flat: ours max
bracketing states in total
total gap on the losing problems
flat families' gap
m>=2 on flat states %
band fun6 with band
log item: x-space control CHA
CHA pure.ratio.derived3.project place
CHA exp_abs.bracket.both total
credit CHA vs itp_oracle
both sets itp_oracle
best constant-free configuration
its total over both sets
Section 8 and negative items 12-16 (the development grids):
OK flat.ratio.derived3.project APS got 3029 want 3029
OK flat.ratio.derived3.project CHA got 3198 want 3198
OK flat.ratio.derived3.project guarantee got 0 want 0
OK flat2.ratio.derived3.project APS got 2803 want 2803
OK flat2.ratio.derived3.project CHA got 3341 want 3341
OK flat2.ratio.derived3.project guarantee got 0 want 0
OK flatcreep.ratio.derived3.project APS got 2878 want 2878
OK flatcreep.ratio.derived3.project CHA got 3354 want 3354
OK flatcreep.ratio.derived3.project guarantee got 0 want 0
OK flatstickmid.ratio.derived3.project APS got 6897 want 6897
OK flatstickmid.ratio.derived3.project CHA got 3019 want 3019
OK flatstickmid.ratio.derived3.project guarantee got 0 want 0
OK flatstickexit.ratio.derived3.project APS got 3125 want 3125
OK flatstickexit.ratio.derived3.project CHA got 2503 want 2503
OK flatstickexit.ratio.derived3.project guarantee got 0 want 0
OK flatjudge.ratio.derived3.project APS got 3009 want 3009
OK flatjudge.ratio.derived3.project CHA got 2486 want 2486
OK flatjudge.ratio.derived3.project guarantee got 0 want 0
OK flatanalogmem.ratio.derived3.project APS got 2814 want 2814
OK flatanalogmem.ratio.derived3.project guarantee got 0 want 0
OK half.ratio.derived3.project APS got 2564 want 2564
OK half.ratio.derived3.project guarantee got 0 want 0
OK sticky with order from the failed promise, APS got 6781 want 6781
OK sticky midpoint: flat families got 1057 want 1057
OK bisection: flat families got 1044 want 1044
OK Chandrupatla: flat families got 1068 want 1068
OK split from the order: flat families, low end got 743 want 743
OK split from the order: flat families, high end got 847 want 847
OK item 16: remembered + split, with the step pattern, CHA got 2117 want 2117
OK item 16: one-state analogy + split, with the step pattern, CHA got 2221 want 2221
Section 8 diagnostics (fresh runs):
OK flag rises on canonical problems got 142 want 142
OK flag + analogy + m>=2 on canonical problems got 3 want 3
OK flag + analogy + m>=2 on flat problems got 20 want 20
OK flag + analogy + m>=2 on fun2 got 0 want 0
OK flag + analogy + m>=2 on fun8 got 0 want 0
OK flag + analogy + m>=2 on fun9 got 0 want 0
OK mean entry state, one state got '12.9' want '12.9'
OK mean entry state, remembered got '8.2' want '8.2'
OK order from the failed promise is inf got (163, 174) want (163, 174)
OK sqrt|x| sgn x, base rule got 84 want 84
OK sin on [3,4], base rule got 24 want 24
OK sin on [3,4], A' got 7 want 7
Table 8 (A' on the protocol of section 4, sweep_aprime.json):
OK A' APS got 2438 want 2438
OK A' CHA got 2005 want 2005
OK A' both got 4443 want 4443
OK A' solved got 193 want 193
OK A' earned budget got (3608, 2807) want (3608, 2807)
OK A' under every safeguard: violations got 0 want 0
OK A' safeguards measured got 7 want 7
OK A' APS vs chandrupatla got (103, 20, 30) want (103, 20, 30)
OK A' CHA vs chandrupatla got (29, 4, 7) want (29, 4, 7)
OK A' APS vs toms748 got (127, 6, 20) want (127, 6, 20)
OK A' CHA vs toms748 got (38, 2, 0) want (38, 2, 0)
OK A' APS vs brent got (118, 17, 18) want (118, 17, 18)
OK A' CHA vs brent got (33, 3, 2) want (33, 3, 2)
OK A' APS vs itp_oracle got (126, 7, 20) want (126, 7, 20)
OK A' CHA vs itp_oracle got (35, 1, 4) want (35, 1, 4)
OK A' APS vs Chandrupatla p (2 digits) got 1.5e-10 want 1.5e-10
OK A' CHA vs Chandrupatla p (1 digit) got 0.0003 want 0.0003
OK A' below Chandrupatla, APS % got 12.5 want 12.5
OK A' below Chandrupatla, CHA % got 17.9 want 17.9
OK A' fun9 vs Chandrupatla got (65, 55) want (65, 55)
OK A' ahead on wide-bracket families got 7 want 7
OK A' level on fun7 got True want True
Table 9 (ablation of A' on six corpora, sweep_ablation_all.json):
OK ablation -regime got (-7, 1013, 1660, 324, 77, 2269) want (-7, 1013, 1660, 324, 77, 2269)
OK ablation -point got (-5, 479, 667, 272, 30, 1672) want (-5, 479, 667, 272, 30, 1672)
OK ablation -sub got (-6, 0, -7, 16, 63, 893) want (-6, 0, -7, 16, 63, 893)
OK ablation -close got (130, 35, 97, 78, 135, -2) want (130, 35, 97, 78, 135, -2)
OK ablation -half got (249, 65, 82, -16, -5, 1) want (249, 65, 82, -16, -5, 1)
OK ablation -exp got (31, 143, 100, -21, -2, 87) want (31, 143, 100, -21, -2, 87)
OK ablation -G2 got (-22, -6, 123, -30, -10, 34) want (-22, -6, 123, -30, -10, 34)
OK ablation -contra got (11, -12, 29, 0, 1, -1) want (11, -12, 29, 0, 1, -1)
Section 9 (out-of-sample corpora):
OK corpus APS size got 154 want 154
OK corpus CHA size got 45 want 45
OK corpus NEW size got 206 want 206
OK corpus RJL size got 78 want 78
OK corpus APP size got 132 want 132
OK corpus D size got 240 want 240
OK Table 10 APS totals got (2259, 2595, 2707, 2636) want (2259, 2595, 2707, 2636)
OK Table 10 APS vs Chandrupatla got (103, 20, 31) want (103, 20, 31)
OK Table 10 APS A' violations got 0 want 0
OK Table 10 CHA totals got (1074, 1505, 3238, 3088) want (1074, 1505, 3238, 3088)
OK Table 10 CHA vs Chandrupatla got (31, 4, 10) want (31, 4, 10)
OK Table 10 CHA A' violations got 0 want 0
OK Table 10 NEW totals got (3290, 3716, 6395, 5746) want (3290, 3716, 6395, 5746)
OK Table 10 NEW vs Chandrupatla got (108, 33, 65) want (108, 33, 65)
OK Table 10 NEW A' violations got 0 want 0
OK Table 10 RJL totals got (1371, 1566, 2455, 2339) want (1371, 1566, 2455, 2339)
OK Table 10 RJL vs Chandrupatla got (44, 14, 20) want (44, 14, 20)
OK Table 10 RJL A' violations got 0 want 0
OK Table 10 APP totals got (1382, 1382, 1503, 1756) want (1382, 1382, 1503, 1756)
OK Table 10 APP vs Chandrupatla got (48, 52, 32) want (48, 52, 32)
OK Table 10 APP A' violations got 0 want 0
OK Table 10 D totals got (3656, 5717, 8267, 8084) want (3656, 5717, 8267, 8084)
OK Table 10 D vs Chandrupatla got (177, 35, 28) want (177, 35, 28)
OK Table 10 D A' violations got 0 want 0
OK Brent / TOMS 748 bound violations over 855 got (118, 84) want (118, 84)
OK all 855 totals got (13032, 16481, 24565, 23649) want (13032, 16481, 24565, 23649)
OK corpus 4 without targeted got (2403, 3569, (107, 31, 22)) want (2403, 3569, (107, 31, 22))
OK corpus 4 simple roots got (737, 747, (43, 22, 15)) want (737, 747, (43, 22, 15))
OK Table 11 D1 got (433, 419, (20, 8, 12)) want (433, 419, (20, 8, 12))
OK Table 11 D2 got (1163, 1805, (39, 1, 0)) want (1163, 1805, (39, 1, 0))
OK Table 11 D3 got (924, 1784, (37, 1, 2)) want (924, 1784, (37, 1, 2))
OK Table 11 D4 got (304, 328, (23, 14, 3)) want (304, 328, (23, 14, 3))
OK Table 11 D5 got (503, 1017, (25, 8, 7)) want (503, 1017, (25, 8, 7))
OK Table 11 D6 got (329, 364, (33, 3, 4)) want (329, 364, (33, 3, 4))
OK near-cancelling class of corpus 1 got (491, 391) want (491, 391)
OK corpus 1, variant carried: total vs Chandrupatla got (3492, 3899) want (3492, 3899)
OK corpus 1, variant carried: per problem got (105, 34, 67) want (105, 34, 67)
OK corpus 1, with defect: per problem got (95, 33, 78) want (95, 33, 78)
OK corpus 1, with defect: exponential class got (9, 5, 26) want (9, 5, 26)
OK Roots.jl, variant carried: totals got (1464, 1566) want (1464, 1566)
OK Roots.jl, variant carried: per problem got (38, 10, 30) want (38, 10, 30)
OK applied, variant carried: totals got (1580, 1382) want (1580, 1382)
every number stated in the paper matches the sweep data.