
== MAIN (78 out-of-sample)
  base           total   1884  solved  78/78  violations 0
  A              total   1464  solved  78/78  violations 0
  B              total   1537  solved  78/78  violations 0
  chandrupatla   total   1566  solved  78/78  violations 0
  brent          total   2455  solved  78/78  violations 11
  toms748        total   2339  solved  78/78  violations 9
  A: R1 ✓ (1464 vs 1566)  R2 ✗ (38/10/30 p=0.4)  R3 ✓ (viol 0, solved 78 vs 78)  -> does NOT transfer
       vs brent    total 1464 vs 2455  per problem 47/6/25 p=0.013
       vs toms748  total 1464 vs 2339  per problem 55/3/20 p=6.5e-05
  B: R1 ✓ (1537 vs 1566)  R2 ✗ (32/11/35 p=0.81)  R3 ✓ (viol 0, solved 78 vs 78)  -> does NOT transfer
       vs brent    total 1537 vs 2455  per problem 43/9/26 p=0.053
       vs toms748  total 1537 vs 2339  per problem 46/6/26 p=0.024

== STRICT (f24-f91, out-of-sample)
  base           total   1734  solved  66/66  violations 0
  A              total   1340  solved  66/66  violations 0
  B              total   1385  solved  66/66  violations 0
  chandrupatla   total   1454  solved  66/66  violations 0
  brent          total   2333  solved  66/66  violations 11
  toms748        total   2216  solved  66/66  violations 9
  A: R1 ✓ (1340 vs 1454)  R2 ✗ (35/6/25 p=0.25)  R3 ✓ (viol 0, solved 66 vs 66)
       vs brent    total 1340 vs 2333  per problem 41/3/22 p=0.023
       vs toms748  total 1340 vs 2216  per problem 48/2/16 p=7.7e-05
  B: R1 ✓ (1385 vs 1454)  R2 ✗ (29/7/30 p=1)  R3 ✓ (viol 0, solved 66 vs 66)
       vs brent    total 1385 vs 2333  per problem 37/8/21 p=0.048
       vs toms748  total 1385 vs 2216  per problem 41/6/19 p=0.0062

== IN-SAMPLE overlaps (13, for reference)
  base           total    142  solved  13/13  violations 0
  A              total    153  solved  13/13  violations 0
  B              total    154  solved  13/13  violations 0
  chandrupatla   total    135  solved  13/13  violations 0
  brent          total    128  solved  13/13  violations 0
  toms748        total    151  solved  13/13  violations 0
  A: R1 ✗ (153 vs 135)  R2 ✗ (4/2/7 p=0.55)  R3 ✓ (viol 0, solved 13 vs 13)
       vs brent    total 153 vs 128  per problem 4/1/8 p=0.39
       vs toms748  total 153 vs 151  per problem 8/2/3 p=0.23
  B: R1 ✗ (154 vs 135)  R2 ✗ (3/0/10 p=0.092)  R3 ✓ (viol 0, solved 13 vs 13)
       vs brent    total 154 vs 128  per problem 3/0/10 p=0.092
       vs toms748  total 154 vs 151  per problem 5/4/4 p=1

PER PROBLEM, out-of-sample (A / B / Chandrupatla / Brent / TOMS748), * = unsolved:
  f01: 8 / 10 / 3 / 10 / 9
  f02: 12 / 12 / 10 / 13 / 10
  f03: 12 / 42 / 12 / 12 / 17
  f04: 9 / 9 / 9 / 10 / 11
  f07: 10 / 13 / 11 / 12 / 11
  f09: 10 / 9 / 10 / 10 / 11
  f10: 10 / 10 / 10 / 9 / 8
  f11: 11 / 10 / 10 / 9 / 7
  f16: 15 / 10 / 8 / 7 / 9
  f21: 10 / 9 / 9 / 10 / 10
  f22: 8 / 9 / 10 / 10 / 10
  f23: 9 / 9 / 10 / 10 / 10
  f24: 26 / 26 / 43 / 122 / 88
  f25: 27 / 27 / 43 / 106 / 76
  f26: 26 / 26 / 41 / 121 / 95
  f27: 17 / 25 / 14 / 32 / 25
  f28: 21 / 21 / 47 / 119 / 73
  f29: 16 / 16 / 12 / 30 / 25
  f30: 17 / 17 / 12 / 13 / 14
  f31: 5 / 13 / 13 / 12 / 18
  f32: 41 / 11 / 12 / 13 / 14
  f33: 17 / 20 / 12 / 14 / 13
  f34: 9 / 9 / 10 / 9 / 10
  f35: 20 / 22 / 16 / 11 / 15
  f36: 49 / 52 / 47 / 39 / 47
  f37: 19 / 19 / 17 / 31 / 29
  f38: 51 / 48 / 47 / 47 / 51
  f39: 59 / 58 / 47 / 73 / 97
  f40: 51 / 47 / 47 / 49 / 77
  f42: 9 / 9 / 11 / 10 / 10
  f43: 25 / 25 / 44 / 106 / 91
  f44: 30 / 30 / 44 / 104 / 106
  f45: 43 / 43 / 44 / 117 / 123
  f46: 14 / 15 / 14 / 15 / 19
  f47: 14 / 13 / 15 / 15 / 18
  f48: 14 / 19 / 17 / 16 / 15
  f49: 13 / 13 / 10 / 11 / 11
  f50: 10 / 13 / 10 / 12 / 13
  f51: 9 / 9 / 9 / 10 / 10
  f52: 18 / 16 / 14 / 17 / 22
  f53: 11 / 11 / 12 / 13 / 11
  f54: 7 / 7 / 7 / 8 / 9
  f55: 8 / 10 / 9 / 11 / 9
  f56: 9 / 10 / 9 / 11 / 10
  f57: 11 / 11 / 11 / 12 / 16
  f58: 8 / 9 / 10 / 10 / 10
  f59: 12 / 10 / 9 / 13 / 13
  f60: 7 / 7 / 8 / 8 / 8
  f61: 10 / 12 / 11 / 11 / 11
  f63: 9 / 11 / 10 / 10 / 10
  f64: 8 / 8 / 9 / 9 / 9
  f65: 8 / 8 / 9 / 9 / 9
  f66: 14 / 13 / 12 / 10 / 13
  f67: 11 / 11 / 12 / 12 / 12
  f68: 7 / 8 / 9 / 8 / 8
  f69: 8 / 8 / 10 / 10 / 11
  f70: 10 / 10 / 12 / 11 / 13
  f71: 5 / 13 / 11 / 13 / 16
  f72: 29 / 25 / 48 / 110 / 112
  f73: 26 / 26 / 43 / 120 / 91
  f74: 26 / 26 / 43 / 122 / 91
  f75: 13 / 18 / 27 / 66 / 61
  f76: 44 / 44 / 43 / 43 / 43
  f77: 51 / 43 / 43 / 46 / 66
  f78: 48 / 48 / 43 / 44 / 45
  f79: 47 / 43 / 40 / 40 / 39
  f80: 23 / 27 / 29 / 21 / 21
  f81: 57 / 44 / 50 / 45 / 39
  f82: 4 / 4 / 5 / 4 / 5
  f83: 32 / 25 / 48 / 106 / 76
  f84: 8 / 8 / 13 / 8 / 19
  f85: 14 / 53 / 15 / 12 / 13
  f86: 19 / 21 / 13 / 16 / 17
  f87: 5 / 33 / 11 / 12 / 14
  f88: 15 / 15 / 12 / 13 / 13
  f89: 16 / 14 / 14 / 14 / 19
  f90: 16 / 15 / 12 / 15 / 14
  f91: 14 / 14 / 10 / 13 / 15
rootsjl done
