ARC Analysis Results

Pass@1
0.0944
Pass@2
0.1111
Oracle
0.1542
Total Tasks
120
Filter tasks:

Task: 0934a4d8

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 135a2760

Example 0

Input

Ground Truth

Top 1 — votes: 28 — Incorrect

Top 2 — votes: 27 — Incorrect

Task: 136b0064

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 13e47133

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 142ca369

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 716 — Incorrect

Top 2 — votes: 80 — Incorrect

Task: 16b78196

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 16de56c4

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 4 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 11 — Incorrect

Top 2 — votes: 11 — Incorrect

Task: 1818057f

Example 0

Input

Ground Truth

Top 1 — votes: 157 — Incorrect

Top 2 — votes: 42 — Incorrect

Task: 195c6913

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 1ae2feb7

Example 0

Input

Ground Truth

Top 1 — votes: 9 — Incorrect

Top 2 — votes: 7 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 11 — Incorrect

Top 2 — votes: 9 — Incorrect

Example 2

Input

Ground Truth

Top 1 — votes: 44 — Incorrect

Top 2 — votes: 42 — Incorrect

Task: 20270e3b

Example 0

Input

Ground Truth

Top 1 — votes: 125 — Incorrect

Top 2 — votes: 32 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 14 — Incorrect

Top 2 — votes: 11 — Incorrect

Task: 20a9e565

Example 0

Input

Ground Truth

Top 1 — votes: 127 — Incorrect

Top 2 — votes: 63 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 32 — Incorrect

Top 2 — votes: 24 — Incorrect

Task: 21897d95

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 2 — Incorrect

Top 2 — votes: 2 — Incorrect

Task: 221dfab4

Example 0

Input

Ground Truth

Top 1 — votes: 402 — Correct

Top 2 — votes: 70 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 193 — Correct

Top 2 — votes: 40 — Incorrect

Task: 247ef758

Example 0

Input

Ground Truth

Top 1 — votes: 17 — Incorrect

Top 2 — votes: 8 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 10 — Incorrect

Top 2 — votes: 9 — Incorrect

Task: 269e22fb

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 271d71e2

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: 28a6681f

Example 0

Input

Ground Truth

Top 1 — votes: 285 — Incorrect

Top 2 — votes: 202 — Incorrect

Task: 291dc1e1

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: 2b83f449

Example 0

Input

Ground Truth

Top 1 — votes: 27 — Incorrect

Top 2 — votes: 21 — Incorrect

Task: 2ba387bc

Example 0

Input

Ground Truth

Top 1 — votes: 9 — Incorrect

Top 2 — votes: 9 — Incorrect

Task: 2c181942

Example 0

Input

Ground Truth

Top 1 — votes: 41 — Incorrect

Top 2 — votes: 36 — Incorrect

Task: 2d0172a1

Example 0

Input

Ground Truth

Top 1 — votes: 8 — Incorrect

Top 2 — votes: 7 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 9 — Incorrect

Top 2 — votes: 6 — Incorrect

Task: 31f7f899

Example 0

Input

Ground Truth

Top 1 — votes: 161 — Incorrect

Top 2 — votes: 68 — Incorrect

Task: 332f06d7

Example 0

Input

Ground Truth

Top 1 — votes: 121 — Incorrect

Top 2 — votes: 112 — Incorrect

Task: 35ab12c3

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 36a08778

Example 0

Input

Ground Truth

Top 1 — votes: 199 — Incorrect

Top 2 — votes: 155 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 38007db0

Example 0

Input

Ground Truth

Top 1 — votes: 314 — Incorrect

Top 2 — votes: 239 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 682 — Incorrect

Top 2 — votes: 515 — Incorrect

Task: 3a25b0d8

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 6 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 46 — Incorrect

Top 2 — votes: 29 — Incorrect

Task: 3dc255db

Example 0

Input

Ground Truth

Top 1 — votes: 229 — Incorrect

Top 2 — votes: 139 — Incorrect

Task: 3e6067c3

Example 0

Input

Ground Truth

Top 1 — votes: 33 — Incorrect

Top 2 — votes: 24 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 65 — Incorrect

Top 2 — votes: 41 — Incorrect

Task: 409aa875

Example 0

Input

Ground Truth

Top 1 — votes: 391 — Incorrect

Top 2 — votes: 179 — Incorrect

Task: 446ef5d2

Example 0

Input

Ground Truth

Top 1 — votes: 2 — Incorrect

Top 2 — votes: 2 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 45a5af55

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: 4a21e3da

Example 0

Input

Ground Truth

Top 1 — votes: 313 — Correct

Top 2 — votes: 62 — Incorrect

Task: 4c3d4a41

Example 0

Input

Ground Truth

Top 1 — votes: 1160 — Incorrect

Top 2 — votes: 615 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 1001 — Incorrect

Top 2 — votes: 797 — Incorrect

Task: 4c416de3

Example 0

Input

Ground Truth

Top 1 — votes: 1530 — Incorrect

Top 2 — votes: 448 — Incorrect

Task: 4c7dc4dd

Example 0

Input

Ground Truth

Top 1 — votes: 213 — Incorrect

Top 2 — votes: 100 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 241 — Incorrect

Top 2 — votes: 96 — Incorrect

Task: 4e34c42c

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 53fb4810

Example 0

Input

Ground Truth

Top 1 — votes: 97 — Incorrect

Top 2 — votes: 43 — Incorrect

Task: 5545f144

Example 0

Input

Ground Truth

Top 1 — votes: 652 — Incorrect

Top 2 — votes: 150 — Incorrect

Task: 581f7754

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 58490d8a

Example 0

Input

Ground Truth

Top 1 — votes: 76 — Incorrect

Top 2 — votes: 52 — Incorrect

Task: 58f5dbd5

Example 0

Input

Ground Truth

Top 1 — votes: 8 — Incorrect

Top 2 — votes: 7 — Incorrect

Task: 5961cc34

Example 0

Input

Ground Truth

Top 1 — votes: 2547 — Incorrect

Top 2 — votes: 146 — Incorrect

Task: 5dbc8537

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 62593bfd

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 64efde09

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 65b59efc

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: 67e490f4

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 6e453dd6

Example 0

Input

Ground Truth

Top 1 — votes: 41 — Incorrect

Top 2 — votes: 35 — Incorrect

Task: 6e4f6532

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 6 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 8 — Incorrect

Top 2 — votes: 7 — Incorrect

Task: 6ffbe589

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 71e489b6

Example 0

Input

Ground Truth

Top 1 — votes: 3010 — Correct

Top 2 — votes: 137 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 2899 — Correct

Top 2 — votes: 246 — Incorrect

Task: 7491f3cf

Example 0

Input

Ground Truth

Top 1 — votes: 254 — Incorrect

Top 2 — votes: 250 — Incorrect

Task: 7666fa5d

Example 0

Input

Ground Truth

Top 1 — votes: 3475 — Correct

Top 2 — votes: 87 — Incorrect

Task: 78332cb0

Example 0

Input

Ground Truth

Top 1 — votes: 188 — Incorrect

Top 2 — votes: 82 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 79 — Incorrect

Top 2 — votes: 68 — Incorrect

Task: 7b0280bc

Example 0

Input

Ground Truth

Top 1 — votes: 25 — Incorrect

Top 2 — votes: 15 — Incorrect

Task: 7b3084d4

Example 0

Input

Ground Truth

Top 1 — votes: 8 — Incorrect

Top 2 — votes: 6 — Incorrect

Task: 7b5033c1

Example 0

Input

Ground Truth

Top 1 — votes: 152 — Incorrect

Top 2 — votes: 138 — Incorrect

Task: 7b80bb43

Example 0

Input

Ground Truth

Top 1 — votes: 658 — Correct

Top 2 — votes: 513 — Incorrect

Task: 7c66cb00

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 6 — Incorrect

Task: 7ed72f31

Example 0

Input

Ground Truth

Top 1 — votes: 168 — Incorrect

Top 2 — votes: 101 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 106 — Incorrect

Top 2 — votes: 66 — Incorrect

Task: 800d221b

Example 0

Input

Ground Truth

Top 1 — votes: 251 — Incorrect

Top 2 — votes: 147 — Correct

Task: 80a900e0

Example 0

Input

Ground Truth

Top 1 — votes: 16 — Incorrect

Top 2 — votes: 10 — Incorrect

Task: 8698868d

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 88bcf3b4

Example 0

Input

Ground Truth

Top 1 — votes: 27 — Incorrect

Top 2 — votes: 23 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 88e364bc

Example 0

Input

Ground Truth

Top 1 — votes: 359 — Incorrect

Top 2 — votes: 222 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 156 — Incorrect

Top 2 — votes: 138 — Incorrect

Task: 89565ca0

Example 0

Input

Ground Truth

Top 1 — votes: 34 — Incorrect

Top 2 — votes: 24 — Incorrect

No data

Task: 898e7135

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 8b7bacbf

Example 0

Input

Ground Truth

Top 1 — votes: 424 — Incorrect

Top 2 — votes: 184 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 422 — Incorrect

Top 2 — votes: 308 — Incorrect

Task: 8b9c3697

Example 0

Input

Ground Truth

Top 1 — votes: 1510 — Correct

Top 2 — votes: 232 — Incorrect

Task: 8e5c0c38

Example 0

Input

Ground Truth

Top 1 — votes: 28 — Incorrect

Top 2 — votes: 24 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 106 — Incorrect

Top 2 — votes: 102 — Incorrect

Task: 8f215267

Example 0

Input

Ground Truth

Top 1 — votes: 457 — Incorrect

Top 2 — votes: 406 — Incorrect

Task: 8f3a5a89

Example 0

Input

Ground Truth

Top 1 — votes: 1255 — Incorrect

Top 2 — votes: 638 — Incorrect

Task: 9385bd28

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 4 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 97d7923e

Example 0

Input

Ground Truth

Top 1 — votes: 800 — Incorrect

Top 2 — votes: 546 — Incorrect

Task: 981571dc

Example 0

Input

Ground Truth

Top 1 — votes: 1602 — Correct

Top 2 — votes: 131 — Incorrect

Task: 9aaea919

Example 0

Input

Ground Truth

Top 1 — votes: 73 — Incorrect

Top 2 — votes: 72 — Incorrect

Task: 9bbf930d

Example 0

Input

Ground Truth

Top 1 — votes: 50 — Incorrect

Top 2 — votes: 49 — Incorrect

Task: a251c730

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: a25697e4

Example 0

Input

Ground Truth

Top 1 — votes: 42 — Incorrect

Top 2 — votes: 27 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 12 — Incorrect

Top 2 — votes: 11 — Incorrect

Task: a32d8b75

Example 0

Input

Ground Truth

Top 1 — votes: 10 — Incorrect

No data

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: a395ee82

Example 0

Input

Ground Truth

Top 1 — votes: 61 — Incorrect

Top 2 — votes: 20 — Incorrect

Task: a47bf94d

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: a6f40cea

Example 0

Input

Ground Truth

Top 1 — votes: 8 — Incorrect

Top 2 — votes: 6 — Incorrect

Task: aa4ec2a5

Example 0

Input

Ground Truth

Top 1 — votes: 410 — Correct

Top 2 — votes: 35 — Incorrect

Task: abc82100

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: b0039139

Example 0

Input

Ground Truth

Top 1 — votes: 18 — Incorrect

Top 2 — votes: 11 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 23 — Incorrect

Top 2 — votes: 9 — Incorrect

Task: b10624e5

Example 0

Input

Ground Truth

Top 1 — votes: 32 — Incorrect

Top 2 — votes: 19 — Incorrect

Task: b5ca7ac4

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: b6f77b65

Example 0

Input

Ground Truth

Top 1 — votes: 60 — Incorrect

Top 2 — votes: 30 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 9 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: b99e7126

Example 0

Input

Ground Truth

Top 1 — votes: 552 — Incorrect

Top 2 — votes: 198 — Incorrect

Task: b9e38dc0

Example 0

Input

Ground Truth

Top 1 — votes: 11 — Incorrect

Top 2 — votes: 10 — Incorrect

Task: bf45cf4b

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: c4d067a0

Example 0

Input

Ground Truth

Top 1 — votes: 39 — Incorrect

Top 2 — votes: 21 — Incorrect

Task: c7f57c3e

Example 0

Input

Ground Truth

Top 1 — votes: 171 — Incorrect

Top 2 — votes: 165 — Incorrect

Task: cb2d8a2c

Example 0

Input

Ground Truth

Top 1 — votes: 15 — Incorrect

Top 2 — votes: 12 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 27 — Incorrect

Top 2 — votes: 25 — Incorrect

Task: cbebaa4b

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: d35bdbdc

Example 0

Input

Ground Truth

Top 1 — votes: 118 — Incorrect

Top 2 — votes: 104 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 1511 — Incorrect

Top 2 — votes: 185 — Incorrect

Example 2

Input

Ground Truth

Top 1 — votes: 517 — Correct

Top 2 — votes: 195 — Incorrect

Task: d59b0160

Example 0

Input

Ground Truth

Top 1 — votes: 41 — Incorrect

Top 2 — votes: 38 — Correct

Task: d8e07eb2

Example 0

Input

Ground Truth

Top 1 — votes: 238 — Incorrect

Top 2 — votes: 203 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 953 — Incorrect

Top 2 — votes: 156 — Incorrect

Task: da515329

Example 0

Input

Ground Truth

Top 1 — votes: 234 — Correct

Top 2 — votes: 34 — Incorrect

Task: db0c5428

Example 0

Input

Ground Truth

Top 1 — votes: 2453 — Correct

Top 2 — votes: 184 — Incorrect

Task: db695cfb

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: dbff022c

Example 0

Input

Ground Truth

Top 1 — votes: 228 — Incorrect

Top 2 — votes: 160 — Incorrect

Task: dd6b8c4b

Example 0

Input

Ground Truth

Top 1 — votes: 1410 — Incorrect

Top 2 — votes: 573 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 93 — Incorrect

Top 2 — votes: 79 — Incorrect

Task: de809cff

Example 0

Input

Ground Truth

Top 1 — votes: 243 — Correct

Top 2 — votes: 115 — Incorrect

Task: dfadab01

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: e12f9a14

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: e3721c99

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 7 — Incorrect

Top 2 — votes: 6 — Incorrect

Task: e376de54

Example 0

Input

Ground Truth

Top 1 — votes: 161 — Incorrect

Top 2 — votes: 159 — Incorrect

Task: e8686506

Example 0

Input

Ground Truth

Top 1 — votes: 329 — Incorrect

Top 2 — votes: 71 — Incorrect

Task: e87109e9

Example 0

Input

Ground Truth

Top 1 — votes: 111 — Incorrect

Top 2 — votes: 27 — Incorrect

Task: edb79dae

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: eee78d87

Example 0

Input

Ground Truth

Top 1 — votes: 1235 — Incorrect

Top 2 — votes: 419 — Incorrect

Task: f560132c

Example 0

Input

Ground Truth

Top 1 — votes: 10 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: f931b4a8

Example 0

Input

Ground Truth

Top 1 — votes: 18 — Incorrect

Top 2 — votes: 10 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 312 — Incorrect

Top 2 — votes: 176 — Incorrect

Task: faa9f03d

Example 0

Input

Ground Truth

Top 1 — votes: 652 — Incorrect

Top 2 — votes: 600 — Incorrect

Task: fc7cae8d

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect