ARC Analysis Results

Pass@1
0.0667
Pass@2
0.0833
Oracle
0.1056
Total Tasks
120
Filter tasks:

Task: 0934a4d8

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 135a2760

Example 0

Input

Ground Truth

Top 1 — votes: 9 — Incorrect

Top 2 — votes: 9 — Incorrect

Task: 136b0064

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 13e47133

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 142ca369

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 30 — Incorrect

Top 2 — votes: 25 — Incorrect

Task: 16b78196

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 16de56c4

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 4 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 1818057f

Example 0

Input

Ground Truth

Top 1 — votes: 16 — Incorrect

Top 2 — votes: 10 — Incorrect

Task: 195c6913

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 1ae2feb7

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 6 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 11 — Incorrect

Top 2 — votes: 7 — Incorrect

Example 2

Input

Ground Truth

Top 1 — votes: 18 — Incorrect

Top 2 — votes: 15 — Incorrect

Task: 20270e3b

Example 0

Input

Ground Truth

Top 1 — votes: 22 — Incorrect

Top 2 — votes: 8 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 9 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 20a9e565

Example 0

Input

Ground Truth

Top 1 — votes: 17 — Incorrect

Top 2 — votes: 14 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 7 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 21897d95

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 2 — Incorrect

Top 2 — votes: 2 — Incorrect

Task: 221dfab4

Example 0

Input

Ground Truth

Top 1 — votes: 141 — Correct

Top 2 — votes: 10 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 30 — Correct

Top 2 — votes: 17 — Incorrect

Task: 247ef758

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: 269e22fb

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 271d71e2

Example 0

Input

Ground Truth

Top 1 — votes: 2 — Incorrect

Top 2 — votes: 2 — Incorrect

Task: 28a6681f

Example 0

Input

Ground Truth

Top 1 — votes: 68 — Incorrect

Top 2 — votes: 59 — Incorrect

Task: 291dc1e1

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: 2b83f449

Example 0

Input

Ground Truth

Top 1 — votes: 21 — Incorrect

Top 2 — votes: 20 — Incorrect

Task: 2ba387bc

Example 0

Input

Ground Truth

Top 1 — votes: 9 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 2c181942

Example 0

Input

Ground Truth

Top 1 — votes: 28 — Incorrect

Top 2 — votes: 12 — Incorrect

Task: 2d0172a1

Example 0

Input

Ground Truth

Top 1 — votes: 7 — Incorrect

Top 2 — votes: 6 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 6 — Incorrect

Task: 31f7f899

Example 0

Input

Ground Truth

Top 1 — votes: 18 — Incorrect

Top 2 — votes: 17 — Incorrect

Task: 332f06d7

Example 0

Input

Ground Truth

Top 1 — votes: 43 — Incorrect

Top 2 — votes: 37 — Incorrect

Task: 35ab12c3

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 36a08778

Example 0

Input

Ground Truth

Top 1 — votes: 64 — Incorrect

Top 2 — votes: 61 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 38007db0

Example 0

Input

Ground Truth

Top 1 — votes: 41 — Incorrect

Top 2 — votes: 26 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 180 — Incorrect

Top 2 — votes: 81 — Incorrect

Task: 3a25b0d8

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 4 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 7 — Incorrect

Top 2 — votes: 7 — Incorrect

Task: 3dc255db

Example 0

Input

Ground Truth

Top 1 — votes: 53 — Incorrect

Top 2 — votes: 35 — Incorrect

Task: 3e6067c3

Example 0

Input

Ground Truth

Top 1 — votes: 15 — Incorrect

Top 2 — votes: 9 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 14 — Incorrect

Top 2 — votes: 12 — Incorrect

Task: 409aa875

Example 0

Input

Ground Truth

Top 1 — votes: 152 — Incorrect

Top 2 — votes: 35 — Incorrect

Task: 446ef5d2

Example 0

Input

Ground Truth

Top 1 — votes: 2 — Incorrect

Top 2 — votes: 2 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: 45a5af55

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 2 — Incorrect

Task: 4a21e3da

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 4c3d4a41

Example 0

Input

Ground Truth

Top 1 — votes: 281 — Incorrect

Top 2 — votes: 153 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 394 — Incorrect

Top 2 — votes: 77 — Incorrect

Task: 4c416de3

Example 0

Input

Ground Truth

Top 1 — votes: 370 — Incorrect

Top 2 — votes: 50 — Incorrect

Task: 4c7dc4dd

Example 0

Input

Ground Truth

Top 1 — votes: 20 — Incorrect

Top 2 — votes: 19 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 44 — Incorrect

Top 2 — votes: 34 — Incorrect

Task: 4e34c42c

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 53fb4810

Example 0

Input

Ground Truth

Top 1 — votes: 15 — Incorrect

Top 2 — votes: 10 — Incorrect

Task: 5545f144

Example 0

Input

Ground Truth

Top 1 — votes: 272 — Incorrect

Top 2 — votes: 22 — Incorrect

Task: 581f7754

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 58490d8a

Example 0

Input

Ground Truth

Top 1 — votes: 22 — Incorrect

Top 2 — votes: 18 — Incorrect

Task: 58f5dbd5

Example 0

Input

Ground Truth

Top 1 — votes: 8 — Incorrect

Top 2 — votes: 7 — Incorrect

Task: 5961cc34

Example 0

Input

Ground Truth

Top 1 — votes: 218 — Incorrect

Top 2 — votes: 41 — Incorrect

Task: 5dbc8537

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 2 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: 62593bfd

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 64efde09

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: 65b59efc

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 2 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 2 — Incorrect

Task: 67e490f4

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 6e453dd6

Example 0

Input

Ground Truth

Top 1 — votes: 15 — Incorrect

Top 2 — votes: 12 — Incorrect

Task: 6e4f6532

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 7 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 6ffbe589

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 71e489b6

Example 0

Input

Ground Truth

Top 1 — votes: 453 — Correct

Top 2 — votes: 8 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 465 — Correct

Top 2 — votes: 18 — Incorrect

Task: 7491f3cf

Example 0

Input

Ground Truth

Top 1 — votes: 83 — Incorrect

Top 2 — votes: 49 — Incorrect

Task: 7666fa5d

Example 0

Input

Ground Truth

Top 1 — votes: 410 — Correct

Top 2 — votes: 16 — Incorrect

Task: 78332cb0

Example 0

Input

Ground Truth

Top 1 — votes: 31 — Incorrect

Top 2 — votes: 21 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 25 — Incorrect

Top 2 — votes: 20 — Incorrect

Task: 7b0280bc

Example 0

Input

Ground Truth

Top 1 — votes: 18 — Incorrect

Top 2 — votes: 9 — Incorrect

Task: 7b3084d4

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 7b5033c1

Example 0

Input

Ground Truth

Top 1 — votes: 38 — Incorrect

Top 2 — votes: 29 — Incorrect

Task: 7b80bb43

Example 0

Input

Ground Truth

Top 1 — votes: 168 — Correct

Top 2 — votes: 44 — Incorrect

Task: 7c66cb00

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 7ed72f31

Example 0

Input

Ground Truth

Top 1 — votes: 67 — Incorrect

Top 2 — votes: 51 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 9 — Incorrect

Top 2 — votes: 8 — Incorrect

Task: 800d221b

Example 0

Input

Ground Truth

Top 1 — votes: 99 — Incorrect

Top 2 — votes: 82 — Correct

Task: 80a900e0

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: 8698868d

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 88bcf3b4

Example 0

Input

Ground Truth

Top 1 — votes: 9 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 88e364bc

Example 0

Input

Ground Truth

Top 1 — votes: 77 — Incorrect

Top 2 — votes: 66 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 30 — Incorrect

Top 2 — votes: 30 — Incorrect

Task: 89565ca0

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: 898e7135

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: 8b7bacbf

Example 0

Input

Ground Truth

Top 1 — votes: 89 — Incorrect

Top 2 — votes: 53 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 117 — Incorrect

Top 2 — votes: 34 — Incorrect

Task: 8b9c3697

Example 0

Input

Ground Truth

Top 1 — votes: 290 — Correct

Top 2 — votes: 14 — Incorrect

Task: 8e5c0c38

Example 0

Input

Ground Truth

Top 1 — votes: 10 — Incorrect

Top 2 — votes: 8 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 31 — Incorrect

Top 2 — votes: 19 — Incorrect

Task: 8f215267

Example 0

Input

Ground Truth

Top 1 — votes: 136 — Incorrect

Top 2 — votes: 95 — Incorrect

Task: 8f3a5a89

Example 0

Input

Ground Truth

Top 1 — votes: 232 — Incorrect

Top 2 — votes: 14 — Incorrect

Task: 9385bd28

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: 97d7923e

Example 0

Input

Ground Truth

Top 1 — votes: 218 — Incorrect

Top 2 — votes: 99 — Incorrect

Task: 981571dc

Example 0

Input

Ground Truth

Top 1 — votes: 388 — Correct

Top 2 — votes: 36 — Incorrect

Task: 9aaea919

Example 0

Input

Ground Truth

Top 1 — votes: 26 — Incorrect

Top 2 — votes: 10 — Incorrect

Task: 9bbf930d

Example 0

Input

Ground Truth

Top 1 — votes: 17 — Incorrect

Top 2 — votes: 16 — Incorrect

Task: a251c730

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: a25697e4

Example 0

Input

Ground Truth

Top 1 — votes: 21 — Incorrect

Top 2 — votes: 17 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 9 — Incorrect

Top 2 — votes: 6 — Incorrect

Task: a32d8b75

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: a395ee82

Example 0

Input

Ground Truth

Top 1 — votes: 10 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: a47bf94d

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: a6f40cea

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: aa4ec2a5

Example 0

Input

Ground Truth

Top 1 — votes: 11 — Incorrect

Top 2 — votes: 8 — Incorrect

Task: abc82100

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: b0039139

Example 0

Input

Ground Truth

Top 1 — votes: 18 — Incorrect

Top 2 — votes: 10 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 8 — Incorrect

Top 2 — votes: 6 — Incorrect

Task: b10624e5

Example 0

Input

Ground Truth

Top 1 — votes: 7 — Incorrect

Top 2 — votes: 6 — Incorrect

Task: b5ca7ac4

Example 0

Input

Ground Truth

Top 1 — votes: 2 — Incorrect

Top 2 — votes: 2 — Incorrect

Task: b6f77b65

Example 0

Input

Ground Truth

Top 1 — votes: 46 — Incorrect

Top 2 — votes: 15 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: b99e7126

Example 0

Input

Ground Truth

Top 1 — votes: 16 — Incorrect

Top 2 — votes: 11 — Incorrect

Task: b9e38dc0

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: bf45cf4b

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 2 — Incorrect

Task: c4d067a0

Example 0

Input

Ground Truth

Top 1 — votes: 21 — Incorrect

Top 2 — votes: 18 — Incorrect

Task: c7f57c3e

Example 0

Input

Ground Truth

Top 1 — votes: 85 — Incorrect

Top 2 — votes: 50 — Incorrect

Task: cb2d8a2c

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: cbebaa4b

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: d35bdbdc

Example 0

Input

Ground Truth

Top 1 — votes: 16 — Incorrect

Top 2 — votes: 9 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 291 — Incorrect

Top 2 — votes: 14 — Incorrect

Example 2

Input

Ground Truth

Top 1 — votes: 48 — Incorrect

Top 2 — votes: 47 — Incorrect

Task: d59b0160

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 6 — Incorrect

Task: d8e07eb2

Example 0

Input

Ground Truth

Top 1 — votes: 47 — Incorrect

Top 2 — votes: 33 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 104 — Incorrect

Top 2 — votes: 81 — Incorrect

Task: da515329

Example 0

Input

Ground Truth

Top 1 — votes: 27 — Correct

Top 2 — votes: 15 — Incorrect

Task: db0c5428

Example 0

Input

Ground Truth

Top 1 — votes: 468 — Correct

Top 2 — votes: 8 — Incorrect

Task: db695cfb

Example 0

Input

Ground Truth

Top 1 — votes: 3 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: dbff022c

Example 0

Input

Ground Truth

Top 1 — votes: 71 — Incorrect

Top 2 — votes: 50 — Incorrect

Task: dd6b8c4b

Example 0

Input

Ground Truth

Top 1 — votes: 219 — Incorrect

Top 2 — votes: 52 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 65 — Incorrect

Top 2 — votes: 21 — Incorrect

Task: de809cff

Example 0

Input

Ground Truth

Top 1 — votes: 58 — Incorrect

Top 2 — votes: 36 — Correct

Task: dfadab01

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: e12f9a14

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: e3721c99

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 5 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: e376de54

Example 0

Input

Ground Truth

Top 1 — votes: 16 — Incorrect

Top 2 — votes: 15 — Incorrect

Task: e8686506

Example 0

Input

Ground Truth

Top 1 — votes: 22 — Incorrect

Top 2 — votes: 19 — Incorrect

Task: e87109e9

Example 0

Input

Ground Truth

Top 1 — votes: 6 — Incorrect

Top 2 — votes: 5 — Incorrect

Task: edb79dae

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 4 — Incorrect

Task: eee78d87

Example 0

Input

Ground Truth

Top 1 — votes: 128 — Incorrect

Top 2 — votes: 117 — Incorrect

Task: f560132c

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Task: f931b4a8

Example 0

Input

Ground Truth

Top 1 — votes: 4 — Incorrect

Top 2 — votes: 3 — Incorrect

Example 1

Input

Ground Truth

Top 1 — votes: 110 — Incorrect

Top 2 — votes: 37 — Incorrect

Task: faa9f03d

Example 0

Input

Ground Truth

Top 1 — votes: 83 — Incorrect

Top 2 — votes: 61 — Incorrect

Task: fc7cae8d

Example 0

Input

Ground Truth

Top 1 — votes: 5 — Incorrect

Top 2 — votes: 4 — Incorrect