|
|
|
|
|
|
|
Each of the judges (numbered 1 to 9) conversed with each of the ten terminals (labelled A to J) in a randomised order. They gave each terminal a "humanness" score, on the following scale:
|
Score |
Was your conversational partner a human or a machine? |
|
0 |
Partner not accessible, or severe system malfunction |
|
1 |
Definitely a machine |
|
2 |
Probably a machine |
|
3 |
Could be a machine or a human; undecided |
|
4 |
Probably a human |
|
5 |
Definitely a human |
Fractional values were permitted. After every judge had conversed with every terminal, their score sheets were collected and their scores were entered into a spreadsheet, which calculated the mean humanness score for each terminal.
The final results, ranked in decreasing mean humanness, are as follows:
|
Rank |
Entity |
Contestant |
T |
J1 |
J2 |
J3 |
J4 |
J5 |
J6 |
J7 |
J8 |
J9 |
Mean |
|
1 |
Confederate 2 |
|
H |
4.50 |
5.00 |
4.80 |
3.50 |
5.00 |
5.00 |
1.00 |
5.00 |
1.00 |
3.867 |
|
2 |
Confederate 1 |
|
D |
5.00 |
2.00 |
4.10 |
1.00 |
5.00 |
4.90 |
5.00 |
5.00 |
1.00 |
3.667 |
|
3 |
Jabberwock |
Juergen Pirner |
C |
1.00 |
1.00 |
1.25 |
4.00 |
1.20 |
2.00 |
3.00 |
2.90 |
1.00 |
1.928 |
|
4 |
Elbot |
Fred Roberts |
J |
1.00 |
1.00 |
3.50 |
1.00 |
1.50 |
1.50 |
2.00 |
2.60 |
1.00 |
1.678 |
|
5 = |
Eugene Goostmann |
Vladimir Veselov and |
A |
1.10 |
1.00 |
2.20 |
1.00 |
1.00 |
2.50 |
2.00 |
3.00 |
1.00 |
1.644 |
|
5 = |
Jabberwacky |
Rollo Carpenter |
F |
1.00 |
2.00 |
1.45 |
1.00 |
1.30 |
2.25 |
2.00 |
2.80 |
1.00 |
1.644 |
|
7 |
Lucy |
Saskia van der Elst |
I |
1.00 |
1.00 |
1.10 |
1.00 |
1.10 |
1.50 |
3.00 |
1.70 |
1.00 |
1.378 |
|
8 |
Markbot |
Mark Connell |
B |
1.00 |
1.00 |
1.50 |
1.00 |
1.00 |
1.70 |
1.00 |
1.60 |
2.00 |
1.311 |
|
9 |
ALICE |
Richard Wallace |
E |
1.00 |
1.00 |
1.70 |
1.00 |
1.00 |
2.00 |
1.00 |
1.90 |
1.00 |
1.289 |
|
10 |
Gabber |
Peter Neuendorffer |
G |
1.00 |
2.00 |
1.10 |
1.00 |
1.10 |
1.00 |
1.00 |
1.50 |
1.00 |
1.189 |
As both human confederates outranked the computer entries, the Silver Award was not presented.
The Bronze Award went to Jabberwock, the "most human bot" according to the judges.
|
|
|
|
|