Loebner Prize Contest 2003 'Can machines think?' - Alan Turing, 1950

Results

Methodology

Each of the judges (numbered 1 to 9) conversed with each of the ten terminals (labelled A to J) in a randomised order. They gave each terminal a "humanness" score, on the following scale:

Score

Was your conversational partner a human or a machine?

0

Partner not accessible, or severe system malfunction

1

Definitely a machine

2

Probably a machine

3

Could be a machine or a human; undecided

4

Probably a human

5

Definitely a human

 

Fractional values were permitted. After every judge had conversed with every terminal, their score sheets were collected and their scores were entered into a spreadsheet, which calculated the mean humanness score for each terminal.

Final Results

The final results, ranked in decreasing mean humanness, are as follows:

Rank

Entity

Contestant

T

J1

J2

J3

J4

J5

J6

J7

J8

J9

Mean

1

Confederate 2

 

H

4.50

5.00

4.80

3.50

5.00

5.00

1.00

5.00

1.00

3.867

2

Confederate 1

 

D

5.00

2.00

4.10

1.00

5.00

4.90

5.00

5.00

1.00

3.667

3

Jabberwock

Juergen Pirner

C

1.00

1.00

1.25

4.00

1.20

2.00

3.00

2.90

1.00

1.928

4

Elbot

Fred Roberts

J

1.00

1.00

3.50

1.00

1.50

1.50

2.00

2.60

1.00

1.678

5 =

Eugene Goostmann

Vladimir Veselov and
Eugene Demchenko

A

1.10

1.00

2.20

1.00

1.00

2.50

2.00

3.00

1.00

1.644

5 =

Jabberwacky

Rollo Carpenter

F

1.00

2.00

1.45

1.00

1.30

2.25

2.00

2.80

1.00

1.644

7

Lucy

Saskia van der Elst

I

1.00

1.00

1.10

1.00

1.10

1.50

3.00

1.70

1.00

1.378

8

Markbot

Mark Connell

B

1.00

1.00

1.50

1.00

1.00

1.70

1.00

1.60

2.00

1.311

9

ALICE

Richard Wallace

E

1.00

1.00

1.70

1.00

1.00

2.00

1.00

1.90

1.00

1.289

10

Gabber

Peter Neuendorffer

G

1.00

2.00

1.10

1.00

1.10

1.00

1.00

1.50

1.00

1.189

 

As both human confederates outranked the computer entries, the Silver Award was not presented.

The Bronze Award went to Jabberwock, the "most human bot" according to the judges.