Sunday 20 February 2011

IBM Watson and Jeopardy

IBM's Thomas J Watson Research Center in Yorktown Heights, NY has recently completed a new grand challenge - to program a computer to play the quiz game "Jeopardy".

I have been following this (as have many many other people) - and it has been absolutely fascinating.
This link (http://www.youtube.com/watch?v=3bifUJCyMwI) to a youtube of the final session should also give you links to the previous sessions over the 3 days.  The link to details of Watson (http://www-943.ibm.com/innovation/us/watson/) will no doubt also give you the relevant video links, and much more.

Basically, IBM have developed a natural language processing and deep analytic question and answer system, using massively parallel processing and huge amounts of memory (as stated here: 2880 processor cores in 90 Power 750 computers and 15 terabytes of RAM) to implement a system which can answer any sort of general knowledge question (which have been asked in a variety of ways, including through association, analogy, puns, etc), and to get so many correct that Watson totally beat the best human players.

The results were fascinating.

At the end of the first day, Ken Jennings was on $4,800, Brad Rutter was on $10,400 but Watson was a massive $35,734 (I also answered the questions as they appeared on the screen and achieved $22,400 - although one can not completely equate the results, since the physical presence of having to press the button first when the light comes on and then answer was not the same for me watching it on a computer screen).

At the end of the second and final day, Brad scored $5,600 before final jeopardy, wagered the lot to obtain $11,200 which totaled him $21,600 over the 2 days.

Ken did much better on the second day, managing a pre-final jeopardy score of $18,200 but only wagered $1,000 to finish with $19,200, to total $24,000 for the 2 days.

But Watson.  Well, he (since we can really be anthropomorphic here) scored $23,440 before final jeopardy, wagered $17,973 to make his daily score $41,413 and a massive total of $77,147 for the 2 days.

(By the way, I managed $14,000 for the second day, wagered the lot and got the final jeopardy answer correct (it was Bram Stoker) to finish with $28,000 on the day and $50,400 over the 2 days).

The prize money of $1,000,000 awarded to Watson was donated by IBM to World Vision and to the World Community Grid, whereas half the second prize of $300,000 (to Ken) and $200,000 (to Brad) was donated to other charities.

Two important take-aways from this brilliant piece of research.

Firstly, this technology from IBM has so so many uses - not just in the medical field (as the first offerings appear to be) but also in the energy and resources fields, the urban planning fields, and certainly in the legal and justice fields.  The ability to ingest natural language materials (such as legislation, case law, briefs, submissions, depositions, statements, judgments and miscellaneous other materials) and then to answer complicated questions concerning that material (and link to associated material not previously related to the matter) will be extremely important in the future.

Secondly, IBM Watson was truly amazing.  Certainly a breakthrough in technology.  But the human beings standing there, that did pretty well against the massive machine, were still, themselves, rather incredible.  Humans, in essence, are still mighty powerful.  The Jeopardy show had to be filmed on a special set built in the IBM Research Facility, because the computer system comprising Watson took up a whole room and was too massive to move.  Whereas Ken and Brad simply walked into where ever they were needed and did their thing.  Mind you, computer systems in the 1960's and 1970's took whole rooms - and their capability would now be eclipsed by an iPad or small notebook computer.  Twenty years from now, Watson will definitely be in the palm of one's hand (in one form or another).

No comments: