By Jo
Best
Between them, they'd racked up over $5
million in winnings on the television quiz show Jeopardy.
They were the best players the show had produced over its decades-long
lifetime: Ken Jennings had the longest unbeaten run at 74 winning appearances,
while Brad Rutter had earned the biggest prize pot with a total of $3.25
million.
Rutter and Jennings were
Jeopardy-winning machines. And in early 2011, they agreed to an exhibition
match against an opponent who'd never even stood behind a Jeopardy podium before.
But this Jeopardy unknown had spent
years preparing to take on the two giants in the $1m match, playing 100 games
against past winners in an effort to improve his chances of winning.
That opponent didn't smile, offered
all his answers in the same emotionless tone, and wouldn't sit in the same room
as his fellow contestants. He had to work too hard at keeping his cool and was
so noisy, it was thought he was too disruptive to take the podium in person. He
was kept in a back room, his answers piped into the studio.
You wouldn't know by looking at him
what he was thinking - maybe you'd spot just a tinge of colour when he was
puzzling over a particularly hard question.
The contender started out with a run
of winning answers - he knew his Beatles songs, Olympic history, literary
criminals. Sure, he wasn't too familiar with his Harry Potter, but he stretched
out a lead nonetheless, leaving Rutter and Jennings trailing thousands of
dollars behind.
But questions on decades tripped him
up, and Rutter fought back, piling up enough cash to unsettle anyone who'd bet
on the outcome of the match. By the end of the first of the special exhibition
match shows, you'd have been hard pushed to work out which was safest with your
money.
But then Double Jeopardy happened.
The upstart powered through the big questions, winning even with guesses he was
far from convinced about, and placing odd bets that came good.
By the end of the second episode,
the unknown had $25,000 more than his closest opponent, Rutter. Rutter and
Jennings looked increasingly uncomfortable as it begun to look like they'd get
a pasting from the new boy, bobbing in frustration as their opponent buzzed in
before them time and time again.
"I, for one, welcome our new
computer overlords" Ken Jennings' response to losing to an exhibition
Jeopardy match to Watson
Jennings managed a late fightback in
the third episode, but the new opponent gradually clawed back enough
money to
make it a close run.
All three correctly answered the
last question 'William Wilkinson's 'An account of the principalities of
Wallachia and Moldavia' inspired this author's most famous novel' with 'who is
Bram Stoker?' but Jennings appended his response with: "I for one welcome
our new computer overlords".
He, and Rutter, had lost to Watson -
a room-sized beast of a machine made by IBM and named after the company's
founder Thomas J Watson.
Watson, consisting of ten racks of
ten Power 750 servers, had to be kept apart from the human contestants because
of the roar of its cooling system and was represented at the podium by an
avatar of IBM's Smarter Planet logo, whose moving lines would go green when
Watson had cracked a thorny problem, orange when the answer was wrong.
While Watson had the questions
delivered in text rather than by listening to the quizmaster, he played the
game like his human counterparts: puzzle over the question, buzz in, give the
answer that's most likely to be right, tot up some prize money.
And Watson was right a lot of the
time. He won the game with $77,147 leaving Rutter and Jennings in the dust with
$21,600 and $24,000 respectively.
It turned out that the real Jeopardy winning machine was, well, a machine.
Three nights, two people, one machine and $1 million: the victory of IBM's Watson over two human contestants on Jeopardy was the first, and possibly only, time the machine impressed itself on the general public's consciousness.
IBM Watson defeated two of
Jeopardy's greatest champions. Image: IBM
But even before Watson secured its
now-famous win, IBM was working on how to turn the cute quiz show-dominating
machine into a serious business contender.
Watson began life five years before
its TV appearance, when IBM Research execs were searching for the next
"Grand Challenge" for the company. IBM
periodically runs these Grand Challenges, selected projects that pit man
against machine, have international appeal, are easy to grasp and attract
people into working in science and maths fields. Along with Watson, the Grand
Challenges have spawned Deep Blue, the machine that famously beat grand master Garry Kasparov at chess, and
the Blue Gene supercomputer.
In the mid-2000s, IBM was on the
lookout for its next Grand Challenge. Paul Horn, then director of IBM Research,
was in favour of trying to develop a machine that could win the Turing Test, a
way to measure machine intelligence by having a system attempt to fool a human
into thinking that they're having a conversation with another person.
As challenging as passing the Turing
Test is - no machine has yet done it - it was felt that it wouldn't perhaps
light up the public's imagination as other projects had. But were there any
related challenges that could still bring those elements of competing against
humans and understanding human speech together?
"Beating a human in Jeopardy is
a step in that direction - the questions are complicated and nuanced, and it
takes a unique type of computer to have a chance of beating a human by answering
those type of questions. I was running the research division and I was bugging
people in the organisation, in particular [former EVP in IBM's software group]
Charles Lickel," Horn said.
Lickel was inspired to take on the
challenge of building a Jeopardy-winning computer after having dinner with his
team. "We were at a steak house in Fishtail, New York. In the middle of
dinner, all of a sudden the entire restaurant cleared out to the bar - I turned
to my team and asked 'what's going on?'. It was very odd. I hadn't really been
following Jeopardy, but it turned out it was when Ken Jennings was having his
long winning streak, and everyone wanted to find out if he would win again that
night, and they'd gone to the bar to see," Lickel said. Jennings won once
again that night, and still holds the longest unbeaten run on Jeopardy with 74
appearances undefeated.
"They initially said no, it's a
silly project to work on, it's too gimmicky, it's not a real computer science
test, and we probably can't do it anyway" IBM researchers' first take
on building a machine that could win Jeopardy
The idea of a quiz champion machine
didn't immediately win his team around, with many of Lickel's best staff saying
they didn't believe a machine could compete with, let alone beat, flesh and
blood champions.
"They initially said no, it's a
silly project to work on, it's too gimmicky, it's not a real computer science
test, and we probably can't do it anyway," said Horn.
Nonetheless, a team sufficiently
adventuresome to take on the challenge of building a Jeopardy winner was found.
It was still a small project and
thoughts of commercialisation weren't uppermost in anyone's mind - Grand
Challenges were demonstration projects, whose return for the company was more
in the buzz they created than in a contribution to the bottom line. If
commercialisation happened, great - but for now, Watson was just a bit of a
moonshot for IBM.
Due to the initial size of the
effort, it was funded from the research group's everyday budget and didn't
require sign-off from Big Blue's higher-ups, meaning it could operate free of
the commercial pressures of most projects.
Jeopardy's quirk is that instead of
the quizmaster setting questions and contestants providing the answer, the quizmaster provides the answers, known as 'clues' in Jeopardy-speak, to which
contestants provide a question. Not only would the machine need to be able to
produce questions for the possible clues that might come its way on Jeopardy,
it would need to be able to first pull apart Jeopardy's tricksy clues - work
out what was being asked - before it could even provide the right response.
Jeopardy host Alex Trebeck and
the IBM team talk about the exhibition match with Watson. Image: IBM
For that, IBM developed DeepQA, a
massively parallel software architecture that examined natural language content
in both the clues set by Jeopardy and in Watson's own stored data, along with
looking into the structured information it holds. The component-based system,
built on a series of pluggable components for searching and weighting
information, took about 20 researchers three years to reach a level where it
could tackle a quiz show performance and come out looking better than its human
opponents.
First up, DeepQA works out what the
question is asking, then works out some possible answers based on the
information it has to hand, creating a thread for each. Every thread uses
hundreds of algorithms to study the evidence, looking at factors including what
the information says, what type of information it is, its reliability, and how
likely it is to be relevant, then creating an individual weighting based on
what Watson has previously learned about how likely they are to be right. It
then generated a ranked list of answers, with evidence for each of its options.
The information that DeepQA would
eventually be able to query for Jeopardy was 200 million pages of information,
from a variety of sources. All the information had to be locally stored -
Watson wasn't allowed to connect to the Internet during the quiz - and
understood, queried and processed at a fair clip: in a Jeopardy's case, Watson
had to spit out an answers in a matter of seconds to make sure it was first to
the buzzer.
"When I left IBM in end of
2007, Watson was an embryonic project," said Horn. "It had three
people in Charles Lickel's area that got the data from the old Jeopardy
programmes and were starting to train the machine. It could barely beat a five
year old at that time. The projection was 'god knows how long it would take to
beat an adult, let alone a grand champion'. Then over time when it looked like
they started to have a chance, Dave under the leadership of John Kelly grew the
project into something substantial," said Horn.
While there's still debate over
exactly when the idea of making Watson pay its way finally took shape at IBM, when
Watson took to the stage for its Jeopardy-winning performance, the show
featured IBM execs talking about possible uses for the system in healthcare,
and moves to establish a Watson business unit began not long after the Jeopardy
show aired.
IBM's then-CEO Sam Palmisano and its
current CEO Ginni Rometty, under whose remit Watson fell at the time, began
discussions in the weeks after the win, and the project was moved from under
the wing of IBM Research and into the IBM Software group.
In August of 2011, the Watson
business unit proper came into being, headed up by Manoj Saxena, who'd joined
IBM some years earlier when the company he worked for, Webify, was acquired by
IBM.
Saxena was the unit's employee number
one. Within three months, he had been joined by 107 new Watson staffers, mostly
technologists in the fields of natural language processing and machine
learning.
Healthcare had already been
suggested as the first industry Watson should target for commercial offerings,
but there were no plans to confine it just to medicine. Any
information-intensive industry was fair game, anywhere were there were huge
volumes of unstructured and semi-structured data that Watson could ingest,
understand and process quicker than its human counterparts. Healthcare might be
a starting point, but banking, insurance, and telecoms were all in the firing
line.
But how do you turn a quiz show
winner into something more business-like? First job for the Watson team was to
get to grips with the machine they'd inherited from IBM Research, understand
the 41 separate subsystems that went into Watson, and work out what needed to
be fixed up before Watson could put on its suit and tie.
In the Watson unit's first year, the
system got sped up and slimmed down. "We serialised the threads and how
the software worked and drove up the performance," Saxena said. "The
system today compared to the Jeopardy system is approximately 240 percent
faster and it is one-sixteenth the size. The system that was the size of a
master bedroom will now run in a system the size of the vegetable drawer in
your double-drawer refrigerator."
Another way of looking at it: a
single Power 750 server, measuring nine inches high, 18 inches wide and 36
inches deep, and weighing in at around 100 pounds. Having got the system to a
more manageable size for businesses, it set about finding customers to take it
on.
IBM had healthcare pegged as its first vertical for
Watson from the time of the Jeopardy win. However, while Jeopardy Watson and
healthcare Watson share a common heritage, they're distinct entities: IBM
forked the Watson code for its commercial incarnation.
"The system that was the size
of a master bedroom will now run in a system the size of the vegetable drawer
in your double-drawer refrigerator" Watson VP Manoj Saxena on the
shrinking Watson
Jeopardy Watson had one task - get
an answer, understand it, and find the question that went with it. It was a
single user system - had three quizmasters put three answers to it, it would
have thrown the machine into a spin. Watson had to be retooled for a scenario
where tens, hundreds, however many, clinicians would be asking questions at
once, and not single questions either - complex conversation with several
related queries one after the other, all asked in non-standard formats. And, of
course, there was the English language itself with all its messy complexity.
"There were fundamental areas
of innovation that had to be done to go beyond Jeopardy - there was a
tremendous amount of pre-processing, post-processing and tooling that we have
added around the core engines," added Saxena. "It's the equivalent of
getting a Ferrari engine then trying to build a whole race car around it. What
we inherited was the core engine, and we said 'Okay, let's build a new thing
that does all sort of things the original Jeopardy system wasn't required to
do'."
To get Watson from Jeopardy to oncology,
there were three processes that the Watson team went through: content
adaptation, training adaptation, and functional adaptation - or, to put it
another way, feeding it medical information and having it weighted
appropriately; testing it out with some practice questions; then making any
technical adjustments needed - tweaking taxonomies, for example.
The content adaptation for
healthcare followed the same path as getting Watson up to speed for the quiz
show: feed it information, show it what right looks like, then let it guess
what right looks like and correct it if it's wrong. In Jeopardy, that meant
feeding it with thousands of question and answer pairs from the show, and then
demonstrating what a right response looked like. Then it was given just the
answers, and asked to come up with the questions. When it went wrong, it was
corrected. Through machine learning, it would begin to get a handle on this
answer-question thing, and modify its algorithms accordingly.
"It would be fed many cases
where the history was known and proper treatment was known, and then, analogous
to training for Jeopardy, it's been given cases and then it suggests
therapies," Kohn said.
Some data came from what IBM
describes as a Jeopardy-like game called Doctor's Dilemma, whose questions
include 'the syndrome characterized by joint pain, abdominal pain, palpable
purpura, and a nephritic sediment?'. (The answer, of course, is
Henoch-Schönlein purpura.)
The training, says Kohn, "is an
ongoing process, and Watson is rapidly improving its ability to make reasonable
recommendations the oncologists think are helpful."
By 2012, there were two healthcare
organisations that had started piloting Watson.
Wellpoint, one of the US biggest
insurers, was one of the pair of companies that helped define the application
of Watson in health. The other was Memorial Sloan-Kettering Cancer Center
(MSKCC), an organisation IBM already had a relationship with and which is
located not far from both IBM's own Armonk headquarters and the research
laboratories in York Heights, New York that still house the first Watson.
And it was this relationship that
helped spur Watson's first commercial move into working in the field of cancer
therapies. While using Watson as a diagnosis tool might be its most obvious
application in healthcare, using it to assist in choosing the right therapy for
a cancer patient made even more sense. MSKCC was a tertiary referral centre -
by the time patients arrived, they already had their diagnosis.
So Watson was destined first to be
an oncologist's assistant, digesting reams of data - MSKCC's own, medical
journals, articles, patients notes and more - along with patients' preferences
to come up with suggestions for treatment options. Each would be weighted
accordingly, depending on how relevant Watson calculated they were.
Unlike its Jeopardy counterpart,
healthcare Watson also has the ability to go online - not all its data has to
be stored. And while Watson had two million pages of medical data from 600,000
sources to swallow, it could still make use of the general knowledge garnered
for Jeopardy - details from Wikipedia, for example. (What it doesn't use,
however, is the Urban Dictionary. Fed into Watson late last year, it was reportedly removed after answering a researcher's query with
the word "bullshit". "We did find some interesting
responses, so we had to shut that down," Saxena said diplomatically.
"That is not to be repeated, because it would be seen as very improper in
certain cases, and we had to teach Watson the right business behaviour.")
This chart, done a year after
Watson's Jeopardy win, shows some of its rapid progress. Image: IBM
As such, the sources are now medical
publications like Nature and the British Medical Journal. And there are other
safety nets too.
"In the teaching phase, a
doctor - a cancer care specialist in this case - sits down and asks questions
of Watson and corrects the machine learning. The doctor and a data scientist
are sitting next to each other, correcting Watson. Spurious material, or
conflicted material or something from a pharmaceutical company that the doctor
feels may be biased - that is caught during the training cycle," added
Saxena.
WellPoint and MSKCC used Watson as
the basis for systems that could read and understand volumes of medical
literature and other information - patients' treatment and family histories,
for example, as well as clinical trials and articles in medical journals - to
assist oncologists by recommending courses of treatment.
A year of working with both
organisations has produced commercial products: Interactive
Care Insights for Oncology, and the WellPoint Interactive Care Guide and
Interactive Care Reviewer. Interactive Care Insights for Oncology provides
suggestions for treatment plans for lung cancer patients, while New WellPoint
Interactive Care Guide and Interactive Care Reviewer reviews clinicians'
suggested treatments against their patients' plans and is expected to be in use
at 1,600 healthcare providers this year.
Watson has bigger ambitions than a
clinician's assistant, however. Its medical knowledge is around that of a first
year medical student, according to IBM, and the company hopes to have Watson
pass the general medical licensing board exams in the not too distant future.
"Our work today is in the very
early stages around practice of medicine, around chronic care diseases. We're
starting with cancer and we will soon add diabetes, cardiology, mental health,
other chronic diseases. And then our work is on the payment side, where we are
streamlining the authorisation and approval process between hospitals, clinics
and insurance companies," Saxena said.
The ultimate aim for Watson is to be
an aid to diagnosis - rather than just suggesting treatments for cancer, as it
does today, it could assist doctors in identifying the diseases that bring
people to the clinics in the first place.
Before then, there is work to be
done. While big data vendors often trumpet the growth of unstructured data and
the abandoning of relational databases, for Watson, it's these older sources of
data that present more of a problem.
"Watson works specifically with
natural language - free text or text-like information - and that's
approximately 80 percent of the huge volumes of healthcare information
available to us," said Kohn. "Then there's the 20 percent that is
structured data - basically, numerical data - or images, like MRIs, CAT scans,
and so on. Watson does not process structured data directly and it doesn't
interpret images. It can interpret the report attached to an image, but not the
image itself."
In addition, IBM is working on
creating a broader healthcare offering that will take it beyond its oncology
roots.
"Even though Watson is working
with these two organisations, what the designers and computer scientists are
focusing on [is] that whatever they develop is generalisable, it's not just niche
for cancer therapy and especially for the several cancers we're working with.
We're using it as a learning process to create algorithms and methodologies
that would be readily generalisable to any area of healthcare. They don't have
to have to say, right, we have oncology under control, now let's start again
with family practice or cardiology," Kohn said.
Citi and IBM have been
collaborating on business systems since the days of the early mainframes.
Image: IBM
Watson has also already found some
interest in banking. Citi is using Watson to improve customer experience with the bank and
create new services. It's easy to see how Watson could be put to use, say,
deciding whether a borderline-risk business customer is likely to repay the
loan they've applied for, or used to pick out cases of fraud or identity theft
before customers may be aware they're happening.
Citi is still early in its Watson
experiments. A spokeswoman said the company is currently just "exploring
use cases".
From here on in, rather than being
standalone products, the next Watson offerings to hit the market will be
embedded into products in the IBM Smarter Planet product line. They're expected
to appear in the second half of the year.
The first such Smarter Planet
product appeared in May: IBM Engagement Advisor. The idea behind the
Engagement Advisor, aimed at contact centres, is that customer service agents
can query their employers' databases and other information sources using
natural language while they're conducting helpline conversations with their
clients. One of the companies testing out the service is Australia's ANZ bank,
where it will be assisting call centre staff with making financial services
recommendations to people who ring up.
Ask Watson goes far beyond
what Apple's Siri can do, IBM believes. Image: IBM
Watson could presumably one day
scour available evidence for the best time to find someone able to talk and
decide the communication channel most likely to generate a positive response,
or pore over social media for disgruntled customers and provide answers to
their problems in natural language.
There are also plans to change how
Watson's delivered, too. Instead of just interacting with it via a call centre
worker, customers will soon be able to get to grips with the Engagement
Advisor. Rather than have some call centre agent read out Watson generated
information to a customer with, say, a fault with their new washing machine or
a stock-trader wanting advice on updating their portfolio, the consumer and
trader could just quiz Watson directly from their phone or tablet, by typing
their query straight into a business' app. Apps with Watson under the hood
should be out in the latter half of this year, according to Forbes.
IBM execs have also previously
suggested that Watson could end up a supercharged version of Siri,
where people will be able to speak directly into their phone and pose a complex
question for Watson to answer - a farmer holding up his smartphone to take
video of his fields, and asking Watson when to plant corn, for example.
IBM is keen to spell out the
differences between Watson and Siri. "Watson knows what it knows - and by
listening, learning and using human-like thinking capabilities uncovers
insights from Big Data, Watson also quickly ascertains what it doesn't know.
Siri, on the other hand, simply looks for keywords to search the web for lists
of options that it chooses one from," the company says. But, the comparison
holds: Watson could certainly have a future as your infinitely knowledgeable
personal assistant.
"Watson also quickly ascertains
what it doesn't know. Siri, on the other hand, simply looks for keywords to
search the web for lists of options that it chooses one from" IBM on
Watson vs Siri
While adding voice-recognition
capabilities to Watson should be no great shakes for IBM given its existing
partnerships, such a move would require Watson to be able to recognise images
(something IBM's already working on) that would require Watson to query all
sorts of sources of information including newspapers, books, photos,
repositories of data that have been made publicly available, social media and
the internet at large. That Watson should take on such a role in the coming
years, especially if the processing goes on in an IBM datacentre and not on the
mobile itself, as you would expect, is certainly within the realms of the
possible.
As IBM seeks to embed Watson's
capabilities into more and more products, how far does the company think Watson
will spread in the coming years? It will only say gnomically, "as we
continue to scale our capabilities, we intend to make Watson available as a set
of services in many industries." Want a better answer? Better ask Watson.
It was so amazing to find out that a robot can do such many things, especially this so-called watson. Sir Tom introduces this to us. we were amazed upon watching watson's video.
It was so amazing to find out that a robot can do such many things, especially this so-called watson. Sir Tom introduces this to us. we were amazed upon watching watson's video.
No comments:
Post a Comment