IBM’s new AI supercomputer has been shown to take on humans in a debate and, according to its creators, ‘win’.
Named Project Debater, the ‘unsettling’ AI was pitted against two humans in the first public demonstration of its ability at an event in San Francisco.
It competed in what is known as computational argumentation, where debaters know a subject, present a position and defend it against opposition.
The computer delivered its opening argument by pulling in evidence from its huge database, mainly made up from news articles and journals.
It then listened to a professional human debater’s counter-argument and spent four minutes rebutting it.
Although the crowd agreed the human beings were unmatched in delivery, it decided the machine made better arguments with more substance.
But a study of its responses has revealed many of the responses were plagiarised with full sentences lifted from well-known sources. raising the question of how intelligent the AI really is.
IBM Project Debater gathers information from millions of newspaper and magazine articles saved in its database to engage in long-form discussion about a given topic
IBM’s AI system is not connected to the internet, but instead uses a stash of hundreds of millions of newspaper and magazine articles on ‘about 100 areas of knowledge’ which the AI is able to draw upon to build its argument.
According to IBM Principle Investigator for Project Debator Dr Noam Slonim, the computer is not designed to simply regurgitate arguments from existing pieces.
Speaking on BBC Radio 4’s Today programme, Dr Slonim said: ‘It is not copying a whole article, it is not even copying a whole paragraph either, it is picking single sentences – and in some cases just clauses – taken from many different articles, then it glues that together into a coherent, persuasive narrative.’
To make sure the resulting debate is compelling to watch, IBM also programmed the AI to add jokes into its narrative to keep the crowd entertained.
During its rebuttal, the machine is also designed to suggest its opponent was lying during their argument.
This is a technique used by humans to distract when they do not have a strong argument to rely upon, something the IBM team wanted to replicate with the machine.
The AI-powered machine was required to make a four-minute introductory speech, a four-minute rebuttal to the experienced human debater’s arguments, and finish with a two-minute closing statement on the topic
During the first showcase of the technology, two debates were presented on-stage – whether government subsidise space exploration and if the use of telemedicine should be increased.
IBM Project Debater was pitted against human debaters Noa Ovadia and Dan Zafrir, who each took the reigns for one of the debates.
In both cases, the audience judged that the machine outperformed in its ability to present a wider body of information in its argument.
However, the human beings were universally judged to be better at delivery in their on-stage speeches.
In the end, IBM Debater could not convince the audience to side with its argument in the debate on whether space exploration should be subsidised.
But the machine won the second debate, convincing more audience members with its arguments on telemedicine usage than its human opponent.
IBM has not taught its AI to judge the reliability of the information it draws upon, something that critics believe could lead to the machine using biased sources to successfully argue its point.
Brhmie Balaram, who works as a senior researcher at the Royal Society for the encouragement of Arts, Manufactures and Commerce (RSA) told the Today programme that there were ‘ethical concerns’ with this approach to an AI debate.
‘It’s about considering what sort of information it is analysing.
‘There’s a lot of people who are concerned the data that is going into these machines might be biased, and therefore, there are ethical concerns. But also, where is it getting this data from? If it is about healthcare, it’s raising concerns about privacy.’
However, the team behind IBM Debater believe the ability for the machine to argue both sides of the debate means it can never be seen as biased.
‘The computer is certainly less biased than humans,’ argued Dr Slonim.
‘The computer can argue in favour of both positions, so this is one important point.
‘The second is that, [the computer] can help us to take more informed decisions. Let’s say that we are debating or not to legalise cannabis — a topic that I thought about recently.
‘If we’re debating this topic, obviously we need the technology that will be able to quickly suggest to us: what are the relevant claims that people are making with respect to this topic, what is the relevant evidence that people are suggesting in respect to these claims?’
IBM hopes the project will eventually be used to help present an evidence-based argument on a topic that removes any bias, emotion, or ambiguity.
These arguments can be used to help humans make difficult decisions when there is no black-or-white answer.
Like the human participants, the machine did not have any prior knowledge about what the debate was about. Pictured is Noam Slonim, an IBM researcher
During the live on-stage debate about space, the machine said space exploration was beneficial to the economy. Pictured is Hayah Eichler, a professional debater who has previously debated with IBM Project Debater
Brhmie Balaram of the RSA suggested IBM Project Debater could be deployed to help make critical decisions in sectors including healthcare, criminal justice, and hiring and firing in the workplace.
Should this happen, Balaram believes the public will be frightened by the lack of human input.
‘What humans want is another human in the loop,’ she told BBC Radio 4.
‘They don’t just want the machine to make a decision of a prediction on its own, they want a human to be there to be able to take that into consideration as part of a number of other factors that they use to make a decision.’
Project Debater is still a research project, however, the company claims that some of the technology which underpins the new system has already started to seep into other IBM projects.
IBM is no stranger to grand, public displays of what it claims is superior artificial intelligence.
The Armonk-based computer debuted its Watson supercomputer by trouncing human contestants on the US television gameshow Jeopardy, back in 2011. Before that, the company’s Deep Blue system beat world chess champion Garry Kasparov.
HOW DOES ARTIFICIAL INTELLIGENCE LEARN?
AI systems rely on artificial neural networks (ANNs), which try to simulate the way the brain works in order to learn.
ANNs can be trained to recognise patterns in information – including speech, text data, or visual images – and are the basis for a large number of the developments in AI over recent years.
Conventional AI uses input to ‘teach’ an algorithm about a particular subject by feeding it massive amounts of information.
AI systems rely on artificial neural networks (ANNs), which try to simulate the way the brain works in order to learn. ANNs can be trained to recognise patterns in information – including speech, text data, or visual images
Practical applications include Google’s language translation services, Facebook’s facial recognition software and Snapchat’s image altering live filters.
The process of inputting this data can be extremely time consuming, and is limited to one type of knowledge.
A new breed of ANNs called Adversarial Neural Networks pits the wits of two AI bots against each other, which allows them to learn from each other.
This approach is designed to speed up the process of learning, as well as refining the output created by AI systems.