Top “Gaokao" Scorers Outperformed AI Robots In Round One Math Exam
摘要： After Google’s AlphaGo beat Go world champion Ke Jie 3:0, many people begin to worry where AI will defeat human intelligence next. However, in the recent contests with some top “gaokao” scorers, AI robots were defeated. How come?
After Google’s AlphaGo beat Go world champion Ke Jie 3:0, many people begin to worry where AI will defeat human intelligence next.
However, in the recent competition between AI learning robots and some top “gaokao” (China’s college entrance examination) scorers, human intelligence outperformed AI.
After the annual math “gaokao” ended yesterday afternoon, another exam started between the AI test taker and some of the top scorers in Beijing. Aidam, an AI smart learning robot designed by Haster Learner Technology, appeared in public for the first time and joined a contest with six top liberal arts “gaokao” scorers to finish a standard “gaokao” math test, both subjective and objective questions included. After they finished the exam, both the AI robot and six top scorers were rated and evaluated based on the standard assessment criteria.
To everyone’s amazement, however, the AI robot was defeated. While the three groups of top scorers scored 146, 140 and 119, respectively, Aidam scored 134 out of 150.
AI suffered another defeat in the Chengdu contest. AI-MATHS, a smart math “gaokao” robot developed by a company in Chengdu, Sichuan province under the Human Brain Project of National 863 Project also took a standard math “gaokao” test and finished the Beijing College Entrance Math Exam for Liberal Arts Students with 22 minutes with the internet being shut down.
AI-MATHS scored 105, with a full score was 150. Later, AI-MATHS finished the National College Entrance Math Exam II with 10 minutes and scored 100 out of 150.
To be more specific, AI-MATHS is an AI “gaokao” robot program officially initiated by The Ministry of Science and Technology under the Human Brain Project of National 863 Project on July 21st, 2015.
After the contest, Duo Ligang, the top scorer of liberal arts in the 2015 Xinjiang “gaokao” (his group scored 119) was amazed at Aidam’s score, saying that “the result was beyond our expectation and we did make some mistakes in some of our calculation”.
For sure, Aidam didn’t outperform humans completely when compared with the highest human scorer. However, it was still quite amazing to score 134 within 10 minutes, something unimaginable for most students.
Chen Ruifeng, Chief Scientist at Haster Learner, explained how Aidam worked as follows:
Above all, Aidam needs to interpret the test questions into formal language;
Secondly, Aidam will search across the knowledge network and identify the related knowledge points. After a series of reasoning, Aidam will find the best solution;
Thirdly, Aidam will translate the formal language into natural language and fill in the answer.
This won’t be possible without a gigantic database and years of technological accumulation. According to Haster Learner’s public data, .it has already accumulated 70 million users as of May 2015 and solved over 10 billion questions with an accuracy rate of 93 per cent.
But why math, not other subjects?
“There might be one thousand Hamlet for one thousand readers, but only one Pythagoras' theorem around the world,” Zhang Kailei, the founder and CEO of Haster Learner, explained.
For one thing, logic and reasoning are mainly tested in subjects such as math and physics and it is easier to tell the right and wrong; for another, there are both simple Multiple Choice questions and complicated Answer Questions in math tests where a lot of reasoning are involved. Therefore, math tests are indeed one of the best “battlegrounds” to test AI.
For robots, however, the first step poses the biggest challenge, especially regarding Application Questions, since AI robots have to be able to understand the question first and then solve it.
For insiders, having AI robots sit “gaokao” exam is not the fundamental goal. Instead, it’s more like a simulation and extension of human intelligence. At present, “gaokao” is an effective tool for evaluating people’s knowledge level, comprehension and reasoning ability.
“Our initial purpose is to achieve personalized learning and significantly improve learning efficiency and result,” Zhang told TMTPost in the joint interview.
“There are around 3,529 test points during middle school period, and a student only needs to do three to four questions to grasp each test point. Therefore, one only needs to finish around 10,000 questions to grasp all the test points. However, most students have to face 30,000 to 40,000 exercises, though 3/4 of them are unnecessary. In other words, 3/4 of students’ time is wasted,” Zhang added.
Based on a deep learning network and a semantic analyzer, the program shall continuously strengthen itself through constant training. While both a knowledge map and common sense are needed to solve most Application Questions, Haster Learner improves its proximity to the real world based on a lifelike simulator of questions.
Besides, Haster Learner is actively promoting the development of AI technologies, such as image recognition, natural language interpretation and deep learning, etc. and integrating them into a smart learning robot. Haster Learner makes it possible to grade test papers, carry out personalized tutoring and perfect teaching plans under the current educational background.
Nevertheless, Chen Ruifeng, Chief Scientist of Haster Learner’s, didn't think that AI can replace teachers. No matter how, robots can not replace teachers and teach students how to solve questions. For example, robots can not explain why it’s wrong to solve a question this way or that way, what’s wrong with this or that way of thinking and where different ways of thinking should be adopted. However, AI can help teachers identify whether a student has grasped a knowledge point or not, and whether it will be a total waste of time to continue do exercises over a knowledge point.
Still, it is worth noticing that all the major online education platforms featuring test assistant tools and question databases in China haven’t found a profit point yet. Zhang also revealed to TMTPos that it hadn’t achieved profitability yet. Nevertheless, it has started to scale.
We also learn that the AI robot will sit the “gaokao” exam again in 2019 and 2020. At the same time, iFLYTEK, a key Chinese software enterprise dedicated to the research of intelligent speech and language technologies and the leading enterprise in the “Big Data-based Human-Like Intelligence Technology and System Project” of the 12th Five-Year Plan, is also busy developing an AI robot to sit the Chinese, Geography and History exam in the 2019 Standard College Entrance Exam.
[The article is published and edited with authorization from the author @Li Chengcheng please note source and hyperlink when reproduce.]
Translated by Levin Feng (Senior Translator at PAGE TO PAGE), working for TMTpost.