Which objective tests have been developed to test the intelligence of AI?



Which objective and measurable tests have been developed to test the intelligence of AI?

The classical test is the Turing Test, which has objective criteria and is measurable since it can be measured what percentage of the jury is fooled by the AI.

I am looking for other, more modern tests.


Posted 2016-08-03T13:01:20.740

Reputation: 1 401


I voted to close this question since there are many different possible ways to develop tests with objective criteria and measurability, and it'd be hard to mention them all exhaustively. I would recommend reading this article about a proposed plan to replace the Turing Test with an "Olympics" and this blog post about the Lovelace Tests...

– Left SE On 10_6_19 – 2016-08-04T03:35:26.833

1...for a good sense of how such 'objective' and measurable tests are designed in the modern day. But just because these tests can be objective doesn't mean that the results can be valid or useful to have. After all, even the Turing Test itself can be arguably beaten, which is why we have proposals for alternatives to the Turing Test. – Left SE On 10_6_19 – 2016-08-04T03:36:38.770



For there to be an objective test for AI intelligence, there has to be an objective test for human intelligence. Each of the standard testing organizations claim theirs to be the most objective, but none of the test creators publish rigorous systems theory to verifiably justify their claim.

The user stories for human selections are varied.

  • I want to find people to work on my team so the project is completed on time and in budget.
  • I need to admit the right people for our training so we can have success stories to tell.
  • We need a focus group with people who can communicate well in groups and think about our products and their market.

With artificial intelligence the bar is much lower.

  • We need to sort grapefruits for canning, fruit stand sale, use as a sweetener, or use as composting matter in a way to maximize profit.
  • We need to beat the other automated drone pilot companies.
  • Most of the world has online business but their documents are redundant with many nearly identical documents from copying and pasting, and most of them are poorly organized. We need an automated librarian that organizes and indexes them.

The questioner may have hoped for some general intelligence test for AI better than Turing's imitation game, the current standardized intelligence testing only works on humans. The AI that exists can't, without specifically training for the standardized tests they are going to take, achieve anything like a normal human score. Cognition is something AI hopes to achieve.

Here is some work in that area.


Posted 2016-08-03T13:01:20.740

Reputation: 356