Human or Not, An AI Game That Challenges Human Intelligence

Do you love to play challenging games? If yes, then here is one for you. ‘Human or Not’, is an AI-based game that asks you to spot the difference between a human and an AI bot. The game results revealed that more than 30 percent of the players failed the challenge.

AI Is Everywhere

Everyone is familiar with Artificial Intelligence, commonly known as AI. People who are not familiar with it can understand it as a computer science field that develops computer systems requiring human intelligence to perform tasks. Though AI is trending these days, it has not taken over the world completely. However, it has a wide range of applications across various industries.

AI is used in healthcare, gaming, finance, cybersecurity, transportation and many other sectors. Not only this, but it is also used in image and voice recognition. Also, AI-powered chatbots and virtual assistants have become quite good at responding to every type of input and providing the desired output for queries. Recently, AI has been used in developing a game based on the classical Turing test to determine whether a machine can act as a human and fool the person playing the game.

The Turing Test

The Turing test, also known as the imitation game, is a test that checks the machine’s ability to show human-like intelligence and responses. It is test created by Alan M. Turing, an English mathematician and computer scientist, in 1950. The test consists of three terminals out of which one is physically separated from the other two.

One terminal is operated by the computer, while the other two are operated by humans. During the test, one of the humans acts as a questioner, and the other human and the computer act as respondents. The questioner asks various questions related to a specific subject using a given format for a particular period. The answers are not judged by their accuracy but on how close they are to the human response. If the questioner is unable to detect which responses came from the computer and which ones from the human, then the machine passes the test and is said to have human intelligence. 

Human or Not? The AI Game

AI21 Labs, an AI lab and product company, created an online game known as Human or Not which is based on the Turing Test concept. The game was launched around a month ago. It allows you to chat with someone for two minutes. During the game, you can ask any question, advice or discuss any topic. You get 20 seconds to write your responses. After two minutes of interaction, the game asks you to guess whether you talked to an AI bot or a human.

The game can helps users get more used to the world of AI bots

To make the game more challenging, AI21 Labs used different Large Language Models (LLMs) (deep learning AI models that can mimic human intelligence), including GPT-4, Cohere, Claude, and its own model Jurassic-2. These LLMs help AI tools and chatbots generate human-like responses. Besides this, the company also created its own framework that would allow the bot character to act differently in each game.

The game developers recorded millions of conversations to check how many people were able to pass the challenge. They collected the data and found that 68 percent of the players detected the difference between the human and AI bot, while the other 32 percent failed the challenge.

Overall, the AI21 Labs’ analysis showed that 73 percent of the players guessed right when chatting with a human. On the other hand, 60 percent of players got the right answer when chatting with the AI bot. The company also analysed the response data across different countries. Out of 17 countries, France had the highest percentage of correct guesses which was 71 percent. India scored the least at 63.5 percent. 

Players tried a few tricks to distinguish between the human and the bot. For instance, they started using slang words and made grammatical errors while chatting. But these tricks didn’t always work. With AI, the models in the game were trained to use slang words and non-grammatical sentences to confuse the players.

Then, participants tried to talk about current affairs. However, the bots were able to answer as the models were connected to the internet and knew about these events. Next, the players started to talk about their personal life and illegal activities. As the models were developed according to the ethical sub-routines, the bots were unable to answer. This trick worked and participants were able to differentiate the bot from the human. 

AI21 Labs will study the findings and data in more detail and work with other renowned AI labs and researchers on this game project. The company aims to help policymakers, researchers and the public understand AI and see it as a productive friend in the online world.

“We started this experiment with some basic questions about people’s ability to distinguish between humans and machines, given the crazy development of AI in the past year or so, and we found some answers for that,” said Amos Meron, the designer of the game and the creative product lead at AI21 Labs.

 “More importantly, though, is that we now have new and more important questions we need to think about,” added Meron. “Given that, at least in some cases people can’t tell the difference, what interactions do people want and should experience online with bots? Should they be informed about the fact they’re talking to a machine? What policies should we have in place? Of course, we don’t have the answers to these questions, but we hope this experiment helped start the conversation earlier rather than later because we assume the technology is only going to get even better soon.”

Try out the game ‘Human or Not‘ and let us know in the comments whether you guessed it right or not.

 If you enjoyed reading our articles, please consider supporting us by buying our geeky merchandise on Instagram.

Alternatively, you could buy us a coffee or subscribe to our newsletter.