Researchers at Department of Computational and Data Sciences at Indian Institute of Science (IISc) have developed the first computational model aimed at Pictionary- an image based word guessing game.
In 1997, Deep Blue, a chess-playing computer developed by IBM, beat the then world chess champion, Gary Kasparov, to become the first machine to defeat a human in Chess. Very soon after, games like Double jeopardy, GO!, and online poker followed, with computers beating every challenge thrown at them, in these games. With tools like deep learning and advances in artificial intelligence, today computers are able to compare, and in some cases even surpass, human intelligence, especially at playing games.
Pictionary is a word guessing parlour game, usually played in teams. Team members try and guess a word from a set of images, drawn by another team member. Unlike chess and Go, wherein players follow a set of concrete rules, pictionary is guessing game, based on an image drawn by another human. Often as the game carries on the images also evolve, with several images forming the clue. The team guessing has to take cues from the different images to arrive at the correct word.
Computers today use programs like Visual Question Answering (VQA) to make sense of what they see around them, by answering a set image based questions. VQA is also considered a marker for gauging progress in computer vision.
For their study, the researchers brought VQA and games together, to develop a model that could play pictionary. The researchers used Sketch Question Answering (Sketch-QA) a pictionary style guessing task, where human guessers try and guess an object based on hand-drawn images. The images are revealed to the guessers, one stroke at a time, with the humans being prompted to guess the object after each new stroke.
“We have introduced a novel guessing task called SketchQA to crowd-source Pictionary-style open-ended guesses for object line sketches as they are drawn. The resulting dataset, dubbed GUESSWORD-160, contains 16624 guess sequences of 1108 subjects across 160 object categories” explain the researchers about their work.
With the dataset in place, the researchers came up with models for generating guesses based on hand-drawn images. Using machine learning the computers ‘learn’ about the different decisions and associations made by humans. Over time the computer model can start imitating the human decision making, by guessing the same way as we do, and in some cases, making the same mistakes as humans.
“Experimental results demonstrate the promise of our approach for Pictionary and similarly themed games. In future, we wish to also explore computational models for optimal guessing, i.e. models which aim to guess the sketch category as early and as correctly as possible.” conclude the researchers regarding the future direction of this work.