Understanding Human Vision

Professor John Tsotsos

Professor John Tsotsos

Professor John Tsotsos (Canada Research Chair, Computational Vision) was recently interviewed by CTV news. Listen to him (click on the image) discuss how his research has disproved the highly influential 1958 theory of Donald Broadbent, early selection theory, that the human brain selects interesting portions of an image to process preferentially. The modern counterpart of this is the Saliency Map Theory of Christof Koch and Shimon Ullman published in 1985, which claims that the interesting regions are processed by the brain one at a time, in order of their salience, a numerical score of how interesting a region is.

Tsotsos’ team found however, that salience is not needed at all for the simple task of quickly deciding what an image depicts. Moreover, none of the current algorithms within artificial intelligence (AI) for this task come close to matching human performance, which is remarkably good. On the other hand, salience computation does play a primary role in determining where humans move their eyes, and it is eye movement that selects portions of a scene to process next.

“Our study looks at this for vision and tests the leading algorithms that compute the saliency measure and asks the question ‘are those algorithms performing at the same level as humans do on these images’? For example, if the task is to determine if there is a cat in a scene, does the saliency algorithm pick out the cat correctly? The study showed that these algorithms are far from doing as well as humans,” said Tsotsos.

Tsotsos says this finding has important ramifications for our understanding of human vision and human visual processing especially for diagnosing vision pathologies, such as aspects of autism.

“When you want to diagnose issues in vision, you’re basing on it how the healthy visual processing system should work. What we’ve done with this study is added a piece of the puzzle to how the ‘healthy’ system works which then would change how you compare an anomaly in order to be able to diagnose it.”

Related News