What is the nature of representations in visual cortex?
Characterizing what the "neurons are doing" in visual cortex is a central guiding problem of systems visual neuroscience. We approach this problem by the following process:
Two of our on-going projects in this direction include:
Characterizing what the "neurons are doing" in visual cortex is a central guiding problem of systems visual neuroscience. We approach this problem by the following process:
- taking models of the neural encoding in higher visual cortical areas (see here)
- analyzing the models to see what the model's "internals" are doing
- checking these observations against neural data.
Two of our on-going projects in this direction include:
What Visual Functions are Encoded in Inferior Temporal Cortex?
Once the brain has formatted visual information effectively, what types of decisions made using this information?
Our recent work, both computational and experimental, suggests that Inferior Temporal (IT) cortex is much more than just about object recognition. In fact, it may contain a compact general-purpose visual representation from which many distinct visual properties of images, including object position, size, and pose, can easily be decoded. The same observations also suggests that properties usually considered "low-level", such as position, may not be as low-level as originally thought, prompting us to rethink the types of information aggregation and compression mechanisms operating in the ventral stream.
Once the brain has formatted visual information effectively, what types of decisions made using this information?
Our recent work, both computational and experimental, suggests that Inferior Temporal (IT) cortex is much more than just about object recognition. In fact, it may contain a compact general-purpose visual representation from which many distinct visual properties of images, including object position, size, and pose, can easily be decoded. The same observations also suggests that properties usually considered "low-level", such as position, may not be as low-level as originally thought, prompting us to rethink the types of information aggregation and compression mechanisms operating in the ventral stream.
What is the dimensionality and capacity of the human visual object representation? Many visual psychophysics experiments hypothesize a perceptual space whose axes encode key features on which judgements are made. In this project, we developed a method for back-inferring the perceptual representation from large-scale behavior data. We then used this method to characterize human object recognition perceptual space for an image set containing many tens of thousands of images in approximately one hundred semantic categories, with data collected using online psychophysics involving nearly a thousand subjects. Our results suggest that the human visual object representation is consistent with a 20-dimensional space of representational features, with the capacity for representing approximately 10,000 distinct object categories. Given the scale and precision of the dataset, we were also able to make direct comparisons to neural data, finding that the object layout in the inferred human perceptual space correlates highly with those from the neural population representation measured in Inferior Temporal (IT) cortex.