Human visual cortex and deep convolutional neural networks care deeply about object background
A: We embedded objects in differing amount of background complexities - segmented (none), low, medium and high complexity; B: Deeper models resemble human performance, shallow networks show worse performance with more complex backgrounds; C: All models represent image backgrounds in their intermediate layers; but the better performing models become background invariant (i.e. discards background represenation)Abstract
Deep convolutional neural networks (DCNNs) are able to partially predict brain activity during object categorization tasks, but factors contributing to this predictive power are not fully understood. Our study aimed to investigate the factors contributing to the predictive power of DCNNs in object categorization tasks.
We compared the activity of four DCNN architectures with electroencephalography (EEG) recordings obtained from 62 human subjects during an object categorization task. Previous physiological studies on object categorization have highlighted the importance of figure-ground segregation - the ability to distinguish objects from their backgrounds. Therefore, we investigated whether figure-ground segregation could explain the predictive power of DCNNs.
Using a stimulus set consisting of identical target objects embedded in different backgrounds, we examined the influence of object background versus object category within both EEG and DCNN activity. Crucially, the recombination of naturalistic objects and experimentally-controlled backgrounds creates a challenging and naturalistic task, while retaining experimental control.
Our results showed that early EEG activity (<100ms) and early DCNN layers represent object background rather than object category. We also found that the ability of DCNNs to predict EEG activity is primarily influenced by how both systems process object backgrounds, rather than object categories. We demonstrated the role of figure-ground segregation as a potential prerequisite for recognition of object features, by contrasting the activations of trained and untrained (i.e. random weights) DCNNs.
These findings suggest that both human visual cortex and DCNNs prioritize the segregation of object backgrounds and target objects in order to perform object categorization. Altogether, our study provides new insights into the mechanisms underlying object categorization as we demonstrated that both human visual cortex and DCNNs care deeply about object background.
Jessica Loke
Doctoral Researcher in Computational Neuroscience
My research interests include visual perception and cognition in humans and artificial neural networks networks.