Top

Journal of Computational Neuroscience

Published in:

05-06-2018

Convolutional neural network models of V1 responses to complex patterns

Authors: Yimeng Zhang, Tai Sing Lee, Ming Li, Fang Liu, Shiming Tang

Published in: Journal of Computational Neuroscience | Issue 1/2019

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In this study, we evaluated the convolutional neural network (CNN) method for modeling V1 neurons of awake macaque monkeys in response to a large set of complex pattern stimuli. CNN models outperformed all the other baseline models, such as Gabor-based standard models for V1 cells and various variants of generalized linear models. We then systematically dissected different components of the CNN and found two key factors that made CNNs outperform other models: thresholding nonlinearity and convolution. In addition, we fitted our data using a pre-trained deep CNN via transfer learning. The deep CNN’s higher layers, which encode more complex patterns, outperformed lower ones, and this result was consistent with our earlier work on the complexity of V1 neural code. Our study systematically evaluates the relative merits of different CNN components in the context of V1 neuron modeling.

previous article Stability of point process spiking neuron models

next article Motor imagery and mental fatigue: inter-relationship and EEG based estimation

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

The images were rescaled to 2/3 of their original sizes; we used this scale because in another study (Zhang et al. 2016) we found that this scale gave the highest representational similarity (Kriegeskorte et al. 2008) between the CNN and neural data among all scales explored; we also tried using raw images without rescaling in the current study and got worse results.

In theory we should exclude these neurons for model evaluation, we did not do it as doing it or not has negligible effects with hundreds of neurons in our data set.

In practice, we performed PCA only on the pure quadratic terms to reduce their dimensionalities to 432 and concatenated the PCAed 432-dimensional pure quadratic terms with the 400-dimensional linear terms to generate the final 882-dimensional input vectors; such method would guarantee that the information from linear terms, which are heavily used in most V1 models, is preserved. We also tried performing PCA on both linear and pure quadratic terms together and two methods made little difference in our experiments.

Adelson, E.H., & Bergen, J.R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America. A, 2(2), 284–299. https://doi.org/10.1364/JOSAA.2.000284. http://josaa.osa.org/abstract.cfm?URI=josaa-2-2-284.CrossRef

Andrews, B.W., & Pollen, D.A. (1979). Relationship between spatial frequency selectivity and receptive field profile of simple cells. The Journal of Physiology, 287(1), 163–176. https://doi.org/10.1113/jphysiol.1979.sp012652.CrossRefPubMedPubMedCentral

Bishop, C.M. (2006). Machine learning and pattern recognition. Information science and statistics. Springer.

Cadena, S.A., Denfield, G.H., Walker, E.Y., Gatys, L.A., Tolias, A.S., Bethge, M., Ecker, A.S. (2017). Deep convolutional models improve predictions of macaque v1 responses to natural images. bioRxiv.

Carandini, M., Demb, J.B., Mante, V., Tolhurst, D.J., Dan, Y., Olshausen, B.A., Gallant, J.L., Rust, N.C. (2005). Do we know what the early visual system does? Journal of Neuroscience, 25(46), 10577–10597. https://doi.org/10.1523/JNEUROSCI.3726-05.2005. http://www.jneurosci.org/cgi/doi/10.1523/JNEUROSCI.3726-05.2005.CrossRefPubMed

Coen-Cagli, R., Kohn, A., Schwartz, O. (2015). Flexible gating of contextual influences in natural vision. Nature Neuroscience, 18, 1648–1655. https://doi.org/10.1038/nn.4128.CrossRefPubMedPubMedCentral

Daugman, J.G. (1985). Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. Journal of the Optical Society of America. A, 2(7), 1160–1169.CrossRef

David, S.V., & Gallant, J.L. (2005). Predicting neuronal responses during natural vision. Network: Computation in Neural Systems, 16(2-3), 239–260. https://doi.org/10.1080/09548980500464030. http://www.tandfonline.com/doi/full/10.1080/09548980500464030.CrossRef

Dayan, P., & Abbott, L.F. (2001). Theoretical neuroscience. Computational and Mathematical Modeling of Neural Systems.

Finn, I.M., & Ferster, D. (2007). Computational diversity in complex cells of cat primary visual cortex. Journal of Neuroscience, 27(36), 9638–9648. https://doi.org/10.1523/JNEUROSCI.2119-07.2007. http://www.jneurosci.org/content/27/36/9638.CrossRefPubMed

Friedman, J., Hastie, T., Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software Articles, 33(1), 1–22. https://doi.org/10.18637/jss.v033.i01.CrossRef

Fukushima, K. (1980). Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36(4), 193–202. https://doi.org/10.1007/BF00344251.CrossRefPubMed

Gollisch, T., & Meister, M. (2010). Eye smarter than scientists believed: neural computations in circuits of the retina. Neuron, 65(2), 150–164. https://doi.org/10.1016/j.neuron.2009.12.009.CrossRefPubMedPubMedCentral

Goodfellow, I.J., Bengio, Y., Courville, A. (2016). Deep learning. MIT Press.

Heeger, D.J. (1992). Half-squaring in responses of cat striate cells. Visual Neuroscience, 9(5), 427–443. https://doi.org/10.1017/S095252380001124X.CrossRefPubMed

Hegdé, J., & Van Essen, D.C. (2007). A comparative study of shape representation in macaque visual areas V2 and V4. Cerebral Cortex, 17(5), 1100–1116.CrossRefPubMed

Hsu, A., Borst, A., Theunissen, F. (2004). Quantifying variability in neural responses and its application for the validation of model predictions. Network: Computation in Neural Systems, 15(2), 91–109.CrossRef

Hubel, D.H., & Wiesel, T.N. (1959). Receptive fields of single neurones in the cat’s striate cortex. The Journal of Physiology, 148(3), 574–591. https://doi.org/10.1113/jphysiol.1959.sp006308.CrossRefPubMedPubMedCentral

Hubel, D.H., & Wiesel, T.N. (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of Physiology, 160(1), 106–154. https://doi.org/10.1113/jphysiol.1962.sp006837.CrossRefPubMedPubMedCentral

Hubel, D.H., & Wiesel, T.N. (1968). Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology, 195(1), 215–243. https://doi.org/10.1113/jphysiol.1968.sp008455.CrossRefPubMedPubMedCentral

Jones, J.P., & Palmer, L.A. (1987a). An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. Journal of Neurophysiology, 58(6), 1233– 1258.CrossRefPubMed

Jones, J.P., & Palmer, L.A. (1987b). The two-dimensional spatial structure of simple receptive fields in cat striate cortex. Journal of Neurophysiology, 58(6), 1187–1211. http://jn.physiology.org/content/58/6/1187.CrossRefPubMed

Kelly, R.C., Smith, M.A., Kass, R.E., Lee, T.S. (2010). Accounting for network effects in neuronal responses using L1 regularized point process models. In Lafferty, J.D., Williams, C.K.I., Shawe-Taylor, J., Zemel, R.S., Culotta, A. (Eds.) Advances in neural information processing systems 23: 24th annual conference on neural information processing systems 2010. Proceedings of a meeting held 6-9 December 2010, Vancouver, British Columbia, Canada (pp. 1099–1107): Curran Associates, Inc. http://papers.nips.cc/paper/4050-accounting-for-network-effects-in-neuronal-responses-using-l1-regularized-point-process-models.

Kindel, W.F., Christensen, E.D., Zylberberg, J. (2017). Using deep learning to reveal the neural code for images in primary visual cortex. ArXiv e-prints, q-bio.NC.

Kingma, D.P., & Ba, J. (2014). Adam: a method for stochastic optimization. CoRR, arXiv:1412.6980.

Klindt, D., Ecker, A.S., Euler, T., Bethge, M. (2017). Neural system identification for large populations separating “what” and “where”. In Guyon, I.I., von Luxburg, U., Bengio, S., Wallach, H.M., Fergus, R., Vishwanathan, S.V.N., Garnett, R. (Eds.) Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, 4-9 December 2017 (pp. 3509–3519). Long Beach.

Köster, U., & Olshausen, B. (2013). Testing our conceptual understanding of V1 function. q-bio.NC. arXiv:1311.0778.

Kotikalapudi, R. (2017). keras-vis: Keras visualization toolkit. https://github.com/raghakot/keras-vis.

Kriegeskorte, N. (2015). Deep neural networks: a new framework for modeling biological vision and brain information processing. Annual Review of Vision Science, 1(1), 417–446.CrossRefPubMed

Kriegeskorte, N., Mur, M., Bandettini, P. (2008). Representational similarity analysis - connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 4. https://doi.org/10.3389/neuro.06.004.2008. https://www.frontiersin.org/article/10.3389/neuro.06.004.2008.CrossRefPubMedPubMedCentral

Krizhevsky, A., Sutskever, I., Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. In Bartlett, P.L., Pereira, F.C.N., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (Eds.) Advances in neural information processing systems 25: 26th annual conference on neural information processing systems 2012. Proceedings of a meeting held December 3-6, 2012 (pp. 1106–1114). Lake Tahoe.

Li, M., Liu, F., Jiang, H., Lee, T.S., Tang, S. (2017). Long-term two-photon imaging in awake macaque monkey. Neuron, 93(5), 1049–1057.e3.CrossRefPubMed

McCullagh, P., & Nelder, J. (1989). Generalized linear models, 2nd edn. Chapman & Hall/CRC Monographs on Statistics & Applied Probability. Taylor & Francis.

McFarland, J.M., Cui, Y., Butts, D.A. (2013). Inferring nonlinear neuronal computation based on physiologically plausible inputs. PLoS Computational Biology, 9(7), e1003143.CrossRefPubMedPubMedCentral

McIntosh, L.T., Maheswaranathan, N., Nayebi, A., Ganguli, S., Baccus, S.A. (2017). Deep learning models of the retinal response to natural scenes. ArXiv e-prints, q-bio.NC.

Olah, C., Mordvintsev, A., Schubert, L. (2017). Feature visualization. Distill. https://distill.pub/2017/feature-visualization.

Paninski, L. (2004). Maximum likelihood estimation of cascade point-process neural encoding models. Network: Computation in Neural Systems, 15(4), 243–262.CrossRef

Park, I.M., & Pillow, J.W. (2011). Bayesian spike-triggered covariance analysis. In Taylor, J.S., Zemel, R.S., Bartlett, P.L., Pereira, F.C.N., Weinberger, K.Q. (Eds.) Advances in neural information processing systems 24: 25th annual conference on neural information processing systems 2011. Proceedings of a meeting held 12-14 December 2011 (pp. 1692–1700). Granada.

Park, I.M., Archer, E., Priebe, N., Pillow, J.W. (2013). Spectral methods for neural characterization using generalized quadratic models. In Burges, C.J.C., Bottou, L., Ghahramani, Z., Weinberger, K.Q. (Eds.) Advances in neural information processing systems 26: 27th annual conference on neural information processing systems 2013. Proceedings of a meeting held December 5-8, 2013 (pp. 2454–2462). Lake Tahoe.

Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A. (2017). Automatic differentiation. In pytorch.

Pillow, J.W., Shlens, J., Paninski, L., Sher, A., Litke, A.M., Chichilnisky, E.J., Simoncelli, E.P. (2008). Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature, 454, 995–999. https://doi.org/10.1038/nature07140.CrossRefPubMedPubMedCentral

Prenger, R., Wu, M.C.K., David, S.V., Gallant, J.L. (2004). Nonlinear V1 responses to natural scenes revealed by neural network analysis. Neural Networks, 17(5–6), 663–679.CrossRefPubMed

Riesenhuber, M., & Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nature Neuroscience, 2, 1019–1025. https://doi.org/10.1038/14819.CrossRefPubMed

Rowekamp, R.J., & Sharpee, T.O. (2017). Cross-orientation suppression in visual area V2. Nature Communications, 8, 15739.CrossRefPubMedPubMedCentral

Rust, N.C., Schwartz, O., Movshon, J.A., Simoncelli, E.P. (2005). Spatiotemporal elements of macaque V1 receptive fields. Neuron, 46(6), 945–956. https://doi.org/10.1016/j.neuron.2005.05.021. http://linkinghub.elsevier.com/retrieve/pii/S089662730500468X.CrossRefPubMed

Schoppe, O., Harper, N.S., Willmore, B.D.B., King, A.J., Schnupp, J.W.H. (2016). Measuring the performance of neural models. Frontiers in Computational Neuroscience, 10, 1929.CrossRef

Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. ArXiv e-prints, cs.CV.

Tang, S., Lee, T.S., Li, M., Zhang, Y., Xu, Y., Liu, F., Teo, B., Jiang, H. (2018). Complex pattern selectivity in macaque primary visual cortex revealed by large-scale two-photon imaging. Current Biology, 28 (1), 38–48.e3.CrossRefPubMed

Theunissen, F.E., David, S.V., Singh, N.C., Hsu, A., Vinje, W.E., Gallant, J.L. (2001). Estimating spatio-temporal receptive fields of auditory and visual neurons from their responses to natural stimuli. Network: Computation in Neural Systems, 12(3), 289–316.CrossRef

Touryan, J., Felsen, G., Dan, Y. (2005). Spatial structure of complex cell receptive fields measured with natural images. Neuron, 45 (5), 781–791. https://doi.org/10.1016/j.neuron.2005.01.029. http://linkinghub.elsevier.com/retrieve/pii/S0896627305000619.CrossRefPubMed

Victor, J.D., Mechler, F., Repucci, M.A., Purpura, K.P., Sharpee, T. (2006). Responses of v1 neurons to two-dimensional hermite functions. Journal of Neurophysiology, 95(1), 379–400. https://doi.org/10.1152/jn.00498.2005. http://jn.physiology.org/content/95/1/379.CrossRefPubMed

Vintch, B., Movshon, J.A., Simoncelli, E.P. (2015). A convolutional subunit model for neuronal responses in macaque v1. Journal of Neuroscience, 35(44), 14829–14841.CrossRefPubMed

Wu, M.C.K., David, S.V., Gallant, J.L. (2006). Complete functional characterization of sensory neurons by system identification. Annual Review of Neuroscience, 29(1), 477–505. https://doi.org/10.1523/JNEUROSCI.2815-13.2015.CrossRefPubMed

Yamins, D., Hong, H., Cadieu, C.F., DiCarlo, J.J. (2013). Hierarchical modular optimization of convolutional networks achieves representations similar to macaque IT and human ventral stream. In Burges, C.J.C., Bottou, L., Ghahramani, Z., Weinberger, K.Q. (Eds.) Advances in neural information processing systems 26: 27th annual conference on neural information processing systems 2013. Proceedings of a meeting held December 5-8, 2013 (pp. 3093–3101). Lake Tahoe.

Yamins, D.L.K., & DiCarlo, J.J. (2016). Using goal-driven deep learning models to understand sensory cortex. Nature Neuroscience, 19(3), 356–365.CrossRefPubMed

Zhang, Y., Massot, C., Zhi, T., Papandreou, G., Yuille, A., Lee, T.S. (2016). Understanding neural representations in early visual areas using convolutional neural networks. In Neuroscience (SfN).

Title: Convolutional neural network models of V1 responses to complex patterns
Authors: Yimeng Zhang
Tai Sing Lee
Ming Li
Fang Liu
Shiming Tang
Publication date: 05-06-2018
Publisher: Springer US
Published in: Journal of Computational Neuroscience / Issue 1/2019
Print ISSN: 0929-5313
Electronic ISSN: 1573-6873
DOI: https://doi.org/10.1007/s10827-018-0687-7

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 1/2019

An exploratory data analysis method for identifying brain regions and frequencies of interest from large-scale neural recordings

Linear feature projection-based real-time decoding of limb state from dorsal root ganglion recordings

Emerging techniques in statistical analysis of neural data

An intracerebral exploration of functional connectivity during word production

Ensembles of change-point detectors: implications for real-time BMI applications

Motor imagery and mental fatigue: inter-relationship and EEG based estimation

Premium Partner