Skip to main content
Top

2024 | OriginalPaper | Chapter

Brain-Inspired Binaural Sound Source Localization Method Based on Liquid State Machine

Authors : Yuan Li, Jingyue Zhao, Xun Xiao, Renzhi Chen, Lei Wang

Published in: Neural Information Processing

Publisher: Springer Nature Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Binaural Sound Source Localization (BSSL) is a remarkable topic in robot design and human hearing aid. A great number of algorithms flourished due to a leap in machine learning. However, prior approaches lack the ability to make a trade-off between parameter size and accuracy, which is a primary obstacle to their further implementation on resource-constrained devices. Spiking Neural Network (SNN)-based models have also emerged due to their inherent computing superiority over sparse event processing. Liquid State Machine (LSM) is a classic Spiking Recurrent Neural Network (SRNN) which has the natural potential of processing spatiotemporal information. LSM has been proved advantageous on numerous tasks once proposed. Yet, to our best knowledge, it is the first proposed BSSL model based on LSM, and we name it BSSL-LSM. BSSL-LSM is lightweight with only 1.04M parameters, which is a considerable reduction compared to CNN (10.1M) and D-BPNN (2.23M) while maintaining comparable or even superior accuracy. Compared to SNN-IID, there is a 10% accuracy improvement for \(10^\circ \) interval localization. To achieve better performance, we introduce Bayesian Optimization (BO) for hyperparameters searching and a novel soft label technique for better differentiating adjacent angles, which can be easily mirrored on related works. Project page: https://​github.​com/​BSSL-LSM.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Algazi, V.R., Duda, R.O., Thompson, D., Avendaño, C.: The CIPIC HRTF database. In: Proceedings of the 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics (Cat. No. 01TH8575), pp. 99–102 (2001) Algazi, V.R., Duda, R.O., Thompson, D., Avendaño, C.: The CIPIC HRTF database. In: Proceedings of the 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics (Cat. No. 01TH8575), pp. 99–102 (2001)
2.
go back to reference Bu, H., Du, J., Na, X., Wu, B., Zheng, H.: AISHELL-1: an open-source mandarin speech corpus and a speech recognition baseline. In: Oriental COCOSDA 2017 (2017, submitted) Bu, H., Du, J., Na, X., Wu, B., Zheng, H.: AISHELL-1: an open-source mandarin speech corpus and a speech recognition baseline. In: Oriental COCOSDA 2017 (2017, submitted)
3.
go back to reference Das, D., Bhattacharya, S., Pal, U., Chanda, S.: PLSM: a parallelized liquid state machine for unintentional action detection. ArXiv abs/2105.09909 (2021) Das, D., Bhattacharya, S., Pal, U., Chanda, S.: PLSM: a parallelized liquid state machine for unintentional action detection. ArXiv abs/2105.09909 (2021)
4.
go back to reference Dávila-Chacón, J., Liu, J., Wermter, S.: Enhanced robot speech recognition using biomimetic binaural sound source localization. IEEE Trans. Neural Netw. Learn. Syst. 30(1), 138–150 (2018)CrossRef Dávila-Chacón, J., Liu, J., Wermter, S.: Enhanced robot speech recognition using biomimetic binaural sound source localization. IEEE Trans. Neural Netw. Learn. Syst. 30(1), 138–150 (2018)CrossRef
5.
go back to reference Desai, D., Mehendale, N.: A review on sound source localization systems. Arch. Comput. Methods Eng. 29(7), 4631–4642 (2022)CrossRef Desai, D., Mehendale, N.: A review on sound source localization systems. Arch. Comput. Methods Eng. 29(7), 4631–4642 (2022)CrossRef
6.
go back to reference Faraji, M.M., Shouraki, S.B., Iranmehr, E.: Spiking neural network for sound localization using microphone array. In: 2015 23rd Iranian Conference on Electrical Engineering, pp. 1260–1265 (2015) Faraji, M.M., Shouraki, S.B., Iranmehr, E.: Spiking neural network for sound localization using microphone array. In: 2015 23rd Iranian Conference on Electrical Engineering, pp. 1260–1265 (2015)
7.
go back to reference Gerstner, W., Kempter, R., van Hemmen, J.L., Wagner, H.: A neuronal learning rule for sub-millisecond temporal coding. Nature 383, 76–78 (1996)CrossRef Gerstner, W., Kempter, R., van Hemmen, J.L., Wagner, H.: A neuronal learning rule for sub-millisecond temporal coding. Nature 383, 76–78 (1996)CrossRef
8.
go back to reference Gerstner, W., Ritz, R., van Hemmen, J.L.: Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biol. Cybern. 69, 503–515 (1993)CrossRefMATH Gerstner, W., Ritz, R., van Hemmen, J.L.: Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biol. Cybern. 69, 503–515 (1993)CrossRefMATH
9.
go back to reference Ghani, A., McGinnity, T.M., Maguire, L.P., McDaid, L.J., Belatreche, A.: Neuro-inspired speech recognition based on reservoir computing (2010) Ghani, A., McGinnity, T.M., Maguire, L.P., McDaid, L.J., Belatreche, A.: Neuro-inspired speech recognition based on reservoir computing (2010)
10.
go back to reference Glackin, B.P., Wall, J.A., Mcginnity, T.M., Maguire, L.P., McDaid, L.J.: A spiking neural network model of the medial superior olive using spike timing dependent plasticity for sound localization. Frontiers Comput. Neurosci. 4 (2010) Glackin, B.P., Wall, J.A., Mcginnity, T.M., Maguire, L.P., McDaid, L.J.: A spiking neural network model of the medial superior olive using spike timing dependent plasticity for sound localization. Frontiers Comput. Neurosci. 4 (2010)
11.
go back to reference Goodman, D.F.M., Pressnitzer, D., Brette, R.: Sound localization with spiking neural networks. BMC Neurosci. 10, 1 (2009)CrossRef Goodman, D.F.M., Pressnitzer, D., Brette, R.: Sound localization with spiking neural networks. BMC Neurosci. 10, 1 (2009)CrossRef
12.
go back to reference Guo, S., et al.: A systolic SNN inference accelerator and its co-optimized software framework. In: Proceedings of the 2019 on Great Lakes Symposium on VLSI (2019) Guo, S., et al.: A systolic SNN inference accelerator and its co-optimized software framework. In: Proceedings of the 2019 on Great Lakes Symposium on VLSI (2019)
13.
go back to reference Jeffress, L.A.: A place theory of sound localization. J. Comp. Physiol. Psychol. 41(1), 35–9 (1948)CrossRef Jeffress, L.A.: A place theory of sound localization. J. Comp. Physiol. Psychol. 41(1), 35–9 (1948)CrossRef
14.
go back to reference Kuang, S., van der Heijden, K., Mehrkanoon, S.: BAST: binaural audio spectrogram transformer for binaural sound localization. ArXiv abs/2207.03927 (2022) Kuang, S., van der Heijden, K., Mehrkanoon, S.: BAST: binaural audio spectrogram transformer for binaural sound localization. ArXiv abs/2207.03927 (2022)
16.
go back to reference Li, Y., Zhang, Y., Zhou, G., Gong, Y.: Bayesian optimization with particle swarm. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2021) Li, Y., Zhang, Y., Zhou, G., Gong, Y.: Bayesian optimization with particle swarm. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2021)
17.
go back to reference Liaquat, M.U., Munawar, H.S., Rahman, A., Qadir, Z., Kouzani, A.Z., Mahmud, M.A.P.: Sound localization for ad-hoc microphone arrays. Energies (2021) Liaquat, M.U., Munawar, H.S., Rahman, A., Qadir, Z., Kouzani, A.Z., Mahmud, M.A.P.: Sound localization for ad-hoc microphone arrays. Energies (2021)
18.
go back to reference Luke, R., McAlpine, D.: A spiking neural network approach to auditory source lateralisation. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1488–1492. IEEE (2019) Luke, R., McAlpine, D.: A spiking neural network approach to auditory source lateralisation. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1488–1492. IEEE (2019)
19.
go back to reference Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002)CrossRefMATH Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002)CrossRefMATH
20.
go back to reference Pang, C., Liu, H., Li, X.: Multitask learning of time-frequency CNN for sound source localization. IEEE Access 7, 40725–40737 (2019)CrossRef Pang, C., Liu, H., Li, X.: Multitask learning of time-frequency CNN for sound source localization. IEEE Access 7, 40725–40737 (2019)CrossRef
21.
go back to reference Reynolds, J.J.M., Plank, J.S., Schuman, C.D.: Intelligent reservoir generation for liquid state machines using evolutionary optimization. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2019) Reynolds, J.J.M., Plank, J.S., Schuman, C.D.: Intelligent reservoir generation for liquid state machines using evolutionary optimization. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2019)
22.
go back to reference Rudnicki, M., Schoppe, O., Isik, M., Völk, F., Hemmert, W.: Modeling auditory coding: from sound to spikes. Cell Tissue Res. 361, 159–175 (2015)CrossRef Rudnicki, M., Schoppe, O., Isik, M., Völk, F., Hemmert, W.: Modeling auditory coding: from sound to spikes. Cell Tissue Res. 361, 159–175 (2015)CrossRef
23.
go back to reference Song, H., Liu, X., Yu, S.: Binaural localization algorithm based on deep learning. Technical Acoust. 41 (2022) Song, H., Liu, X., Yu, S.: Binaural localization algorithm based on deep learning. Technical Acoust. 41 (2022)
24.
go back to reference Tang, C., Ji, J., Lin, Q., Zhou, Y.: Evolutionary neural architecture design of liquid state machine for image classification. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 91–95 (2022) Tang, C., Ji, J., Lin, Q., Zhou, Y.: Evolutionary neural architecture design of liquid state machine for image classification. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 91–95 (2022)
25.
go back to reference Tian, S., Qu, L., Wang, L., Hu, K., Li, N., Xu, W.: A neural architecture search based framework for liquid state machine design. Neurocomputing 443, 174–182 (2021)CrossRef Tian, S., Qu, L., Wang, L., Hu, K., Li, N., Xu, W.: A neural architecture search based framework for liquid state machine design. Neurocomputing 443, 174–182 (2021)CrossRef
26.
go back to reference Vecchiotti, P., Ma, N., Squartini, S., Brown, G.J.: End-to-end binaural sound localisation from the raw waveform. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 451–455. IEEE (2019) Vecchiotti, P., Ma, N., Squartini, S., Brown, G.J.: End-to-end binaural sound localisation from the raw waveform. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 451–455. IEEE (2019)
27.
go back to reference Wall, J.A., McDaid, L.J., Maguire, L.P., McGinnity, T.M.: Spiking neural network model of sound localization using the interaural intensity difference. IEEE Trans. Neural Netw. Learn. Syst. 23(4), 574–586 (2012)CrossRef Wall, J.A., McDaid, L.J., Maguire, L.P., McGinnity, T.M.: Spiking neural network model of sound localization using the interaural intensity difference. IEEE Trans. Neural Netw. Learn. Syst. 23(4), 574–586 (2012)CrossRef
28.
go back to reference Wang, S., et al.: A power efficient hardware implementation of the if neuron model. In: Conference on Advanced Computer Architecture (2018) Wang, S., et al.: A power efficient hardware implementation of the if neuron model. In: Conference on Advanced Computer Architecture (2018)
29.
go back to reference Wu, J., Chua, Y., Zhang, M., Li, H., Tan, K.C.: A spiking neural network framework for robust sound classification. Frontiers Neurosci. 12 (2018) Wu, J., Chua, Y., Zhang, M., Li, H., Tan, K.C.: A spiking neural network framework for robust sound classification. Frontiers Neurosci. 12 (2018)
30.
go back to reference Xiao, X., et al.: Dynamic vision sensor based gesture recognition using liquid state machine. In: International Conference on Artificial Neural Networks (2022) Xiao, X., et al.: Dynamic vision sensor based gesture recognition using liquid state machine. In: International Conference on Artificial Neural Networks (2022)
31.
go back to reference Xu, Y., Afshar, S., Singh, R.K., Wang, R., van Schaik, A., Hamilton, T.J.: A binaural sound localization system using deep convolutional neural networks. In: 2019 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5. IEEE (2019) Xu, Y., Afshar, S., Singh, R.K., Wang, R., van Schaik, A., Hamilton, T.J.: A binaural sound localization system using deep convolutional neural networks. In: 2019 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5. IEEE (2019)
32.
go back to reference Yang, Q., Zheng, Y.: DeepEar: sound localization with binaural microphones. IEEE Trans. Mob. Comput. (2022) Yang, Q., Zheng, Y.: DeepEar: sound localization with binaural microphones. IEEE Trans. Mob. Comput. (2022)
33.
go back to reference Youssef, K., Argentieri, S., Zarader, J.L.: A binaural sound source localization method using auditive cues and vision. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 217–220 (2012) Youssef, K., Argentieri, S., Zarader, J.L.: A binaural sound source localization method using auditive cues and vision. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 217–220 (2012)
34.
go back to reference Yu, X., Wang, L., Chen, C., Tie, J., Guo, S.: Multimodal learning of audio-visual speech recognition with liquid state machine. In: International Conference on Neural Information Processing (2022) Yu, X., Wang, L., Chen, C., Tie, J., Guo, S.: Multimodal learning of audio-visual speech recognition with liquid state machine. In: International Conference on Neural Information Processing (2022)
35.
go back to reference Zheng, H., Wu, Y., Deng, L., Hu, Y., Li, G.: Going deeper with directly-trained larger spiking neural networks. In: AAAI Conference on Artificial Intelligence (2020) Zheng, H., Wu, Y., Deng, L., Hu, Y., Li, G.: Going deeper with directly-trained larger spiking neural networks. In: AAAI Conference on Artificial Intelligence (2020)
36.
go back to reference Zhu, J., et al.: An event based gesture recognition system using a liquid state machine accelerator. In: Proceedings of the Great Lakes Symposium on VLSI 2022 (2022) Zhu, J., et al.: An event based gesture recognition system using a liquid state machine accelerator. In: Proceedings of the Great Lakes Symposium on VLSI 2022 (2022)
37.
go back to reference Zilany, M.S.A., Bruce, I.C., Carney, L.H.: Updated parameters and expanded simulation options for a model of the auditory periphery. J. Acoust. Soc. Am. 135(1), 283–6 (2014)CrossRef Zilany, M.S.A., Bruce, I.C., Carney, L.H.: Updated parameters and expanded simulation options for a model of the auditory periphery. J. Acoust. Soc. Am. 135(1), 283–6 (2014)CrossRef
Metadata
Title
Brain-Inspired Binaural Sound Source Localization Method Based on Liquid State Machine
Authors
Yuan Li
Jingyue Zhao
Xun Xiao
Renzhi Chen
Lei Wang
Copyright Year
2024
Publisher
Springer Nature Singapore
DOI
https://doi.org/10.1007/978-981-99-8067-3_15

Premium Partner