Skip to main content
Erschienen in:
Buchtitelbild

2017 | OriginalPaper | Buchkapitel

Building Knowledge for the Purpose of Lip Speech Identification

verfasst von : Andrzej Czyżewski, Bożena Kostek, Marcin Szykulski, Tomasz E. Ciszewski

Erschienen in: Multimedia and Network Information Systems

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Consecutive stages of building knowledge for automatic lip speech identification are shown in this study. The main objective is to prepare audio-visual material for phonetic analysis and transcription. First, approximately 260 sentences of natural English were prepared taking into account the frequencies of occurrence of all English phonemes. Five native speakers from different countries read the selected sentences in front of three cameras. Video signals, synchronized with audio, were registered and then analyzed. Encountered problems related to video registration and results achieved are discussed.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
2.
Zurück zum Zitat Zhou, Z., Zhao, G., Hong, X., Pietikäinen, M.: A review of recent advances in visual speech decoding. Image Visual Comput. 32(9), 590–605 (2014)CrossRef Zhou, Z., Zhao, G., Hong, X., Pietikäinen, M.: A review of recent advances in visual speech decoding. Image Visual Comput. 32(9), 590–605 (2014)CrossRef
3.
Zurück zum Zitat Zeiler, S., Nicheli, R., Ma, N., Brown, G.J., Kolossa, D.: Robust audiovisual speech recognition using noise-adaptive linear discriminant analysis. ICASSP 2016, 2797–2801 (2016) Zeiler, S., Nicheli, R., Ma, N., Brown, G.J., Kolossa, D.: Robust audiovisual speech recognition using noise-adaptive linear discriminant analysis. ICASSP 2016, 2797–2801 (2016)
4.
Zurück zum Zitat Li, J., Deng, L., Gong, Y., Haeb-Umbach, R.: An overview of noise-robust automatic speech recognition. IEEE Trans. Audio Speech Lang. Process. 22(4), 745–777 (2014) Li, J., Deng, L., Gong, Y., Haeb-Umbach, R.: An overview of noise-robust automatic speech recognition. IEEE Trans. Audio Speech Lang. Process. 22(4), 745–777 (2014)
5.
Zurück zum Zitat Biswas, A., Sahu, P.K., Chandra, M.: Multiple camera in car audio–visual speech recognition using phonetic and visemic information. Comput. Electr. Eng. 47, 35–50 (2015)CrossRef Biswas, A., Sahu, P.K., Chandra, M.: Multiple camera in car audio–visual speech recognition using phonetic and visemic information. Comput. Electr. Eng. 47, 35–50 (2015)CrossRef
6.
Zurück zum Zitat Czyżewski, A., Kostek, B., Ciszewski, T., Majewicz, D.: Language material for English audiovisual speech recognition system development. In: J. Acoust. Soc. Am. 134/5, 4069 (2013) (abstr.) and Proc. Meet. Acoust., 1(20), 1–7, San Francisco, USA, 2.12.2013–6.12.2013 Czyżewski, A., Kostek, B., Ciszewski, T., Majewicz, D.: Language material for English audiovisual speech recognition system development. In: J. Acoust. Soc. Am. 134/5, 4069 (2013) (abstr.) and Proc. Meet. Acoust., 1(20), 1–7, San Francisco, USA, 2.12.2013–6.12.2013
7.
Zurück zum Zitat Leech, G., Rayson, P., Wilson, A.: Word Frequencies in Written and Spoken English: Based on the British National Corpus. Longman, London (2001) Leech, G., Rayson, P., Wilson, A.: Word Frequencies in Written and Spoken English: Based on the British National Corpus. Longman, London (2001)
8.
Zurück zum Zitat Kunka, B., Kupryjanow, A., Dalka, P., Bratoszewski, P., Szczodrak, M., Spaleniak, P., Szykulski, M., Czyżewski, A.: Multimodal English corpus for automatic speech recognition. In: IEEE Signal Processing—Algorithms, Archit. Architectures, Arrangements, and Applications Conference. Proceedings, Poznań, Poland, 26–28, (2013) Kunka, B., Kupryjanow, A., Dalka, P., Bratoszewski, P., Szczodrak, M., Spaleniak, P., Szykulski, M., Czyżewski, A.: Multimodal English corpus for automatic speech recognition. In: IEEE Signal Processing—Algorithms, Archit. Architectures, Arrangements, and Applications Conference. Proceedings, Poznań, Poland, 26–28, (2013)
9.
Zurück zum Zitat Dalka, P., Bratoszewski, P., Czyżewski, A.: Visual lip contour detection for the purpose of speech recognition. In: International Conference on Signals and Electronic Systems, Poznań (2014) Dalka, P., Bratoszewski, P., Czyżewski, A.: Visual lip contour detection for the purpose of speech recognition. In: International Conference on Signals and Electronic Systems, Poznań (2014)
10.
Zurück zum Zitat Bratoszewski, P., Czyżewski, A.: Face profile view retrieval using time of flight camera image analysis. In: Pattern Recognition and Machine Intelligence 2015, vol. 9124, pp. 159–168, Warsaw (2015) Bratoszewski, P., Czyżewski, A.: Face profile view retrieval using time of flight camera image analysis. In: Pattern Recognition and Machine Intelligence 2015, vol. 9124, pp. 159–168, Warsaw (2015)
11.
Zurück zum Zitat Lavagetto, F.: Converting speech into lip movements: a multimedia telephone for hard hearing people. IEEE Trans. Rehab. Eng. 3(1), 90–102 (1995)CrossRef Lavagetto, F.: Converting speech into lip movements: a multimedia telephone for hard hearing people. IEEE Trans. Rehab. Eng. 3(1), 90–102 (1995)CrossRef
12.
Zurück zum Zitat Dupont, S., Luettin, J.: Audio-visual speech modeling for continuous speech recognition. IEEE Trans. Multimedia 2(3), 141–151 (2000)CrossRef Dupont, S., Luettin, J.: Audio-visual speech modeling for continuous speech recognition. IEEE Trans. Multimedia 2(3), 141–151 (2000)CrossRef
13.
Zurück zum Zitat Harnsberger, J.D.: On the relationship between identification and discrimination of non-native nasal consonants. J. Acoust. Soc. Am. 110(1), 489–503 (2001)CrossRef Harnsberger, J.D.: On the relationship between identification and discrimination of non-native nasal consonants. J. Acoust. Soc. Am. 110(1), 489–503 (2001)CrossRef
15.
Zurück zum Zitat Lucey, S., Chen, T., Sridharan, S., Chandran, V.: Integration strategies for audio-visual speech processing: applied to text dependent speaker recognition. IEEE Trans. Multimedia 7(3), 495–506 (2005). doi:10.1109/TMM.2005.846777 CrossRef Lucey, S., Chen, T., Sridharan, S., Chandran, V.: Integration strategies for audio-visual speech processing: applied to text dependent speaker recognition. IEEE Trans. Multimedia 7(3), 495–506 (2005). doi:10.​1109/​TMM.​2005.​846777 CrossRef
Metadaten
Titel
Building Knowledge for the Purpose of Lip Speech Identification
verfasst von
Andrzej Czyżewski
Bożena Kostek
Marcin Szykulski
Tomasz E. Ciszewski
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-43982-2_1