Skip to main content
Erschienen in: International Journal of Speech Technology 1/2017

03.11.2016

Subjective speech quality measurement repeatability: comparison of laboratory test results

verfasst von: Jan Holub, Hakob Avetisyan, Scott Isabelle

Erschienen in: International Journal of Speech Technology | Ausgabe 1/2017

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This article reports on a multi-lab subjective listening experiment aiming at inter-lab and intra-lab test results repeatability verification. An identical set of speech samples corresponding to contemporary networks has been tested by three independent labs deploying ITU-T P.835 methodology. The tests results have been compared regarding Pearson correlation, RMSE, RMSE* and numbers of opposite pair-wise comparisons. The results show the level of inter-lab and intra-lab repeatability in the case of identical test speech samples utilization and thus confirm the subjective tests are highly repeatable in case they follow recommendation requirements strictly. The tests also show differences in results in case subject expectations are set differently using a wider set of test speech samples (as presented in one of the labs).

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Appendix of IEEE Subcommittee on Subjective Measurements IEEE Recommended Practices for Speech Quality Measurements. (1969). IEEE Transactions on Audio and Electroacoustics. Vol 17, pp. 227–246. Appendix of IEEE Subcommittee on Subjective Measurements IEEE Recommended Practices for Speech Quality Measurements. (1969). IEEE Transactions on Audio and Electroacoustics. Vol 17, pp. 227–246.
Zurück zum Zitat European Telecommunications Standards Institute. (2008). Speech processing, transmission and quality aspects (STQ); speech quality performance in the presence of background noise part 3: Background noise transmission—Objective test methods. European Telecommunications Standards Institute, ETSI EG 202 396-3. European Telecommunications Standards Institute. (2008). Speech processing, transmission and quality aspects (STQ); speech quality performance in the presence of background noise part 3: Background noise transmission—Objective test methods. European Telecommunications Standards Institute, ETSI EG 202 396-3.
Zurück zum Zitat European Telecommunications Standards Institute. (2014). Speech and multimedia transmission quality (STQ); speech quality performance in the presence of background noise: Background noise transmission for mobile terminals—Objective test methods. European Telecommunications Standards Institute, ETSI TS 103 106. European Telecommunications Standards Institute. (2014). Speech and multimedia transmission quality (STQ); speech quality performance in the presence of background noise: Background noise transmission for mobile terminals—Objective test methods. European Telecommunications Standards Institute, ETSI TS 103 106.
Zurück zum Zitat Goodman, D. J., & Nash, R. D. (1982). Subjective quality of the same speech transmission conditions in seven different countries. IEEE Transactions on Communications, 30(4), 642–654.CrossRef Goodman, D. J., & Nash, R. D. (1982). Subjective quality of the same speech transmission conditions in seven different countries. IEEE Transactions on Communications, 30(4), 642–654.CrossRef
Zurück zum Zitat 3GPP TR 26.952. (2015). Codec for Enhanced Voice Services (EVS); Performance Characterization. 3GPP TR 26.952. (2015). Codec for Enhanced Voice Services (EVS); Performance Characterization.
Zurück zum Zitat 3GPP TS 26 071. Mandatory speech CODEC speech processing functions; AMR speech Codec; General description 3GPP TS 26 071. Mandatory speech CODEC speech processing functions; AMR speech Codec; General description
Zurück zum Zitat ITU-T Rec. P.800. (1996). Methods for subjective determination of transmission quality, Series P: Telephone transmission quality, ITU, Geneva, am. 1998. ITU-T Rec. P.800. (1996). Methods for subjective determination of transmission quality, Series P: Telephone transmission quality, ITU, Geneva, am. 1998.
Zurück zum Zitat ITU-T Rec. P.835. (2003). Methods for objective and subjective assessment of quality, Series P: Telephone transmission quality, Telephone Installations, Local Line Networks, ITU, Geneva. ITU-T Rec. P.835. (2003). Methods for objective and subjective assessment of quality, Series P: Telephone transmission quality, Telephone Installations, Local Line Networks, ITU, Geneva.
Zurück zum Zitat ITU-T Rec. P.863. (2014). Methods for objective and subjective assessment of speech quality, Series P: Terminals and subjective and objective assessment methods, ITU, Geneva. ITU-T Rec. P.863. (2014). Methods for objective and subjective assessment of speech quality, Series P: Terminals and subjective and objective assessment methods, ITU, Geneva.
Zurück zum Zitat ITU-T TD12rev1. (2009). Statistical evaluation. Procedure for P.OLQA v.1.0, SwissQual AG (Author: Jens Berger), ITUTSG12 Meeting, Geneva, Switzerland, March 10–19, 2009. ITU-T TD12rev1. (2009). Statistical evaluation. Procedure for P.OLQA v.1.0, SwissQual AG (Author: Jens Berger), ITUTSG12 Meeting, Geneva, Switzerland, March 10–19, 2009.
Zurück zum Zitat Pinson, M. H., Janowski, L., Pepion, R., Huynh-Thu, Q., Schmidmer, C., Corriveau, P., et al. (2012). The influence of subjects and environment on audiovisual subjective tests: An international study. IEEE Journal of Selected Topics in Signal Processing, 6(6), 640–651.CrossRef Pinson, M. H., Janowski, L., Pepion, R., Huynh-Thu, Q., Schmidmer, C., Corriveau, P., et al. (2012). The influence of subjects and environment on audiovisual subjective tests: An international study. IEEE Journal of Selected Topics in Signal Processing, 6(6), 640–651.CrossRef
Metadaten
Titel
Subjective speech quality measurement repeatability: comparison of laboratory test results
verfasst von
Jan Holub
Hakob Avetisyan
Scott Isabelle
Publikationsdatum
03.11.2016
Verlag
Springer US
Erschienen in
International Journal of Speech Technology / Ausgabe 1/2017
Print ISSN: 1381-2416
Elektronische ISSN: 1572-8110
DOI
https://doi.org/10.1007/s10772-016-9389-6

Weitere Artikel der Ausgabe 1/2017

International Journal of Speech Technology 1/2017 Zur Ausgabe