Skip to main content
Top

2017 | OriginalPaper | Chapter

15. Quality Evaluation

Author : Tom Bäckström

Published in: Speech Coding

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

For evaluating the performance of speech codecs or their algorithms, we need metrics for quality. Since humans are the ultimate users of most speech codecs, subjective testing is the gold standard in quality measurement. In this chapter we present a range of typical subjective evaluation methods and discuss their strengths and weaknesses. Subjective evaluation requires in general plenty of work and time, whereby there exists a selection of automated objective evaluation methods, which simulate subjective measurements. Objective quality estimation methods therefore provide fast and reproducible measures of quality, even if their reliability can never reach that of extensive subjective measurements.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
1
Applicable only to the listed codecs.
 
Literature
1.
go back to reference 3GPP. TS 26.190, Adaptive Multi-Rate (AMR-WB) Speech Codec (2007) 3GPP. TS 26.190, Adaptive Multi-Rate (AMR-WB) Speech Codec (2007)
2.
go back to reference 3GPP. TS 26.445, EVS Codec Detailed Algorithmic Description; 3GPP Technical Specification (Release 12) (2014) 3GPP. TS 26.445, EVS Codec Detailed Algorithmic Description; 3GPP Technical Specification (Release 12) (2014)
3.
go back to reference CCIR Document 11/17. Subjective Assessment of the Quality of Television Pictures (EBU) (1978–1982) CCIR Document 11/17. Subjective Assessment of the Quality of Television Pictures (EBU) (1978–1982)
4.
go back to reference ISO/IEC 23003–3:2012. MPEG-D (MPEG Audio Technologies), Part 3: Unified Speech and Audio Coding (2012) ISO/IEC 23003–3:2012. MPEG-D (MPEG Audio Technologies), Part 3: Unified Speech and Audio Coding (2012)
5.
go back to reference ITU-R Recommendation BS.1284. General Methods for the Subjective Assessment of Sound Quality (2003) ITU-R Recommendation BS.1284. General Methods for the Subjective Assessment of Sound Quality (2003)
6.
go back to reference ITU-R Recommendation BS.1387. Method for Objective Measurements of Perceived Audio Quality (PEAQ), 2001 ITU-R Recommendation BS.1387. Method for Objective Measurements of Perceived Audio Quality (PEAQ), 2001
7.
go back to reference ITU-R Recommendation BS.562. Subjective Assessment of Sound Quality (1990) ITU-R Recommendation BS.562. Subjective Assessment of Sound Quality (1990)
8.
go back to reference ITU-T Recommendation P.800. Methods for Subjective Determination of Transmission Quality (1996) ITU-T Recommendation P.800. Methods for Subjective Determination of Transmission Quality (1996)
9.
go back to reference ITU-T Recommendation P.835. Subjective Test Methodology for Evaluating Speech Communication Systems that Include Noise Suppression Algorithm (2003) ITU-T Recommendation P.835. Subjective Test Methodology for Evaluating Speech Communication Systems that Include Noise Suppression Algorithm (2003)
10.
go back to reference ITU-T Recommendation P.861. Objective Quality Measurement of Telephoneband (300-3400 Hz) Speech Codecs (1996) ITU-T Recommendation P.861. Objective Quality Measurement of Telephoneband (300-3400 Hz) Speech Codecs (1996)
11.
go back to reference ITU-T Recommendation P.862. Perceptual Evaluation of Speech Quality (PESQ): An Objective Method for End-to-end Speech Quality Assessment of Narrow-band Telephone Networks and Speech Codecs (2001) ITU-T Recommendation P.862. Perceptual Evaluation of Speech Quality (PESQ): An Objective Method for End-to-end Speech Quality Assessment of Narrow-band Telephone Networks and Speech Codecs (2001)
12.
go back to reference ITU-T Recommendation P.863. Perceptual Objective Listening Quality Assessment (POLQA) (2011) ITU-T Recommendation P.863. Perceptual Objective Listening Quality Assessment (POLQA) (2011)
13.
go back to reference Recommendation BS.1534. Method for the Subjective Assessment of Intermediate Quality Levels of Coding Systems. ITU-R (2003) Recommendation BS.1534. Method for the Subjective Assessment of Intermediate Quality Levels of Coding Systems. ITU-R (2003)
Metadata
Title
Quality Evaluation
Author
Tom Bäckström
Copyright Year
2017
DOI
https://doi.org/10.1007/978-3-319-50204-5_15