Skip to main content
Erschienen in: Multimedia Systems 3/2018

18.04.2017 | Regular Paper

MOS estimation model development using ACR listening-opinion tests with Thai users referring to loss effects: a case of G.726 and G.729

verfasst von: Pongpisit Wuttidittachotti, Phisit Khaoduang, Therdpong Daengsi

Erschienen in: Multimedia Systems | Ausgabe 3/2018

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper proposes two models of Mean Opinion Score (MOS) estimation based on Thai users and the Thai language, referring to packet loss effects, for G.726 and G.729 codecs. Based on Thai users and Thai speech referring to packet loss effects in this work, the Absolute Category Rate (ACR) listening tests were conducted with 89 participants and 107 participants for the MOS estimation model development of G.726 and G.729 respectively, while the same tests were conducted with totally 60 participants for the model evaluation of both codecs. Packet loss rates were 0–15% for G.726 with 5 test conditions and G.729 with 6 test conditions; each condition was conducted with at least 16 participants. After gathering the data, the MOS estimation models for both codecs were simply created and then evaluated with the test sets, comparing Perceptual Evaluation of Speech Quality (PESQ), a popular measurement method. For one of the contributions of this study, after the models were evaluated using Mean Absolute Percentage Error (MAPE), it was found that the proposed models for G.726 and G.729 provided better performance than PESQ, particularly by reducing the MAPE by about 30% and 17% respectively, compared to PESQ.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat De Rango, F., Tropea, M., Fazio, P., Marano, S.: Overview on VoIP: subjective and objective measurement methods. Int. J. Comput. Sci. Netw. Secur. 6(1B), 140–153 (2006) De Rango, F., Tropea, M., Fazio, P., Marano, S.: Overview on VoIP: subjective and objective measurement methods. Int. J. Comput. Sci. Netw. Secur. 6(1B), 140–153 (2006)
2.
Zurück zum Zitat ITU-T.: ITU-T Recommendation P.800 Methods for subjective determination of transmission quality (1996) ITU-T.: ITU-T Recommendation P.800 Methods for subjective determination of transmission quality (1996)
3.
Zurück zum Zitat Streijl, R.C., Winkler, S., Hands, D.: Mean opinion score (MOS) revisited: methods and applications limitations and alternatives. Multimed. Syst. 22(2), 213–227 (2016)CrossRef Streijl, R.C., Winkler, S., Hands, D.: Mean opinion score (MOS) revisited: methods and applications limitations and alternatives. Multimed. Syst. 22(2), 213–227 (2016)CrossRef
4.
Zurück zum Zitat Baharudin, M.A.B., Quang, T.M., Kamioka, E.: Improvement of handover performance based on bio-inspired approach with received signal strength and mean opinion score. Arab J. Sci. Eng. 40, 1623–1636 (2015)CrossRef Baharudin, M.A.B., Quang, T.M., Kamioka, E.: Improvement of handover performance based on bio-inspired approach with received signal strength and mean opinion score. Arab J. Sci. Eng. 40, 1623–1636 (2015)CrossRef
7.
Zurück zum Zitat Cai, Z., Kitawaki, N., Yamada, T., Makino, S.: Comparison of MOS evaluation characteristics for Chinese, Japanese and English in IP telephony. In: Proceedings of IUCS 2010, Beijing, pp. 1–4 (2010) Cai, Z., Kitawaki, N., Yamada, T., Makino, S.: Comparison of MOS evaluation characteristics for Chinese, Japanese and English in IP telephony. In: Proceedings of IUCS 2010, Beijing, pp. 1–4 (2010)
8.
Zurück zum Zitat Wuttidittachotti, P., Khaoduang, P., Daengsi, T.: Development of a MOS estimation model for G.729 using listening-opinion tests with Thai speech referring to packet loss effects. In: Proceedings of ISCAIE 2014, Penang, pp. 29–32 (2014) Wuttidittachotti, P., Khaoduang, P., Daengsi, T.: Development of a MOS estimation model for G.729 using listening-opinion tests with Thai speech referring to packet loss effects. In: Proceedings of ISCAIE 2014, Penang, pp. 29–32 (2014)
9.
Zurück zum Zitat Daengsi, T., Wutiwiwatchai, C., Preechayasomboon, A., Sukparungsee, S.: IP telephony: comparison of subjective assessment methods for voice quality evaluation. Walailak J. Sci. Technol. 11(2), 87–92 (2014) Daengsi, T., Wutiwiwatchai, C., Preechayasomboon, A., Sukparungsee, S.: IP telephony: comparison of subjective assessment methods for voice quality evaluation. Walailak J. Sci. Technol. 11(2), 87–92 (2014)
11.
Zurück zum Zitat Sodanil, M., Nitsuwat, S., Haruechaiyasak, C.: Thai word recognition using hybrid MLP-HMM. Int. J. Comput. Sci. Netw. Secur. 10, 103–110 (2010) Sodanil, M., Nitsuwat, S., Haruechaiyasak, C.: Thai word recognition using hybrid MLP-HMM. Int. J. Comput. Sci. Netw. Secur. 10, 103–110 (2010)
12.
Zurück zum Zitat Daengsi, T., Wutiwiwatchai, C., Preechayasomboon, A., Sukparungse S.: A study of VoIP quality evaluation: user perception of voice quality from G.729, G.711 and G.722. In: Proceedings of IEEE CCNC—SS-QoE, Las Vegas, pp. 342–345 (2012) Daengsi, T., Wutiwiwatchai, C., Preechayasomboon, A., Sukparungse S.: A study of VoIP quality evaluation: user perception of voice quality from G.729, G.711 and G.722. In: Proceedings of IEEE CCNC—SS-QoE, Las Vegas, pp. 342–345 (2012)
13.
Zurück zum Zitat Lindeberg, M., Kristiansen, S., Plagemann, T., Goebel, V.: Challenges and techniques for video streaming over mobile ad hoc networks. Multimed. Syst. 17, 51–82 (2011)CrossRef Lindeberg, M., Kristiansen, S., Plagemann, T., Goebel, V.: Challenges and techniques for video streaming over mobile ad hoc networks. Multimed. Syst. 17, 51–82 (2011)CrossRef
14.
Zurück zum Zitat Goudarzi. M.: Evaluation of Voice Quality in 3G Mobile Networks. Thesis, University of Plymouth (2008) Goudarzi. M.: Evaluation of Voice Quality in 3G Mobile Networks. Thesis, University of Plymouth (2008)
15.
Zurück zum Zitat Al-Akhras, M., Zedan, H., John, R., Almomani, I.: Non-intrusive speech quality prediction in VoIP networks using a neural network approach. Neurocomputing 72, 2595–2608 (2009)CrossRef Al-Akhras, M., Zedan, H., John, R., Almomani, I.: Non-intrusive speech quality prediction in VoIP networks using a neural network approach. Neurocomputing 72, 2595–2608 (2009)CrossRef
16.
Zurück zum Zitat Mahdi, A.E., Picovici, D.: Advances in voice quality measurement in modern telecommunications. Dig. Signal Process. 19, 79–103 (2009)CrossRef Mahdi, A.E., Picovici, D.: Advances in voice quality measurement in modern telecommunications. Dig. Signal Process. 19, 79–103 (2009)CrossRef
17.
Zurück zum Zitat Ding, L., Lin, Z., Radwan, A., El-Hennaway, M., Goubran, R.: Non-intrusive single-ended speech quality assessment in VoIP. Speech Commun. 49, 477–489 (2007)CrossRef Ding, L., Lin, Z., Radwan, A., El-Hennaway, M., Goubran, R.: Non-intrusive single-ended speech quality assessment in VoIP. Speech Commun. 49, 477–489 (2007)CrossRef
18.
Zurück zum Zitat Karapantazis, S., Pavlidou, F.-N.: VoIP: a comprehensive survey on a promising technology. Comput. Netw. 53(12), 2050–2090 (2009)CrossRef Karapantazis, S., Pavlidou, F.-N.: VoIP: a comprehensive survey on a promising technology. Comput. Netw. 53(12), 2050–2090 (2009)CrossRef
19.
Zurück zum Zitat ITU-T.: ITU-T Recommendation P.800.1 Mean opinion score (MOS) terminology (2006) ITU-T.: ITU-T Recommendation P.800.1 Mean opinion score (MOS) terminology (2006)
21.
Zurück zum Zitat ITU-T.: ITU-T Recommendation G.729, coding of speech at 8 kb/s using conjugate-structure algebraic-code-excited linear prediction (CS-ACELP) (2012) ITU-T.: ITU-T Recommendation G.729, coding of speech at 8 kb/s using conjugate-structure algebraic-code-excited linear prediction (CS-ACELP) (2012)
22.
Zurück zum Zitat ITU-T.: ITU-T Recommendation G.726, 40, 32, 24, 16 kbit/s adaptive differential PulseCode modulation (ADPCM) (1990) ITU-T.: ITU-T Recommendation G.726, 40, 32, 24, 16 kbit/s adaptive differential PulseCode modulation (ADPCM) (1990)
23.
Zurück zum Zitat ITU-T.: ITU-T Recommendation P.862, Perceptual evaluation of speech quality (PESQ): an objective method for end-to-end speech quality assessment of narrow-band telephone (2001) ITU-T.: ITU-T Recommendation P.862, Perceptual evaluation of speech quality (PESQ): an objective method for end-to-end speech quality assessment of narrow-band telephone (2001)
24.
Zurück zum Zitat Voznak, M., Rozhon, J.: Influence of atmospheric parameters on speech quality in GSM/UMTS. Int. J. Math. Model. Method. Appl. Sci. 6(4), 575–582 (2012) Voznak, M., Rozhon, J.: Influence of atmospheric parameters on speech quality in GSM/UMTS. Int. J. Math. Model. Method. Appl. Sci. 6(4), 575–582 (2012)
25.
Zurück zum Zitat Wuttidittachotti, P., Daengsi, T.: Quality evaluation of mobile networks using VoIP applications: a Case Study with Skype and LINE based-on Stationary Tests in Bangkok. Int. J. Comput. Netw. Inform. Security. 7(12), 28–41 (2015)CrossRef Wuttidittachotti, P., Daengsi, T.: Quality evaluation of mobile networks using VoIP applications: a Case Study with Skype and LINE based-on Stationary Tests in Bangkok. Int. J. Comput. Netw. Inform. Security. 7(12), 28–41 (2015)CrossRef
27.
Zurück zum Zitat Jiang, W., Schulzrinne, H.: Comparison and optimization of packet loss repair methods on VoIP perceived quality under bursty loss. In: Proceedings of NOSSDAV’02, Miami, pp. 73–81 (2002) Jiang, W., Schulzrinne, H.: Comparison and optimization of packet loss repair methods on VoIP perceived quality under bursty loss. In: Proceedings of NOSSDAV’02, Miami, pp. 73–81 (2002)
28.
Zurück zum Zitat Zhang, H., Xie, L., Byun, J., Flynn, P., Shim, Y.: Packet loss burstiness and enhancement to the E-model. In: Proceedings of SNPD/SAWN 2005. Towson, pp. 214–219 (2005) Zhang, H., Xie, L., Byun, J., Flynn, P., Shim, Y.: Packet loss burstiness and enhancement to the E-model. In: Proceedings of SNPD/SAWN 2005. Towson, pp. 214–219 (2005)
29.
Zurück zum Zitat ITU-T.: ITU-T Recommendation G.107 The E-model: a computational model for use in transmission planning (2011) ITU-T.: ITU-T Recommendation G.107 The E-model: a computational model for use in transmission planning (2011)
30.
Zurück zum Zitat Ding, L., Goubran, R.A.: Speech quality prediction in VoIP using the extended E-model. In: Proceedings of IEEE GLOBECOM 2003, San Francisco, vol. 7, pp. 3974–3978 (2003) Ding, L., Goubran, R.A.: Speech quality prediction in VoIP using the extended E-model. In: Proceedings of IEEE GLOBECOM 2003, San Francisco, vol. 7, pp. 3974–3978 (2003)
31.
Zurück zum Zitat Sun, L., Ifeachor, E.C.: Voice quality prediction models and their application in VoIP networks. IEEE Trans. Multimed. 8(4), 809–820 (2006)CrossRef Sun, L., Ifeachor, E.C.: Voice quality prediction models and their application in VoIP networks. IEEE Trans. Multimed. 8(4), 809–820 (2006)CrossRef
32.
Zurück zum Zitat Ren, J., Zhang, H., Zhu, Y., Gao, C. Assessment of effects of different language in VOIP. In: Proceedings of ICALIP 2008, Shanghai, pp. 1624–1628 (2008) Ren, J., Zhang, H., Zhu, Y., Gao, C. Assessment of effects of different language in VOIP. In: Proceedings of ICALIP 2008, Shanghai, pp. 1624–1628 (2008)
33.
Zurück zum Zitat Ren, J., Zhang, C., Huang, W., Mao, D.: Enhancement to E-model on standard deviation of packet delay. In: Proceedings of ICIS 2010, Chengdu, pp. 256–259 (2010) Ren, J., Zhang, C., Huang, W., Mao, D.: Enhancement to E-model on standard deviation of packet delay. In: Proceedings of ICIS 2010, Chengdu, pp. 256–259 (2010)
34.
Zurück zum Zitat Raja, A., Azad, R.M.A., Flanagan, C., Ryan, C.: Evolutionary speech quality estimation in VoIP. Soft. Comput. 15, 89–94 (2011)CrossRef Raja, A., Azad, R.M.A., Flanagan, C., Ryan, C.: Evolutionary speech quality estimation in VoIP. Soft. Comput. 15, 89–94 (2011)CrossRef
35.
Zurück zum Zitat Jiang, C., Huang, P.: Research of monitoring VoIP voice QoS. In: Proceedings of ICICIS 2011, Hong Kong, pp. 499–502 (2011) Jiang, C., Huang, P.: Research of monitoring VoIP voice QoS. In: Proceedings of ICICIS 2011, Hong Kong, pp. 499–502 (2011)
36.
Zurück zum Zitat Assem, H., Malone, D., Dunne, J., O’Sullivan, P.: Monitoring VoIP call quality using improved simplified E-model. In: Proceedings of ICNC 2013, San Diego, pp. 927–931 (2013) Assem, H., Malone, D., Dunne, J., O’Sullivan, P.: Monitoring VoIP call quality using improved simplified E-model. In: Proceedings of ICNC 2013, San Diego, pp. 927–931 (2013)
37.
Zurück zum Zitat Adel, M. et al.: Improved E-model for monitoring quality of multi-party VoIP communications. In: Proceedings of IEEE Globecom Workshops 2013, Atlanta, pp. 1180–1185 (2013) Adel, M. et al.: Improved E-model for monitoring quality of multi-party VoIP communications. In: Proceedings of IEEE Globecom Workshops 2013, Atlanta, pp. 1180–1185 (2013)
39.
Zurück zum Zitat Jung, Y., Manzano, C.: Burst packet loss and enhanced packet loss-based quality model for mobile voice-over internet protocol applications. IET Commun. 8(1), 41–49 (2014)CrossRef Jung, Y., Manzano, C.: Burst packet loss and enhanced packet loss-based quality model for mobile voice-over internet protocol applications. IET Commun. 8(1), 41–49 (2014)CrossRef
40.
Zurück zum Zitat Rahdari, F., Eftekhari, M., Akbari, A., Zeinalkhani, M.: Developing fuzzy models for estimating the quality of VoIP. Iran. J. Fuzzy Syst. 11(1), 49–73 (2014)MathSciNet Rahdari, F., Eftekhari, M., Akbari, A., Zeinalkhani, M.: Developing fuzzy models for estimating the quality of VoIP. Iran. J. Fuzzy Syst. 11(1), 49–73 (2014)MathSciNet
41.
Zurück zum Zitat Triyason, T., Kanthamanon, P.: E-model modification for multi-languages over IP. Elektronika ir Elektrotechnika. 21(1), 82–87 (2015)CrossRef Triyason, T., Kanthamanon, P.: E-model modification for multi-languages over IP. Elektronika ir Elektrotechnika. 21(1), 82–87 (2015)CrossRef
42.
Zurück zum Zitat Takahashi, A., Kurashima, A., Yoshino, H.: Objective assessment methodology for estimating conversational quality in VoIP. IEEE Audio Speech Lang Process. 14(6), 1983–1993 (2006) Takahashi, A., Kurashima, A., Yoshino, H.: Objective assessment methodology for estimating conversational quality in VoIP. IEEE Audio Speech Lang Process. 14(6), 1983–1993 (2006)
43.
Zurück zum Zitat Tsiaras, C., Rösch, M., Stiller, B.: VoIP-based calibration of the DQX model. In: Proceedings of IFIP Networking 2015. Toulouse, pp. 1–9 (2015) Tsiaras, C., Rösch, M., Stiller, B.: VoIP-based calibration of the DQX model. In: Proceedings of IFIP Networking 2015. Toulouse, pp. 1–9 (2015)
44.
Zurück zum Zitat Daengsi, T., Preechayasomboon, A., Sukparungsee, S., Chootrakoo, P., Wutiwiwatchai, C.: The development of a Thai speech set for telephonometry. In: Proceedings of oriental-COCOSDA 2010, Kathmandu, Nepal, paper 53 (2010) Daengsi, T., Preechayasomboon, A., Sukparungsee, S., Chootrakoo, P., Wutiwiwatchai, C.: The development of a Thai speech set for telephonometry. In: Proceedings of oriental-COCOSDA 2010, Kathmandu, Nepal, paper 53 (2010)
45.
Zurück zum Zitat Carbone, M., Rizzo, L.: Dummynet revisited. ACM SIGCOMM Comput. Commun. Rev. 40(2), 12–20 (2010)CrossRef Carbone, M., Rizzo, L.: Dummynet revisited. ACM SIGCOMM Comput. Commun. Rev. 40(2), 12–20 (2010)CrossRef
46.
Zurück zum Zitat Daengsi, T., Yochanang, K., Wuttiditachotti, P.: A study of perceptual VoIP quality evaluation with Thai users and codec selection using voice quality: bandwidth tradeoff analysis. In: Proceedings of ICTC 2013, Jeju, pp. 691–696 (2013) Daengsi, T., Yochanang, K., Wuttiditachotti, P.: A study of perceptual VoIP quality evaluation with Thai users and codec selection using voice quality: bandwidth tradeoff analysis. In: Proceedings of ICTC 2013, Jeju, pp. 691–696 (2013)
48.
Zurück zum Zitat Aydin, G., Karakurt, I., Hamzacebi, C.: Performance prediction of diamond sawblades using artificial neural network and regression analysis. Arab. J. Sci. Eng. 40, 2003–2012 (2015)CrossRef Aydin, G., Karakurt, I., Hamzacebi, C.: Performance prediction of diamond sawblades using artificial neural network and regression analysis. Arab. J. Sci. Eng. 40, 2003–2012 (2015)CrossRef
Metadaten
Titel
MOS estimation model development using ACR listening-opinion tests with Thai users referring to loss effects: a case of G.726 and G.729
verfasst von
Pongpisit Wuttidittachotti
Phisit Khaoduang
Therdpong Daengsi
Publikationsdatum
18.04.2017
Verlag
Springer Berlin Heidelberg
Erschienen in
Multimedia Systems / Ausgabe 3/2018
Print ISSN: 0942-4962
Elektronische ISSN: 1432-1882
DOI
https://doi.org/10.1007/s00530-017-0549-6

Weitere Artikel der Ausgabe 3/2018

Multimedia Systems 3/2018 Zur Ausgabe

Neuer Inhalt