nach oben

Universal Access in the Information Society

Erschienen in:

03.07.2021 | Long Paper

Machine translation from text to sign language: a systematic review

verfasst von: Navroz Kaur Kahlon, Williamjeet Singh

Erschienen in: Universal Access in the Information Society | Ausgabe 1/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

An equal opportunity for all is the basic right of every human being. The deaf society of the world needs to have access to all the information just like hearing people do. For this to happen there should be a mode of direct communication between hearing and deaf people. The need at this time is to automate this communication so as the deaf society is not dependent upon human interpreters. This paper deals with the systematic survey of conventional and state-of-the-art sign language machine translation and sign language generation projects. We used a standard procedure of carrying out a systematic literature review on 148 studies published in 30 reputed journals and 40 premium conferences and workshops. Existing literature about sign language machine translation is broadly classified into three different categories. These categories are further sub-classified into different classifications depending upon the type of machine translation. Studies pertaining to the specified classifications have been presented with their advantages and limitations. Different methods for sign language generation are reported with their benefits and limitations. Manual and automatic evaluation methods used in every study is presented along with their respective performance metrics. We call for increased efforts in presentation of signs to make them an easy and comfortable mode of communication for deaf society. There is also a requirement to improve translation methods and include the contribution of advanced technologies such as deep learning and neural networks to make an optimal translation process between text and sign language.

Nächster Artikel Smart Workplaces for older adults: coping ‘ethically’ with technology pervasiveness

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

Boudlal, A., Lakhoujaja, A.M.: Alkhalil Morpho Sys1: a morphosyntactic analysis system for Arabic texts. In: International Arab Conference on Information Technology, pp. 1–6 (2010)

Achraf, O., Jemni, M.: Designing high accuracy statistical machine translation for sign language using parallel corpus: case study English and American sign language. J. Inf. Technol. Res. 12(2), 134–158 (2019). https://doi.org/10.4018/JITR.2019040108CrossRef

Al-barahamtoshy, O.H., Al-barhamtoshy, H.M.: Arabic text-to-sign (ArTTS ) model from automatic SR system. In: Proceedings of 3rd International Conference on Arab Computational Linguistics, pp. 304–311 (2017). https://doi.org/10.1016/j.procs.2017.10.122

Almeida, I.: Exploring challenges in avatar-based translation from European Portuguese to Portuguese sign language. Master’s Thesis. Instituto Superior Técnico (2014)

Almohimeed, A. et al.: Arabic text to Arabic sign language translation system for the deaf and hearing-impaired community. In: Proceedings of the second workshop on speech and language processing for assistive technologies, pp 101–109 (2011)

Anuja, K. et al.: Design and development of a frame based MT system for English-to-ISL. In: Proceedings of the World congress on nature and biologically inspired computing, pp. 1382–1387 (2009). https://doi.org/10.1109/NABIC.2009.5393721

Araújo, T.M.U.D., et al.: An approach to generate and embed sign language video tracks into multimedia contents. Inf. Sci. (Ny) 281, 762–780 (2014). https://doi.org/10.1016/j.ins.2014.04.008CrossRef

Badler, N.I., et al.: Animation control for real-time virtual humans. Commun. ACM. 42(8), 64–73 (1999). https://doi.org/10.1145/310930.310975CrossRef

Baldassarri, S. et al.: Automatic translation system to Spanish sign language with a virtual interpreter. In: Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction, pp. 196–199 (2009). https://doi.org/10.1007/978-3-642-03655-2_23

10.

Baldassarri, S., et al.: Maxine: a platform for embodied animated agents. Comput. Graph. 32(4), 430–437 (2008). https://doi.org/10.1016/j.cag.2008.04.006CrossRef

11.

Baldassarri, S., Royo-Santas, F.: An automatic rule-based translation system to Spanish sign language (LSE). In: New Trends in Human–Computer Interaction, pp. 1–11. https://doi.org/10.1007/978-1-84882-352-5_1

12.

Bangham, J.A. et al.: Signing for the deaf using virtual humans. In: IEE Seminar on Speech and Language Processing for the Disabled and Elderly People, pp. 1–5, London, UK (2000). https://doi.org/10.1049/ic:20000134

13.

Bangham, J.A. et al.: Virtual signing: capture, animation , storage and transmission—an overview of the ViSiCAST project. In: IEEE Seminar on Speech and Language Processing for Disabled and Elderly People, pp. 6/1–6/7 IEEE Press (2000). https://doi.org/10.1049/ic:20000136

14.

Bishop, C.M.: Mixture density networks. Technical report (1994)

15.

Boulares, M., et al.: Toward an example-based machine translation from written text to ASL using virtual agent animation. Int. J. Comput. Sci. Issues 9, 1 (2012)

16.

Bowden, R. et al.: Learning to recognise dynamic visual content from broadcast footage. https://cvssp.org/projects/dynavis/index.html. Last Accessed 2021/04/04

17.

Braffort, A., et al.: KAZOO: a sign language generation platform based on production rules. Univers. Access Inf. Soc. 15(4), 541–550 (2016). https://doi.org/10.1007/s10209-015-0415-2CrossRef

18.

Braffort, A. et al.: Virtual signer coarticulation in octopus, a sign language generation platform. In: Proceedings of the 9th International Gesture Workshop, Gesture in Embodied Communication and Human–Computer Interaction, pp. 29–32 (2011)

19.

Brour, M., Benabbou, A.: ATLASLang MTS 1: Arabic text language into Arabic sign language machine translation system. In: 2nd International Conference on Intelligent Computing in Data Sciences, pp. 236–245 (2019). https://doi.org/10.1016/j.procs.2019.01.066

20.

Brour, M., Benabbou, A.: ATLASLang NMT: Arabic text language into Arabic sign language neural machine translation. J. King Saud Univ. Comput. Inf. Sci. (2019). https://doi.org/10.1016/j.jksuci.2019.07.006CrossRef

21.

Bungeroth, J. et al.: The ATIS sign language corpus. In: Proceedings of the 6th International Conference on Language Resources and Evaluation, pp. 2943–2946 (2008)

22.

Bungeroth, J., Ney, H.: Statistical sign language translation. In: Workshop on Representation and Processing of Sign Languages, pp. 105–108 (2004)

23.

Buz, B., Gungor, T.: Developing a statistical Turkish sign language translation system for primary school students. In: IEEE International Symposium on Innovations in Intelligent SysTems and Applications, pp. 1–6 (2019). https://doi.org/10.1109/INISTA.2019.8778246

24.

Camgoz, N.C. et al.: Neural sign language translation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 7784–7793 (2018). https://doi.org/10.1109/CVPR.2018.00812

25.

Chan, C. et al.: Everybody dance now. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5933–5942 (2019)

26.

Chung, J. et al.: Empirical evaluation of gated recurrent neural networks on sequence modeling. Presented at the (2014)

27.

Coetzee, L. et al.: The national accessibility portal: an accessible information sharing portal for the South African disability sector. In: Proceedings of the International Cross-Disciplinary Conference on Web Accessibility, pp. 44–53. Banff, Canada (2007). https://doi.org/10.1145/1243441.1243456

28.

Cox, S., et al.: The development and evaluation of a speech-to-sign translation system to assist transactions. J. Hum. Comput. Interact. 16(2), 141–161 (2003). https://doi.org/10.1207/S15327590IJHC1602CrossRef

29.

Da, Q.L., et al.: Converting the vietnamese television news into 3D sign language animations for the deaf. In: Duong, T., Vo, N.S. (eds.) Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 257. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-05873-9_13CrossRef

30.

Dangsaart, S., et al.: Intelligent Thai text—Thai sign translation for language learning. Comput. Educ. 51(3), 1125–1141 (2008). https://doi.org/10.1016/j.compedu.2007.11.008CrossRef

31.

Dangsaart, S., Cercone, N.: Bridging the gap: Thai–Thai sign machine translation. In: Proceedings of the 10th Conference of the Pacific Association for Computational Linguistics, pp. 191–199 (2007)

32.

Dasgupta, T., Basu, A.: Prototype machine translation system from text-to-Indian sign language. In: Proceedings of the 13th International conference on Intelligent User Interfaces, pp. 313–316 (2008). https://doi.org/10.1145/1378773.1378818

33.

Davidson, M.J.: PAULA: a computer-based sign language tutor for hearing adults. In: Intelligent Tutoring Systems Workshop on Teaching with Robots, Agents, and Natural Language Processing, pp. 66–72 (2006)

34.

Delorme, M. et al.: Thumb modelling for the generation of sign language. In: Proceedings of the 9th International Conference on Gesture and Sign Language in Human–Computer Interaction and Embodied Communication, pp. 151–160 (2012). https://doi.org/10.1007/978-3-642-34182-3_14

35.

Denkowski, M., Lavie, A.: Meteor universal: language specific translation evaluation for any target language. In: Proceedings of 9th Workshop on Statistical Machine Translation, pp. 376–380 (2014). https://doi.org/10.3115/v1/W14-3348

36.

Doddington, G.: Automatic evaluation of machine translation quality using N-gram co-occurrence statistics. In: Proceedings of the 2nd International Conference on Human Language Technology Research, pp. 138–145 (2002)

37.

Duarte, A.: Cross-modal neural sign language translation. In: In Proceedings of the 27th ACM International Conference on Multimedia, pp. 1650–1654 (2019). https://doi.org/10.1145/3343031.3352587

38.

Ebling, S. et al.: SMILE Swiss German sign language dataset. In: Proceedings of the International Conference on Language Resources and Evaluation, pp. 19–25 (2018)

39.

Ebling, S., Huenerfauth, M.: Bridging the gap between sign language machine translation and sign language animation using sequence classification. In: Proceedings of the 6th Workshop on Speech and Language Processing for Assistive Technologies, pp. 2–9 (2015). https://doi.org/10.18653/v1/W15-5102

40.

Efthimiou, E. et al.: DICTA-SIGN: sign language recognition, generation, and modelling: a research effort with applications in deaf communication. In: Proceedings of the 4th Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, pp. 80–84 (2009). https://doi.org/10.1007/978-3-642-02707-9_3

41.

Efthimiou, E., et al.: Feature-based natural language processing for GSL synthesis. Sign Lang. Linguist. 10(1), 1–21 (2007). https://doi.org/10.1075/sll.10.1.03eftMathSciNetCrossRef

42.

Efthimiou, E., Dimou, S.F.A.: From grammar-based MT to post-processed SL representations. Univers. Access Inf. Soc. 15(4), 499–511 (2016). https://doi.org/10.1007/s10209-015-0414-3CrossRef

43.

Elliott, R. et al.: The development of language processing support for the ViSiCAST project. In: Proceedings of the 4th International ACM Conference on Assistive Technologies, pp. 101–108 (2000). https://doi.org/10.1145/354324.354349

44.

Eryiğit, C., et al.: Building machine-readable knowledge representations for Turkish sign language generation. Knowl.-Based Syst. 108, 179–194 (2016). https://doi.org/10.1016/j.knosys.2016.04.014CrossRef

45.

Eryiğit, G.: ITU Turkish NLP web service. In: Proceedings of the Demonstrations at the 14th Conference of the European Chapter of the Association for Computational Linguistics, pp. 1–4 (2014). https://doi.org/10.3115/v1/E14-2001

46.

Evita Fotinea, S., et al.: A knowledge-based sign synthesis architecture. Univers. Access Inf. Soc. 6(4), 405–418 (2008). https://doi.org/10.1007/s10209-007-0094-8CrossRef

47.

Fellbaum, C. (ed.): WordNet: An Electronic Lexical Database. MIT Press, Massachusetts (1998)MATH

48.

Filhol, M., et al.: A rule triggering system for automatic text-to-sign translation. Univers. Access Inf. Soc. 15(4), 487–498 (2016). https://doi.org/10.1007/s10209-015-0413-4CrossRef

49.

Filhol, M.: Combining two synchronisation methods in a linguistic model to describe sign language. In: Proceedings of the 9th International Conference on Gesture and Sign Language in Human–Computer Interaction and Embodied Communication, pp. 194–203 (2011). https://doi.org/10.1007/978-3-642-34182-3_18

50.

Glauert, J., et al.: Linguistic modelling and language-processing technologies for Avatar-based sign language presentation. Univers. Access Inf. Soc. 6(4), 375–391 (2008). https://doi.org/10.1007/s10209-007-0102-zCrossRef

51.

Glauert, J.R.W., et al.: VANESSA—a system for communication between deaf and hearing people. Technol. Disabil. 18(4), 207–216 (2006). https://doi.org/10.3233/TAD-2006-18408CrossRef

52.

Gough, N.: Example-based machine translation using the marker hypothesis. PhD thesis. Dublin City University (2005)

53.

Grieve-Smith, A.B.: English to American sign language machine translation of weather reports. In: Proceedings of the Second High Desert Student Conference in Linguistics. High Desert Linguistics Society, pp. 23–30 (1999)

54.

Grieve-Smith, A.B.: SignSynth: A sign language synthesis application using Web3D and Perl. In: Gesture and sign language based human–computer interaction, pp. 134–145. London, UK (2001). https://doi.org/10.1007/3-540-47873-6_14

55.

Hanke, T. et al.: Extending the public DGS corpus in size and depth. In: Proceedings of the 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives, pp. 75–82 (2020)

56.

Hanke, T.: HamNoSys—Representing sign language data in language resources and language processing contexts. In: Proceedings of the LREC workshop on the representation and processing of sign languages, pp. 1–6 (2004)

57.

Hanke, T., Popescu, H.: eSIGN deliverable D2.3: intelligent sign editor (2003)

58.

Heloir, A., Kipp, M.: Real-time animation of interactive agents: specification and realization. Appl. Artif. Intell. 24(6), 510–529 (2010). https://doi.org/10.1080/08839514.2010.492161CrossRef

59.

Hogan, C., Frederking, R.: An evaluation of the multi-engine MT architecture. In: Proceedings of the Conference of the Association for Machine Translation in the Americas, pp. 113–123 (1998). https://doi.org/10.1007/3-540-49478-2_11

60.

Huang, Z., Eli, A.: STEP: a scripting language for embodied agents. In: Proceedings of the Workshop of Lifelike Animated Agents, pp. 1–6 (2002)

61.

Huenerfauth, M.: A linguistically motivated model for speed and pausing in animations of American sign language. ACM Trans. Access. Comput. 2, 2 (2009). https://doi.org/10.1145/1530064.1530067CrossRef

62.

Huenerfauth, M.: A multi-path architecture for machine translation of English text into American sign language animation. In: Proceedings of the Student Workshop at the Human Language Technology Conference/North American Chapter of the Association for Computational Linguistics, pp. 25–30 (2004). https://doi.org/10.3115/1614038.1614043

63.

Huenerfauth, M.: American sign language generation: multimodal NLG with multiple linguistic channels. In: Proceedings of the ACL Student Research Workshop, pp. 37–42 (2005). https://doi.org/10.5555/1628960.1628968

64.

Huenerfauth, M.: An accessibility motivation for an English-to-ASL machine translation system (2004)

65.

Huenerfauth, M.: Generating American sign language classifier predicates for English-to-ASL machine translation. Ph.D thesis. University of Pennsylvania (2006)

66.

Huenerfauth, M.: Spatial representation of classifier predicates for machine translation into American sign language. In: Proceedings of the Workshop on the Representation and Processing of Signed Languages, 4th International Conference on Language Resources and Evaluation, pp. 24–31 (2004)

67.

Huenerfauth, M., Lu, P.: Effect of spatial reference and verb inflection on the usability of sign language animations. Univers. Access Inf. Soc. 11(2), 169–184 (2012). https://doi.org/10.1007/s10209-011-0247-7CrossRef

68.

Huenerfauth, M.P.: A survey and critique of American sign language natural language generation and machine translation systems. Technical report (2003)

69.

Isozaki, H. et al.: Automatic evaluation of translation quality for distant language pairs. In: Proceedings of Conference on Empirical Methods and Natural Language Processing, pp. 944–952 (2010). https://doi.org/10.5555/1870658.1870750

70.

Jemni, M. et al.: A Web-based tool to create online courses for deaf pupils. In: Proceedings of the International Conference on Interactive Mobile and Computer Aided Learning, pp. 1–8. Amman, Jordan (2007)

71.

Jemni, M., Elghoul, O.: A system to make signs using collaborative approach. In: International Conference on Computers for Handicapped Persons. Lecture Notes in Computer Science, pp. 670–677 (2008). https://doi.org/10.1007/978-3-540-70540-6_96

72.

Jemni, M., Elghoul, O.: Towards Web-based automatic interpretation of written text to sign language. In: Proceedings of the 1st International Conference on ICT & Accessibility, pp. 43–48. (2008).

73.

Jung, H.Y., et al.: Word reordering for translation into Korean sign language using syntactically-guided classification. ACM Trans. Asian Low-Resource Lang. Inf. Process. 19(2), 1–20 (2019). https://doi.org/10.1145/3357612CrossRef

74.

Kanis, J. et al.: Czech-sign speech corpus for semantic based machine translation. In: Sojka, P., Kopeček, I., Pala, K. (eds.) Proceedings of 9th International Conference on Text, Speech and Dialogue, pp. 613–620 (2006). https://doi.org/10.1007/11846406_77

75.

Kanis, J., Müller, L.: Automatic Czech—sign speech translation. In: Proceedings of 10th International Conference on Text, Speech and Dialogue, pp. 488–495 (2007). https://doi.org/10.1007/978-3-540-74628-7_63

76.

Kar, P. et al.: INGIT: Limited domain formulaic translation from Hindi strings to Indian sign language. In: International Conference on Natural Language Processing (2007)

77.

Karpouzis, K., Caridakis, G.: Educational resources and implementation of a Greek sign language synthesis architecture. Comput. Educ. 49(1), 54–74 (2007). https://doi.org/10.1016/j.compedu.2005.06.004CrossRef

78.

Katyana, Q.: Google’s neural network learns to translate languages it hasn’t been trained on. https://www.theregister.co.uk/2016/11/17/googles_neural_net_translates_languages_not_trained_on/, Last accessed 2020/09/23

79.

Kayahan, D., Gungor, T.: A hybrid translation system from Turkish spoken language to Turkish sign language. In: IEEE international symposium on innovations in intelligent systems and applications, pp. 1–6 (2019). https://doi.org/10.1109/INISTA.2019.8778347

80.

Kennaway, R.: Synthetic animation of deaf signing gestures. In: 4th International Workshop on Gesture and Sign Language Based Human–Computer Interaction, pp. 146–157 (2002). https://doi.org/10.1007/3-540-47873-6_15

81.

Khan, N.S., et al.: A novel natural language processing (NLP)—based machine translation model for English to Pakistan sign language translation. Cognit. Comput. 12, 748–765 (2020). https://doi.org/10.1007/s12559-020-09731-7CrossRef

82.

Kipp, M. et al.: Sign language avatars: animation and comprehensibility. In: Proceedings of the 10th International Conference on Intelligent Virtual Agents, pp. 113–126 (2011). https://doi.org/10.1007/978-3-642-23974-8

83.

Kitchenham, B., Charters, S.: Guidelines for performing systematic literature reviews in software engineering. Technical report EBSE-2007-01. (2007)

84.

Koehn, P. et al.: Moses: open source toolkit for statistical machine translation. In: Companion Volume to the Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, pp. 177–180 (2007)

85.

Koehn, P. et al.: Statistical phrase-based translation. In: Proceedings of the Human Language Technology and North American Association for Computational Linguistics Conference, pp. 48–54 (2003)

86.

Koleli, E.: A new Greek part-of-speech tagger, based on a maximum entropy classifier. Master’s thesis. Athens University of Economics and Business (2011)

87.

Kouremenos, D., et al.: A novel rule based machine translation scheme from Greek to Greek sign language: production of different types of large corpora and language models evaluation. Comput. Speech Lang. 51, 110–135 (2018). https://doi.org/10.1016/j.csl.2018.04.001CrossRef

88.

Kouremenos, D., et al.: A prototype Greek text to Greek sign language conversion system. Behav. Inf. Technol. 29(5), 467–481 (2010). https://doi.org/10.1080/01449290903420192CrossRef

89.

Kouremenos, D. et al.: Statistical machine translation for Greek to Greek sign language using parallel corpora produced via rule-based machine translation. In: IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), pp. 1–15 (2018)

90.

Kovar, L. et al.: Motion graphs. In: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, pp. 473–482 (2002). https://doi.org/10.1145/566570.566605

91.

Krnoul, Z. et al.: 3D symbol base translation and synthesis of Czech sign speech. In: Proceedings of the 11th International Conference on Speech and Computer, pp. 530–535 (2006)

92.

Krňoul, Z., Železný, M.: Translation and conversion for Czech sign speech synthesis. Lect. Notes Comput. Sci. 4629, 524–531 (2007). https://doi.org/10.1007/978-3-540-74628-7_68CrossRef

93.

Le, H.P. et al.: A hybrid approach to word segmentation of Vietnamese texts. In: Proceedings of 2nd International Conference on Language and Automata Theory and Applications, pp. 240–249 (2008). https://doi.org/10.1007/978-3-540-88282-4_23

94.

López-ludeña, V., et al.: Translating bus information into sign language for deaf people. Eng. Appl. Artif. Intell. 32, 258–269 (2014). https://doi.org/10.1016/j.engappai.2014.02.006CrossRef

95.

López-Ludeña, V., et al.: Automatic categorization for improving Spanish into Spanish Sign Language machine translation. Comput. Speech Lang. 26(3), 149–167 (2012). https://doi.org/10.1016/j.csl.2011.09.003CrossRef

96.

López-Ludeña, V. et al.: Factored translation models for improving a speech into sign language translation system. In: Proceedings of the Conference of the International Speech Communication Association, pp. 1605–1608 (2011)

97.

López-Ludeña, V., et al.: Increasing adaptability of a speech into sign language translation system. Expert Syst. Appl. 40(4), 1312–1322 (2013). https://doi.org/10.1016/j.eswa.2012.08.059CrossRef

98.

López-Ludeña, V., et al.: Methodology for developing an advanced communications system for the deaf in a new domain. Knowl.-Based Syst. 56, 240–252 (2014). https://doi.org/10.1016/j.knosys.2013.11.017CrossRef

99.

Lopez Ludeña, V. et al.: Methodology for developing a speech into sign language translation system in a new semantic domain. In: Proceedings of the Conference on IberSPEECH, pp. 193–203. Madrid, Spain (2012)

100.

Ludeña, V.L., San-segundo, R.: Statistical methods for improving Spanish into Spanish sign language translation. In: Proceedings of the 15th Mexican International Conference on Artificial Intelligence, pp. 1–11 (2016)

101.

Luong, M.T. et al.: Effective approaches to attention-based neural machine translation. In: Conference on Empirical Methods in Natural Language Processing, pp. 1412–1421 (2015). https://doi.org/10.18653/v1/d15-1166

102.

Luqman, H., Mahmoud, S.A.: Automatic translation of Arabic text-to-Arabic sign language. Univers. Access Inf. Soc. 18(4), 939–951 (2019). https://doi.org/10.1007/s10209-018-0622-8CrossRef

103.

Manzano, D.M.: English to ASL translator for SPEECH2SIGNS. (2018)

104.

Marshall, I., Safar, E.: Extraction of semantic representations from syntactic CMU link grammar linkages. In: Recent Advances in Natural Language Processing, pp. 154–159 (2001)

105.

Marshall, I., Safar, E.: Grammar Development for Sign Language Avatar-Based Synthesis . In: Proceedings of the 11th International Conference on Human Computer Interaction. (2005).

106.

Marshall, I., Safar, E.: Sign language generation in an ALE HPSG. In: Proceedings of the 11th International Conference on Head-Driven Phrase Structure Grammar, pp. 189–201 (2004)

107.

Marshall, I., Sáfár, É.: A prototype text to British sign language (BSL) translation system. In: 41st Annual Meeting of the Association for Computational Linguistics, pp. 113–116 (2003). https://doi.org/10.3115/1075178.1075194

108.

Mauser, A., Ney, H.: Automatic evaluation measures for statistical machine translation system optimization. In: Proceedings of the 6th International Conference on Language Resources and Evaluation, pp. 28–30

109.

Mazzei, A. et al.: Deep natural language processing for Italian Sign Language translation. In: Proceedings of the 13th Conference of the Italian Association for Artificial Intelligence, pp. 193–204 (2013). https://doi.org/10.1007/978-3-319-03524-6_17

110.

Mishra, G.S., et al.: Word based statistical machine translation from English text to Indian sign language. ARPN J. Eng. Appl. Sci. 12(2), 481–489 (2017)

111.

Mitkov, R. (ed.): The Oxford Handbook of Computational Linguistics. Oxford University Press, Oxford (2005)MATH

112.

Miyazaki, T. et al.: Proper name machine translation from Japanese to Japanese sign language. In: Language Technology for Closely Related Languages and Language Variants, pp. 67–75 (2014). https://doi.org/10.3115/v1/w14-4209

113.

Morrissey, S.: Assistive technology for deaf people: Translating into and animating Irish sign language. In: Proceedings of the 12th International Conference on Computers Helping People with Special Needs, pp. 8–14 (2008)

114.

Morrissey, S., Way, A.: An example-based approach to translating sign language. In: Workshop example-based machine translation (MT X-05), pp. 109–116 (2005)

115.

Morrissey, S., Way, A.: Joining hands: developing a sign language machine translation system with and for the deaf community. In: Proceedings of the Conference and Workshop on Assistive Technologies for People with Vision & Hearing Impairments, pp. 1–6 (2007)

116.

Morrissey, S., Way, A.: Lost in translation: the problems of using mainstream MT Evaluation metrics for sign language translation. In: Proceedings of the 5th SALTMIL Workshop on Minority Languages at Language Resources and Evaluation Conference, pp. 91–98 (2006)

117.

Morrissey, S., Way, A.: Manual labour: tackling machine translation for sign languages. Mach. Transl. 27(1), 25–64 (2013). https://doi.org/10.1007/s10590-012-9133-1CrossRef

118.

Nagao, M.: Framework of a mechanical translation between Japanese and English by analogy principle. In: Proceedings of the International NATO symposium on artificial and human intelligence, pp. 173–180 (1984). https://doi.org/10.7551/mitpress/5779.003.0038

119.

Needleman, S.B., Wunsch, C.D.: A general method applicable to the search for similarities in the amino acid sequence of two proteins. J. Mol. Biol. 48(3), 443–453 (1970). https://doi.org/10.1016/0022-2836(70)90057-4CrossRef

120.

Nguyen, T.B.D. et al.: A rule based method for text shortening in Vietnamese sign language translation. In: Information Systems Design and Intelligent Applications. Advances in Intelligent Systems and Computing (2018). https://doi.org/10.1007/978-981-10-7512-4_65

121.

Nießen, S. et al.: An evaluation tool for machine translation: fast evaluation for MT research. In: Proceedings of 2nd International Conference on Language Resources and Evaluation, pp. 39–45 (2000)

122.

Othman, A., Tmar, Z.: English-ASL gloss parallel corpus 2012: ASLG-PC12. In: 5th Workshop on the Representation and Processing of Sign Languages (2012)

123.

Otoom, M., Alzubaidi, M.A.: Ambient intelligence framework for real-time speech-to-sign translation. Assist. Technol. 30(3), 119–132 (2018). https://doi.org/10.1080/10400435.2016.1268218CrossRef

124.

Papageorgiou, H. et al.: A unified POS tagging architecture and its application to Greek. In: Proceedings of the 2nd Language Resources and Evaluation Conference, pp. 1455–1462 (2000)

125.

Papineni, K. et al.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002). https://doi.org/10.3115/1073083.1073135

126.

Porta, J., et al.: A rule-based translation from written Spanish to Spanish sign language glosses. Comput. Speech Lang. 28(3), 788–811 (2014). https://doi.org/10.1016/j.csl.2013.10.003CrossRef

127.

Quach, L., Nguyen, C.-N.: Conversion of the Vietnamese grammar into sign language structure using the example-based machine translation algorithm. In: International Conference on Advanced Technologies for Communications, pp. 27–31 (2018). https://doi.org/10.1109/ATC.2018.8587584

128.

Safar, E., Glauert, J.: Computer modelling. In: Pfau, R., et al. (eds.) Sign Language, pp. 1075–1102. De Gruyter Mouton, Berlin (2012). https://doi.org/10.1515/9783110261325.1075CrossRef

129.

Safar, E., Marshall, I.: The architecture of an English-text-to-sign-languages translation system. In: Recent Advances in Natural Language Processing, pp. 223–228. Bulgaria (2001)

130.

Sáfár, É., Marshall, I.: Sign language translation via DRT and HPSG. In: Proceedings of the 3rd International Conference on Intelligent Text Processing and Computational Linguistics, pp. 58–68 (2002). https://doi.org/10.1007/3-540-45715-1_5

131.

San-Segundo, R. et al.: A Spanish speech to sign language translation system for assisting deaf-mute people. In: Proceedings of the 9th International Conference on Spoken Language Processing, pp. 1399–1402 (2006)

132.

San-Segundo, R., et al.: Design, development and field evaluation of a Spanish into sign language translation system. Pattern Anal. Appl. 15(2), 203–224 (2012). https://doi.org/10.1007/s10044-011-0243-9MathSciNetCrossRef

133.

San-Segundo, R., et al.: Proposing a speech to gesture translation architecture for Spanish deaf people. J. Vis. Lang. Comput. 19(5), 523–538 (2008). https://doi.org/10.1016/j.jvlc.2007.06.002CrossRef

134.

San-Segundo, R., et al.: Speech to sign language translation system for Spanish. Speech Commun. 50(11–12), 1009–1020 (2008). https://doi.org/10.1016/j.specom.2008.02.001CrossRef

135.

Sandler, W., Lillo-Martin, D.: Sign Language and Linguistic Universals. J. Linguist. 42(3), 738–742 (2006). https://doi.org/10.1017/CBO9781139163910CrossRef

136.

Saunders, B. et al.: Adversarial training for multi-channel sign language production. In: Proceedings of the British Machine Vision Conference (2020)

137.

Saunders, B. et al.: Everybody sign now: translating spoken language to photo realistic sign language video (2020)

138.

Saunders, B. et al.: Progressive transformers for end-to-end sign language production. In: Proceedings of European Conference on Computer Vision, pp. 687–705 (2020). https://doi.org/10.1007/978-3-030-58621-8_40

139.

Selcuk-Simsek, M., Cicekli, I.: Bidirectional machine translation between Turkish and Turkish sign language: a data-driven approach. Int. J. Nat. Lang. Comput. 6(3), 33–46 (2017). https://doi.org/10.5121/ijnlc.2017.6303CrossRef

140.

Shieber, S.M., Yves, S.: Synchronous tree-adjoining grammars. In: Proceedings of the 13th International Conference on Computational Linguistics, pp. 253–258 (1990). https://doi.org/10.3115/991146.991191

141.

Sleator, D.D., Temperley, D.: Parsing English with a link grammar. Technical report CMU-CS-91-196 (1991)

142.

Snover, M. et al.: Study of translation edit rate with targeted human annotation. In: Proceedings of the 7th Conference of the Association for Machine Translation in the Americas, pp. 223–231. Cambridge, MA (2006)

143.

Souteh, Y., Bouzoubaa, K.: SAFAR platform and its morphological layer. In: Proceedings of the 11th Conference on Language Engineering, pp. 14–15 (2011)

144.

Speers, d’ A.L.: Representation of American sign language for machine translation. Ph.D thesis. Georgetown University (2001)

145.

Stein, D., et al.: Analysis, preparation, and optimization of statistical sign language machine translation. Mach. Transl. 26(4), 325–357 (2012). https://doi.org/10.1007/s10590-012-9125-1CrossRef

146.

Stein, D. et al.: Morpho-syntax based statistical methods for automatic sign language translation. In: Proceedings of the 11th Annual Conference of the European Association for Machine Translation, pp. 169–177. Oslo, Norway (2006)

147.

Stoll, S., et al.: Text2Sign: towards sign language production using neural machine translation and generative adversarial networks. Int. J. Comput. Vis. 128, 891–908 (2020). https://doi.org/10.1007/s11263-019-01281-2CrossRef

148.

Su, H.Y., Wu, C.H.: Improving structural statistical machine translation for sign language with small corpus using thematic role templates as translation memory. IEEE Trans. Audio Speech Lang. Process. 17(7), 1305–1315 (2009). https://doi.org/10.1109/TASL.2009.2016234MathSciNetCrossRef

149.

Suszczanska, N. et al.: Translating Polish texts into sign language in the TGT system. In: 20th IASTED International Multi-Conference on Applied Informatics, pp. 282–287 (2002)

150.

Sutton, C., McCallum, A.: An introduction to conditional random fields By Charles Sutton and Andrew McCallum. Found. Trends Mach. Learn. 4(4), 267–373 (2012). https://doi.org/10.1561/2200000013CrossRef

151.

Tillmann, C. et al.: Accelerated DP based search for statistical translation. In: Proceedings of the 5th European Conference on Speech Communication and Technology, pp. 2667–2670 (1997)

152.

Tokuda, M., Okumura, M.: Towards automatic translation from Japanese into Japanese sign language. Assist. Technol. Artif. Intell. Robot. User Interfaces Nat. Lang. Process. 1458, 97–108 (1998). https://doi.org/10.1007/bfb0055973CrossRef

153.

Toutanova, K. et al.: Feature-rich part-of-speech tagging with a cyclic dependency network. In: Conference of the North American Chapter of the Association for Computational Linguistics & Human Language Technologies on Human Language Technologies, pp. 173–180 (2003)

154.

Veale, T., et al.: The challenges of cross-modal translation: English-to-sign-language translation in the Zardoz system. Mach. Transl. 13(1), 81–106 (1998). https://doi.org/10.1023/A:1008014420317CrossRef

155.

Veale, T., Collins, B.: Space, metaphor and schematization in sign: sign language translation in the ZARDOZ system. In: Proceedings of the 2nd Conference of the Association for Machine Translation in the Americas, pp. 168–179 (1996)

156.

Veale, T., Conway, A.: Cross modal comprehension in ZARDOZ an English to sign-language translation system. In: 4th International Workshop on Natural Language Generation, pp. 67–72 (1994). https://doi.org/10.3115/1641417.1641450

157.

Ventura, L. et al.: Can everybody sign now? Exploring sign language video generation from 2D poses. In: Sign Language Recognition, Translation and Production Workshop (2020)

158.

Veríssimo, V.M., et al.: Towards an open platform for machine translation of spoken languages into sign languages. Mach. Transl. 33(4), 315–348 (2019). https://doi.org/10.1007/s10590-019-09238-5CrossRef

159.

Wang, Z., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861CrossRef

160.

Waterman, M., Smith, T.: Identification of common molecular subsequences. J. Mol. Biol. 147(1), 195–197 (1981). https://doi.org/10.1016/0022-2836(81)90087-5CrossRef

161.

Wray, A., et al.: A formulaic approach to translation at the post office: reading the signs. Lang. Commun. 24, 59–75 (2004). https://doi.org/10.1016/j.langcom.2003.08.001CrossRef

162.

Wu, C.H., et al.: Transfer-based statistical translation of Taiwanese sign language using PCFG. ACM Trans. Asian Lang. Inf. Process. 6(1), 1–18 (2007). https://doi.org/10.1145/1227850.1227851CrossRef

163.

Xu, W. et al.: End-to-end slot alignment and recognition for cross-lingual NLU. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 5052–5063 (2020)

164.

Zhao, L. et al.: A machine translation system from English to American sign language. In: Proceedings of the 4th Conference of the Association of Machine Translation, pp. 293–300 (2000). https://doi.org/10.1007/3-540-39965-8_6

165.

Van Zijl, L.: South African sign language machine translation project. In: Proceedings of the 8th International Conference on Computers and Accessibility, pp. 233–234 (2006). https://doi.org/10.1145/1168987.1169031

166.

Van Zijl, L., Combrink, A.: The South African sign language machine translation project: issues on non-manual sign generation. In: Proceedings of the Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists on IT Research in Developing Countries, pp. 127–134 (2006). https://doi.org/10.1145/1216262.1216276

167.

Van Zijl, L., Olivrin, G.: South African sign language assistive translation. In: Proceedings of the IASTED International Conference on Telehealth/Assistive Technologies, pp. 7–12 (2008)

168.

Zwitserlood, I. et al.: Synthetic signing for the deaf: esign. In: Proceedings of the conference and workshop on assistive technologies for vision and hearing impairment (2004)

169.

Blender Tool. https://www.blender.org/features/animation/. Last accessed 2020/09/04

170.

Curious Labs POSER. https://curious-labs-poser.software.informer.com/6.0/. Last accessed 2020/09/22

171.

ELAN annotation tool. https://www.mpi.nl/corpus/html/elan/. Last accessed 2020/08/09

172.

Google Neural Machine Translation. https://en.wikipedia.org/wiki/Google_Neural_Machine_Translation. Last accessed 2020/09/18

173.

Humanoid Animation. https://www.web3d.org/working-groups/humanoid-animation-hanim. Last accessed 2020/09/23

174.

Maya Tool. https://www.autodesk.in/products/maya/overview?plc=MAYA&term=1-YEAR&support=ADVANCED&quantity=1. Last accessed 2020/08/08

175.

Semantic Role Lists. http://elies.rediris.es/elies11/cap5111.htm. Last accessed 2021/04/05

176.

Unity 3D. https://unity.com/. Last accessed 2020/08/08

177.

VComD:Sign Smith Studio. http://www.vcom3d.com/signsmith.php. Last accessed 2010/08/08

178.

WebGL. https://www.khronos.org/webgl/. Last accessed 2021/04/12

Titel: Machine translation from text to sign language: a systematic review
verfasst von: Navroz Kaur Kahlon
Williamjeet Singh
Publikationsdatum: 03.07.2021
Verlag: Springer Berlin Heidelberg
Erschienen in: Universal Access in the Information Society / Ausgabe 1/2023
Print ISSN: 1615-5289
Elektronische ISSN: 1615-5297
DOI: https://doi.org/10.1007/s10209-021-00823-1

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 1/2023

Effective highlighting modes of graphical user interfaces in visual information search

Accessibility of university websites worldwide: a systematic literature review

Acknowledgement to reviewers for reviews completed during 2022

An analysis of the utility of digital materials for high school students with intellectual disability and their effects on academic success

Interaction of visual interface and academic levels with young students’ anxiety, playfulness, and enjoyment in programming for robot control

The impact of reading fluency level on interactive information retrieval

Premium Partner