Skip to main content
SpringerLink
Log in

Interactive translation prediction versus conventional post-editing in practice: a study with the CasMaCat  workbench

  • Published:
Machine Translation

Abstract

We conducted a field trial in computer-assisted professional translation to compare interactive translation prediction (ITP) against conventional post-editing (PE) of machine translation (MT) output. In contrast to the conventional PE set-up, where an MT system first produces a static translation hypothesis that is then edited by a professional (hence “post-editing”), ITP constantly updates the translation hypothesis in real time in response to user edits. Our study involved nine professional translators and four reviewers working with the web-based CasMaCat  workbench. Various new interactive features aiming to assist the post-editor/translator were also tested in this trial. Our results show that even with little training, ITP can be as productive as conventional PE in terms of the total time required to produce the final translation. Moreover, translation editors working with ITP require fewer key strokes to arrive at the final version of their translation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://www.matecat.com.

  2. In anticipation of the discussion of the CasMaCat  field trial in Sect. 4, we should also point out that editing is different from reviewing, in which a third party reviews and revises the text produced by the initial editor for quality assurance.

  3. In our experiments, we use an EyeLink1000 eye tracker.

  4. http://bridge.cbs.dk/platform/?q=CRITT_TPR-db.

  5. Outliers are defined here as those points that exceed \(Q3+1.5\) times the inter-quartile range, cf. Montgomery (2004).

  6. For a detailed introduction to this kind of analysis, see e.g. Montgomery (2004).

References

  • Alabau V, Leiva LA, Ortiz-Martínez D, Casacuberta F (2012) User evaluation of interactive machine translation systems. In: Proceedings of the 16th Annual Conference of the European Association for Machine Translation, pp 20–23

  • Alabau V, Buck C, Carl M, Casacuberta F, García-Martínez M, Germann U, González-Rubio J, Hill R, Koehn P, Leiva L, Mesa-Lao B, Ortiz-Martínez D, Saint-Amand H, Sanchis-Trilles G, Tsoukala C (2014) Casmacat: A computer-assisted translation workbench. In: Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pp 25–28

  • Alves F, Vale D (2009) Probing the unit of translation in time: aspects of the design and development of a web application for storing, annotating, and querying translation process data. Across Lang Cultures 10(2):251–273

    Article  Google Scholar 

  • Bach N, Huang F, Al-Onaizan Y (2011) Goodness: A method for measuring machine translation confidence. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics, pp 211–219

  • Barrachina S, Bender O, Casacuberta F, Civera J, Cubel E, Khadivi S, Lagarda AL, Ney H, Tomás J, Vidal E, Vilar JM (2009) Statistical approaches to computer-assisted translation. Comput Linguist 35(1):3–28

    Article  MathSciNet  Google Scholar 

  • Brown PF, Della Pietra SA, Della Pietra VJ (1993) The mathematics of statistical machine translation: parameter estimation. Comput Linguist 19(2):263–311

    Google Scholar 

  • Callison-Burch C, Koehn P, Monz C, Post M, Soricut R, Specia L (2012) Findings of the 2012 workshop on statistical machine translation. In: Proceedings of the Seventh Workshop on Statistical Machine Translation, pp 10–51

  • Carl M (2012a) The CRITT TPR-DB 1.0: A database for empirical human translation process research. In: Proceedings of the AMTA 2012 Workshop on Post-Editing Technology and Practice, pp 1–10

  • Carl M (2012b) Translog-II: a program for recording user activity data for empirical reading and writing research. In: Proceedings of the Eighth International Conference on Language Resources and Evaluation, pp 4108–4112

  • Carl M (2014) Produkt- und Prozesseinheiten in der CRITT Translation Process Research Database. In: Ahrens B (ed) Translationswissenschaftliches Kolloquium III: Beiträge zur Übersetzungs- und Dolmetschwissenschaft (Köln/Germersheim). Peter Lang, Frankfurt am Main, pp 247–266

  • Carl M, Kay M (2011) Gazing and typing activities during translation : a comparative study of translation units of professional and student translators. Meta 56(4):952–975

    Article  Google Scholar 

  • Doherty S, O’Brien S, Carl M (2010) Eye tracking as an MT evaluation technique. Mach Transl 24(1):1–13

    Article  Google Scholar 

  • Elming J, Carl M, Balling LW (2014) Investigating user behaviour in post-editing and translation using the Casmacat workbench. In: O’Brien S, Winther Balling L, Carl M, Simard M, Specia L (eds) Post-editing of machine translation: processes and applications. Cambridge Scholar Publishing, Newcastle upon Tyne, pp 147–169

    Google Scholar 

  • Federico M, Cattelan A, Trombetti M (2012) Measuring user productivity in machine translation enhanced computer assisted translation. In: Proceedings of the Tenth Biennial Conference of the Association for Machine Translation in the Americas

  • Flournoy R, Duran C (2009) Machine translation and document localization at adobe: From pilot to production. In: Proceedings of MT Summit XII

  • Green S, Heer J, Manning CD (2013) The efficacy of human post-editing for language translation. In: Proceedings of SIGCHI Conference on Human Factors in Computing Systems, pp 439–448

  • Guerberof A (2009) Productivity and quality in mt post-editing. In: Proceedings of MT Summit XII-Workshop: Beyond Translation Memories: New Tools for Translators MT

  • Guerberof A (2012) Productivity and quality in the post-editing of outputs from translation memories and machine translation. Ph.D. Thesis

  • Just MA, Carpenter PA (1980) A theory of reading: from eye fixations to comprehension. Psychol Rev 87(4):329

    Article  Google Scholar 

  • Koehn P (2009a) A process study of computer-aided translation. Mach Transl 23(4):241–263

    Article  Google Scholar 

  • Koehn P (2009b) A web-based interactive computer aided translation tool. In: Proceedings of ACL-IJCNLP 2009 Software Demonstrations, pp 17–20

  • Krings HP (2001) Repairing texts: empirical investigations of machine translation post-editing processes, vol 5. Kent State University Press, Kent

    Google Scholar 

  • Lacruz I, Shreve GM, Angelone E (2012) Average pause ratio as an indicator of cognitive effort in post-editing: a case study. In: Proceedings of the AMTA 2012 Workshop on Post-Editing Technology and Practice, pp 21–30

  • Langlais P, Foster G, Lapalme G (2000) Transtype: A computer-aided translation typing system. In: Proceedings of the 2000 NAACL-ANLP Workshop on Embedded Machine Translation Systems, pp 46–51

  • Leiva LA, Alabau V, Vidal E (2013) Error-proof, high-performance, and context-aware gestures for interactive text edition. In: Proceedings of the 2013 annual conference extended abstracts on Human factors in computing systems, pp 1227–1232

  • Montgomery D (2004) Introduction to statistical quality control. Wiley, Hoboken

    Google Scholar 

  • O’Brien S (2009) Eye tracking in translation process research: methodological challenges and solutions, Copenhagen Studies in Language, vol 38. Samfundslitteratur, Copenhagen, pp 251–266

    Google Scholar 

  • Ortiz-Martínez D, Casacuberta F (2014) The new Thot toolkit for fully automatic and interactive statistical machine translation. In: Proceedings of the 14th Annual Meeting of the European Association for Computational Linguistics: System Demonstrations, pp 45–48

  • Plitt M, Masselot F (2010) A productivity test of statistical machine translation post-editing in a typical localisation context. Prague Bulletin Math Linguist 93(1):7–16

    Google Scholar 

  • Sanchis-Trilles G, Ortiz-Martínez D, Civera J, Casacuberta F, Vidal E, Hoang H (2008) Improving interactive machine translation via mouse actions. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp 485–494

  • Simard M, Foster G (2013) Pepr: Post-edit propagation using phrase-based statistical machine translation. In: Proceedings of MT Summit XIV, pp 191–198

  • Skadiņš R, Puriņš M, Skadiņa I, Vasiļjevs A (2011) Evaluation of SMT in localization to under-resourced inflected language. In: Proceedings of the 15th International Conference of the European Association for Machine Translation, pp 35–40

Download references

Acknowledgments

This work was supported by the European Union’s 7th Framework Programme (FP7/2007–2013) under grant agreement No 287576 (CasMaCat ).

Author information

Authors and Affiliations

  1. Pattern Recognition and Human Language Technologies Center, Universitat Politècnica de València, Valencia, Spain

    Germán Sanchis-Trilles, Vicent Alabau, Francisco Casacuberta, Jesús González-Rubio, Luis A. Leiva , Daniel Ortiz-Martínez & Enrique Vidal

  2. School of Informatics, University of Edinburgh, Edinburgh, UK

    Christian Buck, Ulrich Germann, Robin L. Hill, Philipp Koehn, Herve Saint-Amand & Chara Tsoukala

  3. Department of International Business Communication, Copenhagen Business School, Copenhagen, Denmark

    Michael Carl, Mercedes García-Martínez & Bartolomé Mesa-Lao

Authors
  1. Germán Sanchis-Trilles
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vicent Alabau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Buck
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Carl
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Francisco Casacuberta
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mercedes García-Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ulrich Germann
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jesús González-Rubio
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Robin L. Hill
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Philipp Koehn
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Luis A. Leiva
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Bartolomé Mesa-Lao
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Daniel Ortiz-Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Herve Saint-Amand
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Chara Tsoukala
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Enrique Vidal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Germán Sanchis-Trilles.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sanchis-Trilles, G., Alabau, V., Buck, C. et al. Interactive translation prediction versus conventional post-editing in practice: a study with the CasMaCat  workbench. Machine Translation 28, 217–235 (2014). https://doi.org/10.1007/s10590-014-9157-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10590-014-9157-9

Keywords

Search

Navigation