Skip to main content
Log in

A corpus-based approach to the multimodal analysis of specialized knowledge

  • Original Paper
  • Published:
Language Resources and Evaluation Aims and scope Submit manuscript

Abstract

Modern communication environments have changed the cognitive patterns of individuals, who are now used to the interaction of information encoded in different semiotic modalities, especially visual and linguistic. Despite this, the main premise of Corpus Linguistics is still ruling: our perception of and experience with the world is conveyed in texts, which nowadays need to be studied from a multimodal perspective. Therefore, multimodal corpora are becoming extremely useful to extract specialized knowledge and explore the insights of specialized language and its relation to non-language-specific representations of knowledge. It is our assertion that the analysis of the image-text interface can help us understand the way visual and linguistic information converge in subject-field texts. In this article, we use Frame-based terminology to sketch a novel proposal to study images in a corpus rich in pictorial representations for their inclusion in a terminological resource on the environment. Our corpus-based approach provides the methodological underpinnings to create meaning within terminographic entries, thus facilitating specialized knowledge transfer and acquisition through images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. The main objective of the MarcoCosta and EcoSistema research projects was to create a single information space for frame-based environmental data in the form of a visual thesaurus of specialized concepts in the domains of integrated coastal management and environmental science.

  2. VariMed is a research project about the cognitive and cultural patterns underlying denominative variation in medical communication. It will result in a multimodal lexical resource in the field of health care with visual support for linguistic research, translation, and technical writing for knowledge dissemination.

References

  • Allwood, J. (2008). Multimodal corpora. In A. Lüdeling & M. Kytö (Eds.), Corpus linguistics: An international handbook (pp. 207–225). Berlin: Mouton de Gruyter.

    Google Scholar 

  • Baddeley, A. D. (1986). Working memory. Oxford: Oxford University Press.

    Google Scholar 

  • Baldry, A. P., & Thibault, P. J. (2006). Multimodal corpus linguistics. In G. Thompson & S. Hunston (Eds.), System and corpus exploring connections (pp. 164–183). London and New York: Equinox.

    Google Scholar 

  • Baldry, A. P., & Thibault, P. J. (2008). Applications of multimodal concordances. Journal of Language and Communication Studies, 41, 11–41.

    Google Scholar 

  • Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.

    Article  Google Scholar 

  • Bowker, L. (2002). Computer-aided translation technology. A practical introduction. Ottawa: University of Ottawa Press.

    Google Scholar 

  • Bowker, L., & Howkins, S. (2006). Variation in the organization of medical terms. Exploring some motivations for term choice. Terminology, 12(1), 79–110.

    Article  Google Scholar 

  • Carney, R. N., & Levin, J. R. (2002). Pictorial illustrations still improve students’ learning from text. Educational Psychology Review, 14(1), 5–26.

    Article  Google Scholar 

  • Faber, P. (2009). The cognitive shift in terminology and specialized translation. MonTI. Monografías de Traducción e Interpretación, no., 1, 107–134.

    Article  Google Scholar 

  • Faber, P. (2011). The dynamics of specialized knowledge representation: Simulational reconstruction or the perception–action interface. Terminology, 17(1), 9–29.

    Article  Google Scholar 

  • Faber, P., León Araúz, P., Prieto Velasco, J. A., & Reimerink, A. (2007). Linking images and words: The description of specialized concepts. International Journal of Lexicography, 20(1), 39–65.

    Article  Google Scholar 

  • Faber, P., Márquez Linares, C., & Vega, M. (2005). Framing terminology: A process-oriented approach. Meta: Journal des traducteurs/Meta: Translators’ Journal, 50(4).

  • Faber, P., Montero Martínez, S., Castro Prieto, M. R., Senso Ruiz, J., Prieto Velasco, J. A., León Araúz, P., et al. (2006). Process-oriented terminology management in the domain of coastal engineering. Terminology, 12(2), 189–213.

    Article  Google Scholar 

  • Fillmore, C. (1982). Frame semantics. In The Linguistic Society of Korea (Ed.), Linguistics in the morning calm (pp. 111–137). Seoul: Hanshin.

    Google Scholar 

  • Fillmore, C. (1985). Frames and the semantics of understanding. Quaderni di Semantica, 6(2), 222–254.

    Google Scholar 

  • Fillmore, C., & Atkins, B. (1992). Towards a frame-based organization of the lexicon: The semantics of RISK and its neighbors. In A. Lehrer & E. Kittay (Eds.), Frames, fields, and contrasts: New essays in semantics and lexical organization (pp. 75–102). Hillsdale: Lawrence Erlbaum.

    Google Scholar 

  • Firth, J. R. (1957). A synopsis of linguistic theory 1930–1955. Studies in Linguistic Analysis (special volume of the Philological Society) (pp. 1–32).

  • Freixa, J. (2006). Causes of denominative variation in terminology: A typology proposal. Terminology, 12(1), 51–77.

    Article  Google Scholar 

  • Jewitt, C. (Ed.). (2009). Handbook of multimodal analysis. London: Routledge.

    Google Scholar 

  • Kress, G., & Van Leeuwen, T. (2002a). Reading images: The grammar of visual design. London: Routledge.

    Google Scholar 

  • Kress, G. R., & Van Leeuwen, T. (2002b). Multimodal discourse: The modes and media of contemporary communication. London: Edward Arnold.

    Google Scholar 

  • León Araúz, P., & Faber, P. (2010). Natural and contextual constraints for domain-specific relations. In V. Barbu Mititelu, V. Pekar, & E. Barbu (Eds.), Proceedings of the Workshop Semantic Relations. Theory and Applications (pp. 12–17). Valletta.

  • Liu, Y., & O’Halloran, K. L. (2009). Intersemiotic texture: Analyzing cohesive devices between language and images. Social Semiotics, 19(4), 367–387.

    Article  Google Scholar 

  • López Rodríguez, C. I. (2001). Tipologías textuales y cohesión en la traducción biomédica inglés-español: un estudio de corpus. PhD Thesis. University of Granada. Available at http://www.ugr.es/~dpto_ti/profesores/cilr-tesis.html. Accessed Feb 2012.

  • López Rodríguez, C. I. (2009). Extracción y representación de conocimiento a partir de corpus. In E. Valero & E. Rambla Alcina (Eds.), Terminología y sociedad del conocimiento (pp. 341–379). Bern: Peter Lang.

    Google Scholar 

  • López Rodríguez, C. I., Buendía Castro, M., & García Aragón, A. (2012). User needs to the test: Evaluating a terminological knowledge base on the environment by trainee translators. Jostrans. The Journal of Specialized Translation, 18, 57–76.

    Google Scholar 

  • Louwerse, M. M., & Jeuniaux, P. (2010). The linguistic and embodied nature of conceptual processing. Cognition, 114, 96–104.

    Article  Google Scholar 

  • Marsh, E. E., & White, M. D. (2003). Taxonomy of relationships between images and text. Journal of Documentation, 59(6), 647–672.

    Article  Google Scholar 

  • Martinec, R., & Salway, A. (2005). A system for image-text relations in new (and old) media. Visual Communication, 4(3), 337–371.

    Article  Google Scholar 

  • McEnery, A., & Wilson, A. (2001). Corpus linguistics. Edinburgh: Edinburgh University Press.

    Google Scholar 

  • Meyer, I., & Mackintosh, K. (1996). The corpus from a terminographer’s viewpoint. International Journal of Corpus Linguistics, 1(2), 257–285.

    Article  Google Scholar 

  • O’Halloran, K. L., & Smith B. A. (accepted for publication). Multimodal text analysis. In C. A. Chapelle (Ed.). Encyclopedia of applied linguistics. New Jersey: Wiley-Blackwell.

  • O’Halloran, K. L., Tan, S., Smith, B. A., & Podlasov, A. (2010). Challenges in designing digital interfaces for the study of multimodal phenomena. Information Design Journal, 18(1), 2–12.

    Article  Google Scholar 

  • Paivio, A. (1971). Imagery and verbal processes. New York: Holt, Rinehart, and Winston.

    Google Scholar 

  • Prieto Velasco, J. A. (2008). Información gráfica y grados de especialidad en el discurso científico-técnico: un estudio de corpus. PhD Thesis, University of Granada. Available at: http://0-hera.ugr.es.adrastea.ugr.es/tesisugr/17491332.pdf. Accessed Feb 2012.

  • Prieto Velasco, J. A. (2009). Traducción e imagen: la información visual en textos especializados. Granada: Tragacanto.

    Google Scholar 

  • Prieto Velasco, J. A., & López Rodríguez, C. I. (2009). Managing graphic information in terminological knowledge bases. Terminology, 15(2), 179–213.

    Article  Google Scholar 

  • Reimerink, A., García de Quesada, M., & Montero Martínez, S. (2010). Contextual information in terminological knowledge bases: A multimodal approach. Journal of Pragmatics, 42(7), 1928–1950.

    Article  Google Scholar 

  • Temmerman, R. (2000). Towards new ways of terminology description: The sociocognitive-approach. Amsterdam/Philadelphia: John Benjamins.

    Google Scholar 

  • Temmerman, R., & Kerremans, K. (2003). Termontography: Ontology building and the sociocognitive approach to terminology description. Prague CIL17-conference.

  • Tercedor, M. I. (2011). The cognitive dynamics of terminological variation. Terminology, 17(2), 181–197.

    Article  Google Scholar 

  • Unsworth, L. (2008). Multimodal semiotics: Functional analysis in contexts of education. London and New York: Continuum.

    Google Scholar 

Download references

Acknowledgments

This research is funded by the Spanish Ministry of Economy and Competitiveness under the project FFI2011-23120.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Juan Antonio Prieto Velasco.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Prieto Velasco, J.A. A corpus-based approach to the multimodal analysis of specialized knowledge. Lang Resources & Evaluation 47, 399–423 (2013). https://doi.org/10.1007/s10579-012-9204-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10579-012-9204-1

Keywords

Navigation