Numbers in simultaneous interpreting: a multimodal analysis
Simultaneous interpreting (SI) is considered to be a highly demanding cognitive task, especially in regard to elements which represent a stumbling block during SI (e.g. numbers, idiomatic expressions, etc.). The current study examines the use of numbers in SI in combination with co-speech gestures. The issue of interpreting numbers has been addressed by different scholars, however it has rarely been studied from the multimodal perspective so far. Given the known role of gesture during processes of lexical retrieval, we hypothesize that the use of co-speech manual gestures plays a significant role in the process of interpreting numbers, i.e. facilitates mental retrieval of the target number. In addition to the hypothesis to be tested, we consider the exploratory question as to whether any gestures with numbers correlate more with an externalized or internalized function, that is: either representation of the numerical concept externally (metaphorical depiction of the quantified entity as an object or space that could be touched or pointed to) or more with self-adapters to help the interpreter manage the cognitive load/stress internally. A corpus of 10 recordings of English-to-Russian SI sessions was analyzed. The interpreting of numbers was analyzed as being correct, incorrect or absent. The analysis of the material showed that the participants mostly interpreted numbers correctly or omitted them in their speech. The results of the analysis of co-speech gestures demonstrated nearly equal use of gesture and no gesture when interpreting numbers, however, there was a significant correlation between interpreting certain number categories and gesture use. Then the following types of gesture functions were analyzed: adapters, pragmatic, representational and deictic. The results indicate that self-adapters were the most frequent gesture type used. This finding can be interpreted based on the literature showing self-adapters ease tension and anxiety and/or help one gain control of stressful situations.
Figures
Cienki, A., Leonteva, A. V., Agafonova, O. V. and Petrov, A. A. (2023). Numbers in simultaneous interpreting: a multimodal analysis, Research Result. Theoretical and Applied Linguistics, 9 (1), 82-98. DOI: 10.18413/2313-8912-2023-9-1-0-6
While nobody left any comments to this publication.
You can be first.
Braun, S. and Clarici, A. (1996). Inaccuracy for numerals in simultaneous interpretation: Neurolinguistic and neuropsychological perspectives, The Interpreters’ Newsletter, 7, 85–102. (In English)
Bylinina, L. and Nouwen, R. (2020). Numeral semantics, Language and Linguistics Compass, 14, e12390. https://doi.org/10.1111/lnc3.12390(In English)
Chu, M. and Kita, S. (2008). Spontaneous gestures during mental rotation tasks: Insights into the microdevelopment of the motor strategy, Journal of Experimental Psychology: General, 137, 706–723. https://doi.org/10.1037/a0013157(In English)
Cienki, A. and Iriskhanova, O. K. (2020). Patterns of multimodal behavior under cognitive load: An analysis of simultaneous interpretation from L2 to L1, Voprosy Kognitivnoy Lingvistiki [Issues in Cognitive Linguistics], 1, 5–11. https://doi.org/10.20916/1812-3228-2020-1-5-11(In English)
Desmet, B., Vandierendonck, M. and Defrancq, B. (2018). Simultaneous interpretation of numbers and the impact of technological support, inFantinuoli, C. (ed.), Interpreting and technology, Language Science Press, Berlin, Germany, 13–27. https://doi.org/10.5281/zenodo.1493281(In English)
Dressel, D. (2020). Multimodal word searches in collaborative storytelling: On the local mobilization and negotiation of coparticipation, Journal of Pragmatics, 170, 37–54. https://doi.org/10.1016/j.pragma.2020.08.010(In English)
Ekman, P. and Friesen, W. V. (1969). The repertoire of non-verbal behavior: Categories, origins, usage, and coding, Semiotica, 1, 49–98. https://doi.org/10.1515/semi.1969.1.1.49(In English)
Freedman, N. (1972). The analysis of movement behavior during the clinical interview, in Siegman, A. W. and Pope, B. (eds.), Studies in dyadic communication, 7, 153–175. (In English)
Fricke, E. (2002). Origo, pointing, and speech: the impact of co-speech gestures on linguistic deixis theory,Gesture, 2 (2), 207–226. https://doi.org/10.1075/gest.2.2.05fri(In English)
Fricke, E. (2013). Towards a unified grammar of gesture and speech: A multimodal approach, in Müller, C., Cienki, A., Fricke, E., Ladewig, S., McNeill, D. and Tessendorf, S. (eds.), Body – Language – CommunicationDe Gruyter Mouton, Berlin, Germany 733–754. https://doi.org/10.1515/9783110261318.733(In English)
Gibbon, D. (2005). Prerequisites for a multimodal semantics of gesture and prosody, Proceedings of the International Workshop on Computational Semantics, 6, 2–15. (In English)
Gile, D. (1995). Fidelity assessment in consecutive interpretation: an experiment,Target. International Journal of Translation Studies, 7 (1), 151–164. https://doi.org/10.1075/target.7.1.12gil(In English)
Gile, D. (2008). Local cognitive load in simultaneous interpreting and its implications for empirical research, FORUM.Revue internationale d’interprétation et de traduction/International Journal of Interpretation and Translation, 6 (2), 59–77. https://doi.org/10.1075/forum.6.2.04gil(In English)
Gile, D. (2009). Basic concepts and models for interpreter and translator training, John Benjamins, Amsterdam/Philadelphia, Netherlands/USA. https://doi.org/10.1075/btl.8(In English)
Gósy, M. (2007). Disfluencies and Self-monitoring, Govor = Speech, XXIV, 91–110. (In English)
Hirrel, L. (2018). , Doctoral dissertation, The University of New Mexico, Albuquerque, New Mexico, USA. (In English)
Kendon, A. (2017). Pragmatic functions of gestures: Some observations on the history of their study and their nature,Gesture, 16 (2), 157–175. https://doi.org/10.1075/gest.16.2.01ken(In English)
Kita, S., Alibali, M. W. and Chu, M. (2017). How do gestures influence thinking and speaking? The gesture-for-conceptualization hypothesis, Psychological Review, 14 (3), 245-266. http://dx.doi.org/10.1037/rev0000059(In English)
Korpal, P. (2017). Linguistic and psychological indicators of stress in simultaneous interpreting, Wydawnictwo Naukowe UAM, Poznan, Poland. (In English)
Korpal, P. and Stachowiak-Szymczak, K. (2020). Combined problem triggers in simultaneous interpreting: Exploring the effect of delivery rate on processing and rendering numbers,Perspectives, 28 (1), 126–143. https://doi.org/10.1080/0907676X.2019.1628285(In English)
Ladewig, S. H. (2014). The cyclic gesture, Body–language–communication: An international handbook on multimodality in human interaction, , 1605–1618. (In English)
Lakoff, G. and Núñez, R. (2000).Where Mathematics Comes From: How the Embodied Mind Brings Mathematics into Being, New York, NY, USA. (In English)
Lapaire, J.-R. (2016). From ontological metaphor to semiotic make-believe: Giving shape and substance to fictive objects of conception with the “globe gesture”, Signo, 41 (70), 29–44. https://doi.org/10.17058/signo.v41i70.6413(In English)
Lin, W., Orton, I., Liu, M. and Mahmoud, M. (2020). Automatic detection of self-adaptors for psychological distress, 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), 371–378. https://doi.org/10.1109/FG47880.2020.00032(In English)
Mandler, G. and Shebo, B. J. (1982). Subitizing: an analysis of its component processes,Journal of experimental psychology: General, 111 (1), 1–22. https://doi.org/10.1037/0096-3445.111.1.1(In English)
Mazza, C. (2001). Numbers in simultaneous interpretation, The Interpreters’ Newsletter, 11, 87–104. (In English)
McNeill, D. (1985). So you think gestures are nonverbal? Psychological Review, 92 (3), 350-371. https://doi.org/10.1037/0033-295X.92.3.350(In English)
McNeill, D. (1992). Hand and mind: What gestures reveal about thought, University of Chicago Press, Chicago, USA. (In English)
Mead, P. (2015). Numbers, in Pöchhacker, F. (ed.), Routledge encyclopedia of interpreting studies, London/New York, UK/USA, 286–288. https://doi.org/10.4324/9781315678467(In English)
Müller, C. (1998). Iconicity and gesture,Oralité et gestualité. Communication multimodale, interaction, L’Harmattan, Paris, France, 321–328. (In English)
Müller, C. (2014). Gestural modes of representation as techniques of depiction, in Müller, C., Cienki, A., Fricke, E., Ladewig, S. H., McNeill, D., and Bressem, J. (eds.), Body – Language – Communication, Berlin, Germany 1687–1702. (In English)
Neff, M., Toothman, N., Bowmani, R., Fox Tree, J. E. and Walker, M. A. (2011). Don’t scratch! Self-adaptors reflect emotional stability, in Vilhjálmsson, H. H., Kopp, S., Marsella, S. and Thorisson, K. R. (eds.), Intelligent Virtual Agents, IVA 2011, Lecture Notes in Computer Science, 6895, 398-411. Reykjavik, Iceland, 398–411. https://doi.org/10.1007/978-3-642-23974-8_43(In English)
Nikolaeva, J. V. (2017). Pragmatic gestures in Russian retellings of "The Pear Stories", The Russian Journal of Cognitive Science, 4 (2-3), 6–12. (In English)
Pinochi, D. (2009). Simultaneous interpretation of numbers: Comparing German and English to Italian. An experimental study, The Interpreters’ Newsletter, 14, 33–57. (In English)
Poyatos, F. (1987/2002). Nonverbal communication in simultaneous and consecutive interpretation: A theoretical model and new perspectives, in Pöchhacker, F. and Shlesinger, M. (eds.), The Interpreting Studies Reader, Routledge, London, UK, 235–246. (In English)
Seeber, K. (2011). Cognitive load in simultaneous interpreting: Existing theories - New models, Interpreting, 13, 176–204. https://doi.org/10.1075/intp.13.2.02see(In English)
Seeber, K. G. (2012). Multimodal input in simultaneous interpreting: An eye-tracking experiment, in Zybatow, L. N., Petrova, A. and Ustaszewski, M. (eds.), Proceeding of the 1st International Conference TRANSLATA, Translation and Interpreting Research: Yesterday–Today–Tomorrow, Peter Lang, Frankfurt a.M., Germany, 341–347. https://doi.org/10.3726/978-3-653-01625-3(In English)
Seeber, K. G. (2015). Simultaneous interpreting, in Mikkelson, H. and Jourdenais, R. (eds.), The Routledge Handbook of Interpreting, Routledge Handbooks Online. DOI:10.4324/9781315745381.ch5 (In English)
Slobin, D. I. (1987). Thinking for speaking, Proceedings of the Thirteenth Annual Meeting of the Berkeley Linguistics Society, 13, 435–445. https://doi.org/10.3765/bls.v13i0.1826(In English)
Stachowiak-Szymczak, K. (2019). Eye Movements and Gestures in Simultaneous and Consecutive Interpreting, Springer, New York, USA. https://doi.org/10.1007/978-3-030-19443-7(In English)
Streeck, J. (2009). Forward-gesturing, Discourse Processes, 46 (2-3), 161–179. https://doi.org/10.1080/01638530902728793(In English)
Timarová, Š. (2012). Working memory in simultaneous interpreting, PhD Thesis, University of Leuven, Leuven, Belgium. https://doi.org/10.1075/intp.16.2.01tim(In English)
The research is financially supported by the Russian Science Foundation, Project 19-18-00357 “Verbal and co-verbal behavior under cognitive load: analyses of speech, gesture, and eye gaze” at Moscow State Linguistic University.