Silent, but salient: gestures in simultaneous interpreting
Salience is regarded as one of the key concepts for cognitive studies of language and communication, however there is limited research on how prominence plays out in multimodal discourse. The present study is aimed at investigating how salience comes through in gestures used by simultaneous interpreters. To distinguish between salient and non-salient gestures by the same participant, two groups of observable parameters were chosen – basic and auxiliary. An empirical study was carried out, based on simultaneous interpreting of the audio of a TED talk (English ® Russian). The video recordings were integrated into ELAN files and annotated for salient gestures, the functions that were realized by them, and the elementary discourse units (EDUs) that the gestures co-occurred with. It was assumed that, first, salient gestures will be observed less frequently than non-salient gestures; second, prominence in gestures will serve the function of representing various aspects of a situation more often than other functions; third, salient gestures will co-occur more often with elementary discourse units (EDU) containing verb phrases, rather than noun phrases. The hypotheses were partially confirmed via quantitative and qualitative analyses which demonstrated that every third gesture was salient; the representative function came second after the pragmatic functions; there was no significant difference between the number of gestures used with verbal and nominal EDUs, though it was observed that salient gestures tend to co-occur with the verbs of physical actions and negation, as well as with the nouns accompanied by attributes denoting high degree of a quality.
Figures
Iriskhanova, O. K., Cienki, A., Tomskaya, M. V. and Nikolayeva, A. I. (2023). Silent, but salient: gestures in simultaneous interpreting, Research Result. Theoretical and Applied Linguistics, 9 (1), 99-114. DOI: 10.18413/2313-8912-2023-9-1-0-7
While nobody left any comments to this publication.
You can be first.
Beattie, G., Webster, K. and Ross, J. (2010). The fixation and processing of the iconic gestures that accompany talk, Journal of Language and Social Psychology, 29 (2), 1-30 https://doi.org/10.1177/0261927X09359589(In English)
Boutet, D. (2010). Structuration physiologique de la gestuelle: Modèle et tests, Lidil, 42, 77-96. https://doi.org/10.4000/lidil.3070(In English)
Boutet, D., Morgenstern, A. and Cienki, A. (2016). Grammatical aspect and gesture in French: A kinesiological approach, Russian Journal of Linguistics, 20 (3), 132-151. (In English)
Bressem, J. (2013). A linguistic perspective on the notation of form features in gestures, in Müller, C., Cienki, A., Fricke, E., Ladewig, S., McNeill, D. and Teßendorf, S. (eds.), Body – Language – Communication: An international handbook on multimodality in human interaction, Vol. 1, Mouton de Gruyter, Berlin, Germany, 1079-1098. (In English)
Cavicchio, F. and Kita, S. (2013). Bilinguals switch gesture production parameters when they switch languages, Proceedings of the Tilburg Gesture Research Meeting (TIGeR 2013), Tilburg University, Netherlands, 305-309.(In English)
Cienki, A. (2013). Cognitive linguistics: Spoken language and gesture as expressions of conceptualization, in Müller, C., Cienki, A., Fricke, E., Ladewig, S., McNeill, D. and Teßendorf, S. (eds.), Body – Language – Communication: An international handbook on multimodality in human interaction, Vol. 1, Mouton de Gruyter, Berlin, Germany, 182-201. https://doi.org/10.1515/9783110261318.182 (In English)
Cienki, A. (2021). From the finger lift to the palm-up open hand when presenting a point: A methodological exploration of forms and functions, Languages and Modalities, 1 (1), 17-30, https://doi.org/10.3897/lamo.1.68914(In English)
Cienki, A. and Iriskhanova, O. K. (2020). Patterns of multimodal behavior under cognitive load: An analysis of simultaneous interpretation from L2 to L1, Voprosy Kognitivnoy Lingvistiki, 1, 5-11. https://doi.org/10.20916/1812-3228-2020-1-5-11(In English)
Cienki, A. and Mittelberg, I. (2013). Creativity in the forms and functions of spontaneous gesture with speech, in Veale, T., Feyaerts, K. and Forceville, C. (eds.), The Agile Mind: A Multi-disciplinary Study of a Multi-faceted Phenomenon, Mouton de Gruyter, Berlin, Germany, 231-252. (In English)
Cienki, A. and Müller, C. (2008). Metaphor, gesture and thought, in Gibbs, R. W. Jr. (ed.), The Cambridge Handbook of Metaphor and Thought, Cambridge University Press, Cambridge, UK, 484-501. (In English)
Eisenstein, J., Barzilay, R. and Davis, R. (2007). Turning lectures into comic books using linguistically salient gestures, Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, Vancouver, British Columbia, Canada, 877-882. (In English)
Fedorova, O. V. and Zherdev, I. Ju. (2019). Follow the hands of the interlocutor! (On strategies for the distribution of visual attention), Experimental Psychology, 12 (1), 98-118. https://doi.org/10.17759/exppsy.2019120108 (In English)
Gile, D. (1997). Conference interpreting as a cognitive management problem, in Danks, J. H., Fountain, S. B., McBeath, M. K. and Shreve, G. M. (eds.), Cognitive Processes in Translation and Interpreting, Sage Publications, Thousand Oaks, London, New Delhi, India, 196-214. (In English)
Givon, T. (1985). Iconicity, Isomorphism, and non-arbitrary coding in syntax, in Haiman, J. (ed.), Iconicity in Syntax, Benjamins, Amsterdam, The Netherlands, 187-219. (In English)
Grishina, E. A. (2017). Russkaya zhestikulyaciya s lingvisticheskoj tochki zreniya (korpusnye issledovaniya) [Russian gesticulation in linguistic aspect: Corpus studies], Yazyki slavyanskoj kul'tury, Izdatel'skij Dom YASK, Moscow, Russia. (In Russian)
Gullberg, M. and Holmqvist, K. (1999). Keeping an eye on gestures: Visual perception of gestures in face-to-face communication, Pragmatics & Cognition, 7 (1), 35-63. https://doi.org/10.1075/pc.7.1.04gul (In English)
Gullberg, M. and Holmqvist, K. (2002). Visual attention towards gestures in face-to-face interaction vs. on screen, in Wachsmuth, I. and Sowa, T. (eds.), Gesture and Sign Language Based Human-Computer Interaction, Springer-Verlag, Berlin, Germany, 206-214. (In English)
Haiman, J. (1985). Natural Syntax, Cambridge University Press, Cambridge, UK. (In English)
Iriskhanova, O. K. and Cienki, A. (2018). The semiotics of gestures in cognitive linguistics: Contributions and challenges, Voprosy Kognitivnoy Lingvistiki, 4, 25-36. https://doi.org/10.20916/1812-3228-2018-4-25-36(In English)
Kendon, A. (2004). Gesture: Visible action as utterance, Cambridge University Press, Cambridge, UK. (In English)
Kibrik, A. A. and Podlesskaya, V. P. (eds.) (2009). Rasskazy o snovideniyakh: korpusnoye issledovaniye ustnogo russkogo diskursa [Nightdream stories: case study of Russian spoken discourse], Languages of Slavic culture, Moscow, Russia. (In Russian)
Langacker, R. (2000). Grammar and Conceptualization, Mouton de Gruyter, Berlin, Germany. (In English)
Marian, V., Blumenfield, H. K. and Kaushanskaya, M. (2007). The language experience and proficiency questionnaire (LEAP-Q) assessing language profiles in bilinguals and multilinguals, Journal of Speech, Language, and Hearing Research, 50, 940-967. https://doi.org/10.1017/S1366728919000038 (In English)
McNeill, D. (1992). Hand and mind: What gestures reveal about thought, University of Chicago Press, Chicago, USA. (In English)
McNeill, D. (2005). Gesture and thought, University of Chicago Press, Chicago, USA. (In English)
Müller, C. (1998). Redebegleitende Gesten. Kulturgeschichte – Theorie – Sprachvergleich [Speech-accompanying gestures. Cultural history – theory – language comparison], Berlin Verlag, Berlin, Germany. (In German)
Müller, C. (2016). From mimesis to meaning: A systematics of gestural mimesis for concrete and abstract referential gestures, in Zlatev, J., Sonesson, G. and Konderak, P. (eds.), Meaning, mind and communication: Explorations in cognitive semiotics, Peter Lang, Frankfurt am Main, Germany. (In English)
Müller, C. and Tag, S. (2010). The dynamics of metaphor: Foregrounding and activating metaphoricity in conversational interaction, Cognitive Semiotics, 10 (6), 85-120. https://doi.org/10.1515/cogsem.2010.6.spring2010.85 (In English)
O’Connor, M. and Cienki, A. (2022). The materiality of lines: The kinaesthetics of bodily movement uniting dance and prehistoric cave art, Frontiers in Communication, 7. https://doi.org/10.3389/fcomm.2022.956967(In English)
Oakley, T. (2009). From attention to meaning: explorations in semiotics,Linguistics and Rhetoric, Peter Lang, Bern–Berlin, Germany. (In English)
Oben, B. and Brône, G. (2015). What you see is what you do: on the relationship between gaze and gesture in multimodal alignment, Language and Cognition, 7 (04), 546-562. https://doi.org/10.1017/langcog.2015.22(In English)
Seeber, K. G. (2013). Cognitive load in simultaneous interpreting, Target, 25 (1), 18-32. https://doi.org/10.1075/target.25.1.03see (In English)
Stachowiak-Szymczak, K. (2019). Eye movements and gestures in simultaneous and consecutive interpreting, Springer, Basel, Switzerland. (In English)
Sweetser, E. (2007). Looking at space to study mental spaces: Co-speech gesture as a crucial data source in cognitive linguistics, in Gonzalez-Marquez, M., Mittelberg, I., Coulson, S. and Spivey, M. J. (eds.), Methods in Cognitive Linguistics, John Benjamins, Amsterdam, Netherlands, 201-224. (In English)
Talmy, L. (1978). Figure and ground in complex sentences, in Greenberg, J. (ed.), Universals of Human Language, Stanford University Press, Stanford, USA, 625-649. (In English)
Tellier, M., Stam, G. and Ghio, A. (2021). Handling language: How future language teachers adapt their gestures to their interlocutor, Gesture, 20 (1), 30-62. https://doi.org/10.1075/gest.19031.tel (In English)
Wagner, P., Malisz, Z. and Kopp, S. (2014). Gesture and speech in interaction: An overview, Speech Communication, 57, 209-232. https://doi.org/10.1016/j.specom.2013.09.008 (In English)
The research was carried out at Moscow State Linguistic University and supported by the Russian Science Foundation (Grant No 19-18-00357)