1. Introduction

Simultaneous interpreting (SI) is regarded as a cognitively demanding task which requires a lot of effort from interpreters. According to the “Tightrope Hypothesis” (Gile, 1995) different processes and requirements emerging during the working process involve nearly the entire cognitive processing capacity of the speaker. As Gile mentions, when total processing capacity requirements are high, interpreters might find it difficult to perform the task. Such situations can occur due to the quality of the source speech e. g. it can be too fast, heavily accented, etc., or it can contain words which are regarded to be difficult to interpret such as unfamiliar names, idiomatic expressions, numbers etc. The overall saturation which might appear due to such triggers can lead to different errors and influence the quality of the target speech (Gile, 2008).

1.1 Simultaneous interpreting and cognitive load

In the literature, the process of SI is commonly viewed from the perspective of the Effort Model (Gile, 2008), а cognitive framework which presents SI as cognitive online operations. Such operations can be grouped into “Listening (and Analysis) Effort”, e.g. operations which help interpreters comprehend the source speech; “Production Effort”, aimed at producing target speech, using self-monitoring and self-correction; and “Memory Effort” operations responsible for storage and retrieval of information in short-term memory; “Coordination Effort” is responsible for attention distribution and shifts between the aforementioned types.

Cognitive load is a multidimensional phenomenon that presumes the existence of a strain on mental processes (i.e., attention, thinking, working memory, etc.) while performing a cognitive task. SI is one such task that is highly demanding of cognitive resources (Gósy, 2007; Dayter, 2020). There is a peculiarity of SI, however: while perceiving the input signal (the source text), the interpreters need to produce the output signal (the interpreting). This process includes information decoding (from one language to the other) and involves different types of mental processes working at the same time: attention, thinking, perception, reasoning and memory. Сognitive load may result in speech disfluencies in SI (Stachowiak-Szymczak, 2019). According to the Cognitive Load Model by K. Seeber (Seeber, 2011) and D. Gile’s Effort Model (Gile, 2009) interpreting incorporates a language comprehension task, a language production task, memory storage and a coordination task. Achieving balance between these cognitive challenges requires self-management in order to reduce cognitive load. Hand gestures can serve such a function, along with interactive and communicative functions (Poyatos, 1987/2002).

Research performed by K. Seeber (Seeber, 2012) shows that simultaneous interpreters are searching for information that is complementary to the speech, which can be available in the form of visual stimuli (e.g., hand / body movements of the speaker, additional information given on the slides of a presentation). This is relevant especially while translating numerals, taking into account that they are associated with an increase of cognitive load.

1.2 Cognitive load and gesture use

Gestures have been demonstrated to play various roles in relation to how people manage a cognitive load. One way is in how they help speakers manage the concepts they are working with. While McNeill’s (1985, 1992) work particularly focused on the role gestures play as people formulate their ideas when speaking, Kita et al. (2017) argue that gestures have a broader self-oriented function, namely in helping people with processes of conceptualization, particularly if they involve spatial concepts. For example, participants in one experiment (Chu and Kita, 2008) viewing unusual three-dimensional objects had to picture what they would look like if rotated in certain directions; the participants made significant use of gestures while imagining rotating the objects, even though they did not speak during the task.

Another well-known function of gestures in assisting people manage a heavy cognitive load is in helping them maintain focus and even calming themselves. Here what are called self-adapters (Ekman and Friesen, 1969) play a fundamental role. Such body-focused movements (Freedman, 1972), such as rubbing one’s hands, stroking one’s hair, etc., play a role not only in calming oneself down, but also in gathering one’s ideas and maintaining a mental focus (Neff et al., 2011). In this regard, gesture use has an internal-regulating function, when one is experiencing various kinds of cognitive stress in general (Lin et al., 2020), and specifically when engaged in the process of SI, as noted in Cienki and Iriskhanova (2020).

1.3 Numbers in interpreting

Lexical units used for naming numbers and numerals are a special category which lies between language and mathematics. Such words are an interesting case of shaping and expressing these abstract categories from our mind into speech. Our cognitive ability to subitize, i.e., to understand the number of things from the first glance, is regarded to be inborn (Mandler and Shebo, 1982). We also have several cognitive capacities that help us to deal with numbers. We have numerosity, which is reflected in our ability to estimate roughly the number of objects in a group; grouping, which allows us to group objects mentally, visually, etc.; ordering and pairing as we put objects into a specific order and group them; memory capacity to keep track of objects that are being counted; exhaustion-detection capacity to detect the absence of objects to be counted; cardinal-number assignment to assign a cardinal number of the group of objects to estimate its size; independent-order capacity to understand that the assigned cardinal number is independent of the order of the counted objects; combinatorial-grouping capacity to put small groups into larger units; and a symbolizing capacity to associate words with numbers (Lakoff and Núñez, 2000).

In general, as we know, numbers are used to count, as this is the first thing we learn about numbers as children through the counting sequence in one’s native language. Numbers express cardinality, which is applied to some set of objects. However, it is argued that cardinality is not the only property of numbers. Numbers possess a certain flexibility, as there are no objects, imagined or real, that they cannot be applied to. They can be applied to objects in three ways: cardinal (used with sets of objects, e.g. five boys), ordinal (used with objects, that are a part of some progression, e.g. the fifth boy) and nominal (used with objects, that are a part of a set, e.g. boy #5). The role of number words is still being debated in linguistics due to their semantics and relations to the context. There are different semantic types of numerals. Numerals are regarded as determiners, modifiers, number-denoting words, and degree quantifiers, and are also used to denote exhaustivity and scope (Bylinina and Nouwen, 2020).

There are two types of representation which are involved in the processing of numbers: syntactic and lexical. Syntactic representation specifies the number of digits involved in writing numbers of that category (e.g. tens, thousands, etc.). Lexical representation adds to the syntactic by specifying the basic quantity (from 1 to 9).

There are various categories which give rise to objective difficulties for interpreters in their work. These categories are known as problem triggers and result in high error rates, compared to the interpretation of other words (Gile, 2009). Numbers are regarded as one of the main categories that pose a difficulty to interpreters of any level (Mazza, 2001; Mead, 2015; Korpal, 2017).

Interpreting numbers can be performed in different ways: when an interpreter is alone (stays isolated in the booth), they can write down numbers; they can use complementary information given on slides; an interpreter can render prepared speech (e. g. a lecture or any other type of prepared talk); when an interpreter in not alone in the booth, their colleague can write down numbers and then they can be read off. A study conducted by C. Collard and B. Defranq with interpreters in the European Parliament showed that error rates for numbers amounted to approximately 18% (Desmet et al., 2018).

There are different reasons why numbers might be difficult to interpret. Numbers are regarded to be very informative as they have neither redundant material (Seeber, 2015), nor a conceptual representation (Timarová, 2012; Seeber, 2015); when we imagine a number, we usually imagine some objects that have this number, rather than the abstract notion itself. Thus, they are less predictable for interpreters and it creates a difficulty (Pinochi, 2009; Mead, 2015). Due to this, interpreters cannot use such strategies as reformulation or paraphrasing, as components of numbers can have only one exact meaning (Pinochi, 2009). It causes a switch between types of interpreting: from ‘intelligent’ to ‘literal’ interpreting (Braun and Clarici, 1996), which might cause more errors.

Numbers might also be difficult to interpret due to the source text and differences in language structures (Pinochi, 2009). If a number is uttered, the chance for it to be held for interpretation later in the utterance is minimal. Moreover, numbers carry precise pieces of information: when this piece of information is omitted or misinterpreted, the communication process may suffer (Korpal and Stachowiak-Szymczak, 2020). Simultaneous interpreting of numbers is considered to be difficult in all languages (Pinochi, 2009).

In the current study we explore: (1) the interpreters’ verbal behavior while they are rendering numbers from the source text into their target language (whether there are more cases of correct interpreting of numbers than incorrect); (2) which subcategory of interpreting numbers is more frequent: ‘correct’, ‘approximation’, ‘absent’ or ‘wrong’; (3) if interpreting numbers is mostly accompanied by gestures, i.e., whether there is a correlation between such categories as ‘correct’, ‘approximation’, ‘wrong’ and the presence of a gesture.

2. Material and methods

The analysis is based on material taken from 10 videos, approximately 10 minutes each, with each produced by a different interpreter. The total video corpus comprises 100 minutes. The participants performed the SI from English into Russian (from L2 into L1). The interpreters were master’s students; beginners, who had been interpreting for less than 3 years; and experienced professionals, who had been interpreting for more than 3 years. They each heard the same portion of a popular science lecture (a TED Talk) in English, only hearing the audio through headphones and not seeing the video (so as not to be influenced by the original speakers’ gestures). Due to the initial purposes of our studies, the interpreters were not allowed to take any objects (e.g. pen and paper) to write down any remarks, including numbers, during the process. Though this is a common practice in the interpreting field, our study focused on how interpreters would handle a high cognitive load using only ‘natural media’ (Gibbon, 2005), including the potential role of gesture with speech. Allowing use of any implements or technology would change the nature of the interpreters’ processes of thinking for speaking (Slobin, 1987), which would have fundamentally altered the nature of the research project.

The video material was recorded using three cameras. A GoPro camera was installed in front of the participants to record the frontal view of the speaker, which allowed us to see their gestures precisely in order to annotate them later. For even better precision, for example in case the interpreter put his/her hands on their knees under the table, a Sony camera was installed on a tripod behind them to see such gestures. The third camera was in the eye-tracking glasses, Tobii Pro II, which the participants had on during the whole process of interpreting. These glasses showed the first-person perspective and allowed the coders to double check hand movements of the speakers (see Figure 1).

Figure 1. The three perspectives that were recorded: frontal, over-the-shoulder, and from the eye-tracking glasses

Рисунок 1. Расположение камер, на которые осуществлялась запись: на столе напротив переводчика, в верхнем углу переводческой кабины и камера на очках-айтрекере

After the videos were recorded, they were annotated in ELAN, a special program used to analyze verbal and nonverbal behavior (https://archive.mpi.nl/tla/elan). For the purposes of our study, we analyzed numerals in terms of their assignment to the object (cardinal, ordinal or nominal) and their use in the source text compared to how they were interpreted by the participants. Then the interpretations of the numbers were put into two major categories according to their accuracy: correct and incorrect. Under ‘correct interpreting’ we understand the direct translation of the source number e. g. five hundred species – пятьсотен/пятьсотживотных or five tons of weight – пятитоннный. The correct category means that the idea is being interpreted without any corruptions and is expressed in similar categories in the target language. The ‘incorrect’ category has three subcategories: approximation, wrong and absent. The first subcategory is assigned to the number if it was interpreted with some corruption in its meaning, thus the idea is partly lost e. g. one hundred years – нескольколет; millions and millions years ­– стомиллионовлет; 1598 (year) – вшестнадцатомвеке. The second category, wrong, was applied in case the main idea of the source number was lost and the word used in the target language had a completely different meaning, e. g. 1844 (year) – вдвадцатомвеке; sixty – шесть; thousands and thousands – сотниисотни. The last category, absent, was marked if the numeral from the source text was not interpreted by a participant at all, thus was absent in their speech.

The analysis of the nonverbal behavior of the participants included the study of their manual co-speech gestures. There were several highlighted types of gestures, taken into account: adapters, representational, pragmatic and deictic.

Adapters are such gestures that have no referential semantic function. They consist of simple movements like scratching oneself, rubbing one’s own fingers, touching one’s hair, etc. However, even without any semantic contribution to the speech, such gestures have an important role as they can show the increase in cognitive load, as speakers often use such gestures to deal with a stressful situation by regaining cognitive control over it (Freedman, 1972). For instance, the speaker in Figure 2 is rubbing her hands during the interpreting. Such small gestures are quite typical for this type of activity, as they are not very visible, but might help to cope with the stress.

Figure 2. Adapter gesture

Рисунок 2. Жест самоадаптер

«десять тысяч птиц»

“ten thousand (birds)”

Representational gestures are hand movements that have semantic meaning conveyed by their form and/or movement. Such gestures are based on the notion of iconicity as they resemble the object/notion they refer to. The way they resemble the referent is usually called a “mode of representation” (Müller, 1998; 2014). In our study we distinguish five modes: holding, molding, embodying, acting and tracing. In the example shown in Figure 3, the speaker uses a holding gesture when representing a number. The holding mode of representing things nonverbally is one of the most frequently used in speech as it can have a metaphorical meaning of a container (see McNeill’s, 1992, discussion of metaphoric gestures and Lapaire, 2016), which helps to put one’s idea into a form. This can be of aid to interpreters, as the representation of the subject might lead to a better search for the equivalent lexical unit in the target language.

Figure 3. Example of a representational gesture

Рисунок3. Репрезентирующий жест

«восемь сотен»

“eight hundred”

Pragmatic gestures are the type of co-speech hand movements that have discourse-related properties, like emphasizing verbal units, helping in searching for words (Streeck, 2009; Ladewig, 2014; Hirrel, 2018; Nikolaeva, 2017; Dressel, 2020) or structuring some parts of discourse (Fricke, 2013); they can also be used to express one’s attitude or evaluation of the subject being talked about. As it is represented in Figure 4, the speaker is using her left hand, raised up, palm open, as she is translating the number one hundred. This gesture can have several discourse functions, as it might help search for the words to translate the construction, but also be as-if presenting part of the discourse, which is being interpreted; the form does not relate iconically to the referent mentioned simultaneously in the speech.

Figure 4. Example of a pragmatic gesture

Рисунок 4. Прагматический жест

«сто (видов)»

“one hundred (species)”

Deictic gestures are different from the types described above in term of their ability to create vectors in space in order to show the location of a concrete or abstract notion. These can be such hand movements as pointing or touching gestures, as they are used to show the direction through the vector, created by hand with the speaker as the starting point or “origo” (Fricke, 2002). As it can be observed in Figure 5, the speaker is touching the table in front of her when enumerating objects. These movements accompany the speech and as they help to locate the notions in space in front of the interpreter; they might bear the function of facilitating the interpreting process by making the objects as if tangible, thus creating an order in correspondence with growth of the amount of objects.

Figure 5. Examples of deictic gestures

Рисунок 5. Дейктический жест

For the purposes of our study, we conducted a semantic analysis of the speech in order to compare the interpreting of the numbers from the source language (English) into the target language of the participants (Russian). Then the analysis of the nonverbal behavior was performed, as gestures were analyzed for their form and semantic properties. The final step was the statistical analysis of the correlation between the categories of gestures and numbers, except for the ‘absent’ category, as it was impossible to predict the exact possible occurrence of the untranslated number in speech, which made it impossible to line it up with any possible corresponding gesture. We examined the co-occurrence of gestures and the aforementioned types of number interpreting using the Pearson x² criterion as it shows the linear dependence of the two categories.

3. Results

The total amount of the numbers appearing during the interpreting was 438 units. Of these, 277 units were interpreted correctly and 160 incorrectly: 98 units were absent, 43 were wrong and 19 were interpreted approximately. Mostly cardinal numbers were omitted, including simple numbers (like 2 and 5) and also more complex ones (50,000, millions and millions). The tendency to omit cardinal numbers might be explained by the absence of entire phrases which included the numbers, as sometimes the interpreters omitted some parts of the source text. However, it cannot be determined whether such gaps were provoked by the presence of numbers in them or were difficult to interpret in general.

The category ‘approximation’, though not frequent in our results, displayed a particular tendency throughout the interpreting of different participants, as they interpreted the same two phrases using approximation, e.g., from half a species to four species; thousands and thousands (of pages). However, these phrases were also sometimes interpreted incorrectly (our category of ‘wrong’). The results of the analysis of this category also showed the predominance of cardinal numbers, but it also showed quite a number of cases when the number of a year (as a time period) was misinterpreted, e.g. in 1884 – в 1848 (this same mistake was made by three of the interpreters)/ в 1544 году. However, such results might be influenced by the predominance of cardinal numbers in the source text in general, in comparison to ordinal or nominal numbers.

The analysis of gesture use revealed that, for the most part, numbers in interpreting were accompanied by gestures (169 cases), however the amount of cases when numbers were used without gestures is not significantly different (117 cases). The most frequent gesture type was adapters (107 cases). Other types were used less frequently: pragmatic gestures (42 cases), representational (13 cases) and deictic (7 cases).

The analysis of the correspondence of gesture types to the interpreting categories showed that the ‘correct’ category was mostly accompanied by no gestures (117 cases) and by adapters (86 cases). It was also the only category accompanied by representational (13 cases) and deictic gestures (7 cases). Similar results were shown by the analysis of the other two categories. The ‘approximation’ category was not accompanied by gestures in 10 cases, and accompanied by adapters (7 cases) and pragmatic gestures (3 cases), whereas the ‘wrong’ category was mostly accompanied by gestures (in 27 out of the 43 instances): of these, 13 cases involved the use of pragmatic gestures and 14 cases – the use of adapters.

The last step was to establish any correlations existing between gestures and categories of interpreting. A statistical analysis was performed to establish any linear correlations (Pearson x²) between the presence/absence of gesture and interpreting categories ‘correct’, ‘wrong’ and ‘approximation’. The results showed the correlation between the presence of a gesture and interpreting categories ‘correct’ (Pearson x² = 0.7) and ‘wrong’ (Pearson x² = 0.86) categories.

During the analysis it was revealed that similar representational gestures can be used with the same lexical units in different interpreting. In the examples below, we can observe that different participants used an embodying gesture (Figure 6) when talking about two positions. The idea behind the gesture is the division of the two notions, showing them metaphorically and through space highlighting the opposition between the positions and their difference.

Figure 6. Embodying gestures

Рисунок 6. Репрезентирующий жест (олицетворение)

Another case of embodying was found in the material which can be called a classical representation of a numeral. The interpreter in Figure 7 used her index finger extended on two occasions when talking about one (thing). The depiction can relate to both category of one, highlighting it, as well as to its referent, embodying it.

Figure 7. Embodying gestures

Рисунок 7. Репрезентирующий жест (олицетворение)

In another case (Figure 8), the participants used a molding gesture when interpreting the phrase 3D model. This gesture here reflects not the numeral, but rather the object it refers to, a model. The idea of a model as something that has physical borders which can be touched and even altered might be common background knowledge shared by people and reflected in the gesture use.  

Figure 8. Molding gestures

Рисунок 8. Репрезентирующий жест (лепка / придание формы)

4. Discussion

The results of the study support the ideas of previous research on numbers in interpreting, which regard them as words difficult to interpret, as nearly half of the numbers (160) were interpreted incorrectly. When the participants seemed to have difficulties in the interpreting, they chose to omit numbers (97 cases) or to use approximation (20 cases). This can indicate that omitting is a more preferable tactic for dealing with the rise in cognitive load during the interpreting.

The results also showed a statistical correlation between one category of interpreting and gesture type. A high correlation between the ‘wrong’ category and the presence of gesture might indicate the importance of hand movements during the interpreting process, especially when a speaker faces a moment of difficulty. The correlation with the ‘correct’ category showed lower results, but they are still sufficient to say that there might be a dependence on gestures, in that gestures might support the idea of speech facilitation during the interpreting. The results, obtained with the ‘approximation’ category might show that either there is not enough data or that this category does not really depend on the presence or absence of a gesture in general and there can be additional factors which influence it.

As it was mentioned above, representational and deictic gestures co-occurred only with the ‘correct’ interpreting of numbers. These results might support the idea of the growth point, introduced by D. McNeill (1992). That is, McNeill argues that idea units unfold in relation to each other moment by moment as they are expressed in speech and gesture. In terms of the present findings, when the interpreters have the number and whatever is being quantified correctly in mind and are able to interpret that correctly, this provides a coherent ‘image’ that has the potential to also appear in gesture. While this was apparently not the situation in the majority of the cases (117 correct interpretations of numbers were not accompanied by any gestures), it is known from previous research (including McNeill, 1992) that gestures are used more often when presenting new information than when speaking about something deemed by the speaker to be already known or less noteworthy. Therefore, the representational gestures may have just come into play when the interpreters were presenting the quantities or objects that were seen as key new information, e.g., один вид, две позиции (one species, two positions). In any case, the point remains that they would need to be able to mentally simulate the referent in some coherent way in order to have a gesturable image.

This contrasts with the cases when the interpretation of numbers was approximate or missing. When the interpreters have some difficulty with the number and/or what is being referred to, it does not provide a coherent growth point of an idea, and so is not capable of being rendered in a representational gesture or conceptualized as something that can be pointed to with a deictic gesture. In these cases, we see the pragmatic gestures coming into use. Rather than depicting or indicating a referent’s location, pragmatic gestures often concern discursive functions, such as emphasizing a point (with a beat movement), highlighting some parts of discourse with a palm open hand gesture or structuring the speech by using the space around the speakers, and thereby simply presenting an idea to an addressee, regardless of the specifics of the idea being presented (Kendon, 2017). In this respect, the production of pragmatic gestures during approximate or incorrect interpreting of numbers could be a device whereby the interpreter facilitates their own speech production through ‘phatic’ behavior—that is, moving themselves to show they are expressing something, regardless of what actual words will come out. This could be a way in which they facilitate their own speech production (i.e., acting like they are about to provide a rendering of the idea from the source text audio).

Adapters, however, were found to be used both with correct, approximate, and incorrect interpretation of numbers. These movements appear to be all-purpose stress-relievers, helping the interpreters to manage the cognitive load on an ongoing basis as they do their work.

5. Conclusion

The research presented here shows advantages of taking a multimodal perspective in researching the process of number interpreting. This approach provides insights into the different processes involved that lead to the different outcomes (correct, approximate, or incorrect interpretation of the numbers) as well as insights into the kinds of conceptualization taking place, and the thinking for speaking involved during interpreting in these moments.

In addition, one could note that in the bulk of gesture research from at least the past 30 years, self-adapters have largely been excluded as an object of study. Many researchers have focused instead on the ostensibly communicative functions of representational, deictic and pragmatic gestures. However, research on high-stress contexts of communication, such as SI, shows frequent use of self-adapters. This suggests avenues for future research, ones which harken back to early work on body-oriented movements, such as that of Ekman and Friesen (1969) and Freedman (1972), cited earlier. While that research arose in part due to interest in what it could offer psychoanalysis, future research might focus on professional contexts of communication under stress, such as SI, in which the use of gesture, and of self-adapters in particular, may provide insights into processes of speech (and idea) production pertinent to those communicative contexts.

In general, the results show that hand gestures might help to deal with the cognitive load in SI and facilitate the process of word search, thus aiding the flow of speech and helping avoid interruptions and omissions. We finish the current study with a recommendation not to restrict oneself when performing interpreting but to use pragmatic and representational gestures when having difficulty in remembering a word and to use adapters to cope with stress and reduce it during this kind of work.