1. Introduction

1.1. Multimodality in pragmatics

Challenges in pragmatics, initially defined by its founder Charles Morris, involve questions of behavioural semiotics, which consists of two key elements: multimodality and sign behaviour. We can now assert that Morris’s conception of pragmatics is fundamentally a multimodal semiotic pragmatics, expanding the traditional pragmatic framework by integrating multiple semiotic modes into the analysis of communication. Multimodal pragmatics is an emerging research area that combines several semiotic modalities – such as spoken language, gestures, facial expressions, and visual elements – to analyze pragmatic competence and communicative interactions (Dicerto et al., 2018: 37–59; Beltrán-Planques and Querol-Julián, 2018; Freigang et al., 2017: 142–155). Pragmatic multimodal competence highlights learners’ abilities to use linguistic (speech) strategies effectively to express themselves adequately, comprehend each other accurately, and communicate successfully within specific contexts (Jiang, 2013). Kress (2009) argues for a social semiotic framework that examines how different semiotic modes – such as speech, gestures, images, and text – combine and interact in contemporary communication, offering insights relevant for analyzing multimodal aspects of public discourses. A multimodal perspective thus broadly positions multimodal pragmatics as an umbrella term that encompasses both verbal and non-verbal elements, crucial for effective communication and essential to consider when studying interlanguage pragmatics.

Therefore, this study’s relevance stems from the importance of non-verbal communication channels in processing and assimilating information. The objective of this study is to investigate how multimodal elements (especially gestures aligned with speech acts) function in the persuasive discourse of youth climate activists. This investigation is guided by two research questions:

1. What types of illocutionary acts dominate their speeches (the verbal component)?

2. How are these speech acts reinforced or modulated by gestures and other nonverbal cues?

According to Kibrik and Èl’bert (2008), only 39% of information is transmitted through verbal communication, whereas prosody conveys 28%, and non-verbal communication accounts for 33%. Furthermore, speech and gesture are inherently inseparable; gestures frequently complement verbal utterances, facilitate communicative goals, and enhance expressiveness (Kendon, 2004). While language remains the primary medium for conveying messages, communicators simultaneously – and often unconsciously – aim to prompt actions, motivate responses, or evoke emotions.

All natural languages employed in social interactions utilize multiple communicative channels. While these channels sometimes provide redundancy (e.g., saying "He turned left" while pointing left), they more often contribute new and distinct information. Enfield (2009) specifically explores how speech and gesture form integrated meaningful units, asserting that utterances are inherently multimodal.

From this perspective, public speaking is viewed as an interaction among various modalities (communication channels and sign systems). Contemporary researchers highlight that, in prototypical oral communication, alongside distinguishing individual words, we capture intonation, pay attention to voice qualities and non-verbal sounds, and interpret the interlocutor’s facial expressions, postures, and movements. Additionally, multimodal communication research emphasizes focusing on both speaker and listener, noting that the distribution of roles among participants significantly influences communicative behaviour.

1.2. Research on multimodality

Multimodal research is commonly divided into two trends: visual and gestural (Iriskhanova, 2022). The visual trend investigates written texts featuring visual elements, such as newspaper articles, comics, posters, graffiti, photographs, advertisements, films, and video clips. Conversely, the kinetic or gestural trend examines public speaking, focusing on body movements and gestures.

Voice quality, amplitude, pitch, and timbre can provide essential supplementary information to utterances. Similarly, the gestural-visual channel can be considered multilayered. For example, a person may look with narrowed or wide-open eyes, briefly or extensively, with or without blinking, each layer potentially conveying independent meanings. As Abercrombie (1968: 55) famously stated, "We speak with our vocal organs, but we speak with our whole body." Ray Birdwhistell (2021: 3), who introduced the term "kinesics" to describe bodily postures, estimated that verbal communication accounts for no more than one third of human interaction content. While this exact figure remains debatable, it is undeniable that using multiple communication channels enhances – and sometimes multiplies – the meaning conveyed through the verbal channel.

The systematic investigation of multimodal communication is comparatively recent and has advanced largely because of digital technologies. Although Greek and Roman orators classified gestures, sustained empirical research did not begin until the latter half of the twentieth century, led by anthropologists, phoneticians, and social psychologists. Sign languages provide a compelling illustration of multimodality: they require the coordinated engagement of multiple articulators according to linguistic rules while fulfilling expressive demands; for instance, facial movements complement the two-handed manual channel (Loos et al., 2022). While hand gestures have been documented in detail, other expressive channels remain understudied. The face, in particular, is a site of numerous overlapping layers – movement of the forehead, eyebrows, mouth, teeth, jaw, head tilt, and gaze – yet it has attracted limited systematic attention. Facial expression is also culturally inflected (Russell, 1995). Japanese interlocutors typically attend to the eyes, whereas Americans tend to focus on the mouth; in Papuan culture, nose wrinkling, contrary to Ekman’s prediction, signals astonishment rather than disgust (Hömke et al., 2018). Laboratories worldwide have begun assembling richly annotated multimodal corpora, yielding important new insights (e.g., Kendrick, 2015). Core concepts of multimodal pragmatics are now being integrated with corpus-based methods that systematically analyze extensive collections of authentic language data, thereby broadening the empirical base of pragmatic research and extending classical pragmatic theory.

Each human utterance is a highly orchestrated event, drawing upon a complex repertoire of communicative resources. Researchers estimate that speakers may utilize over 150 distinct signaling mechanisms – verbal, prosodic, gestural, postural, facial, and ocular. Although not all channels are activated simultaneously, the speaker continuously selects, synchronizes, and modulates these resources in real time. Rather than a solitary act of linguistic encoding (Sidiropoulou, 2020), communication becomes a multimodal performance – more akin to a symphonic composition than to a monophonic verbal string. As such, every spoken act is inherently multimodal, layered with concurrent signals that enrich, frame, or even subvert literal linguistic content.

Multimodal pragmatics, as a field, has rapidly expanded beyond foundational linguistics into a range of applied domains (Norris, 2012). The analytical analysis of research field classification on the topic with the keyword "multimodal pragmatics"– retrieved using Dimensions AI on November 12, 2025, from the Web of Science (WoS) Core Collection Database across all editions, including datasets, grants, patents, clinical trials, and policy documents as shown in Figure 1 – indicates the key research field as Language, Communication and Culture. The procedure below generated a search result of 83,288 articles.

In artificial intelligence and natural language processing, multimodal pragmatics provides key insights into how machines can interpret not only literal verbal content but also contextual cues, emotional tone, and interpersonal stance (Cambria, 2025). In translation studies, it allows for accurate rendering of multimodal texts, including films, advertisements, and social media posts, where meaning often hinges on co-textual visuals or gesture-based framing. In advertising, multimodal argumentation analysis helps deconstruct how emotional and logical appeals are conveyed not just through slogans, but also through imagery, colour, spatial design, and gaze orientation.

In applied linguistics, particularly in second language education, multimodality has become indispensable. Teachers increasingly rely on non-verbal semiotic resources – gestures, facial expressions, digital tools – to scaffold comprehension, manage classroom interaction, and support embodied cognition (Lim, 2020). This educational use of multimodal discourse draws on various theoretical frameworks: social semiotics (Kress, 2009), multimodal pedagogical stylistics (Lei, Zhang, 2022), and systemic functional linguistics. Research shows how these frameworks enable the analysis of pair work, interactive lectures, listening comprehension, and video-based learning (Morell, 2018; Norte Fernández-Pacheco, 2016; Campoy-Cubillo, Querol-Julián, 2021). Jewitt (2006, 2023) explores the implications of integrating digital and multimodal approaches into education, emphasizing the necessity of recognizing multiple semiotic resources to effectively analyze and support contemporary literacy and communication practices. These studies consistently emphasize that meaning is co-constructed through ensembles of semiotic modes and that students’ engagement deepens when verbal input is embedded in a multimodal context.

Furthermore, multimodal corpus approaches to speech act theory enable fine-grained analysis of how communicative intentions are instantiated through gesture-speech synchrony, prosodic framing, and gaze orientation (Huang, 2021; Fang, 2023). These empirical frameworks not only enrich traditional pragmatics but also inform broader discourse studies in political rhetoric, clinical communication, and activist media.

From a cognitive perspective, the discovery of multiple cross-modal areas in the cerebral cortex suggests that human experience is profoundly multimodal. We perceive the world holistically, not through isolated channels. This principle directly informs multimodal learning, which engages multiple sensory and cognitive channels to allow for differentiated instruction (Goodwin, 2017). In this context, visual literacy is not passive recognition but "the ability to analyze the power of an image and its meaning in a particular context" (Duchak, 2014). Effective implementation relies on principles of adaptivity, accessibility, interactivity, and diversity – the latter requiring the integration of resources across verbal, visual, gestural, spatial, and digital semiotic channels (Bouchey et al., 2021).

According to Haryanti et al. (2023), the procedural objective of multimodal pragmatics in linguistics is the development of communicative, analytical, and integrative competences. These scholars argue that multimodal systems train cognitive flexibility and stimulus response discrimination across heterogeneous information sources. Furthermore, they emphasize that creative and critical thinking are essential, arising from the necessity to interpret, reorganize, and re‑express complex input across different media types, since multimodal meaning‑making involves also the integration of embodied and contextual signals in communicative interaction (Bucher, 2025). Recent work on multimodal complexity highlights that communicative competence encompasses the coordination of multiple semiotic modes and layered representational forms, which can be studied across features and registers in situated discourse.

In pragmatic analysis, multimodal learning intersects with the study of speech behavior that provides diagnostic access to the speaker’s intentionality, role identification, social alignment, and identity framing. Speech acts, gestures, and embodied expressions within instructional settings form structured behavioral data relevant to functional, interactional, and affective analysis (Machin, Mayr, 2012).

Pragmatics examines the intentional and contextually motivated selection of lexical units, grounded in both explicit communicative goals and implicit, habitualized speech experience. Over time, repeated patterns of linguistic behavior become internalized, operating below the level of conscious awareness as communicative intuition. Since linguistic behavior is shaped by social stratification – across dimensions such as nationality, age, education, and profession – pragmatic analysis naturally extends to the differentiated speech patterns of social groups.

1.3. Pragmatics in environmental discourse

According to Cox (2013), environmental activists are a prime example of individuals whose rhetorical competence is evident in their public discourse. From a rhetorical-argumentation perspective (Tindale, 2004), their aim is audience-oriented and context-sensitive: reasons are crafted for adherence within specific situations (i.e. environmental discourse) rather than judged only by abstract validity. Like politicians, media figures, and public speakers, they employ multimodal strategies to enhance persuasive impact. Their communicative success derives from a strategic integration of verbal content, prosody, facial expressions, and gesture. The urgency of the ecological crisis necessitates such persuasive multimodal performance, aimed at mobilizing public concern and political response.

Doyle (2012) argues that effective communication of climate change requires overcoming deeply ingrained cultural perceptions that position climate change as distant or future-oriented. She critically examines how visual representations – such as imagery of melting glaciers or endangered animals – often reinforce perceptions of climate change as an external environmental issue rather than an immediate, human-centered concern. Adopting a human‑centered lens, climate narratives gain traction when they link environmental change to lived conditions and responsibilities within local communities, aligning messages with audience values and concerns about everyday life (O’Callaghan et al., 2025). In the same vein, Miller (2025) argues that applying persuasion theory to climate communication—especially through narrative framing and moral appeals tailored to specific audiences—can enhance engagement and motivate climate‑friendly behavior by connecting abstract scientific information with individuals’ lived experience. Emphasizing culturally relevant messaging and story‑driven communication helps audiences interpret and internalize climate narratives in ways that resonate with their values and social identities.

Environmental activism is characterized by intentional communicative action grounded in personal experience and ethical commitment. Activists engage in discourse not only to disseminate information but to provoke affective and behavioral change. Recent research on visual and narrative strategies in climate communication shows that framing environmental messages through relatable, lived experiences and positive, solution‑oriented narratives can deepen engagement and motivate audiences to internalize and act on environmental issues (San Cornelio, Martorell and Ardèvol, 2024). Environmental discourse, hence, refers to the set of communicative practices – linguistic and non-verbal – used to articulate, frame, and circulate ecological concerns in ways that resonate with human values and social identities.

Among young environmental activists, shared social identity and moral conviction provide the motivational core for participation and framing, linking individual beliefs about collective agency with engagement in public action (Brügger, Gubler, Steentjes and Capstick, 2020). Their public speeches reflect deliberate value choice and self-determination, linking personal moral stance to collective agency and anchoring persuasive appeals in lived commitments.

However, despite the significant academic advancement made in multimodal pragmatics by far, the existing pile of literature has received scanty attention of youth climate communication. An investigation is thus necessary to understand how youth environmental activists combine speech acts with gestures to persuade. Existing studies have not systematically integrated speech act theory with gesture analysis in this context. This study addresses that gap by examining whether and how embodied cues (f.ex. gestures) correlate with the illocutionary component of activists’ speech acts. The integrated approach enables a shift from treating gestures as mere accompaniments to speech toward the analysis of complex multimodal performatives. The study empirically identifies and delineates the specifics of an emerging genre – the rhetoric of youth climate activism. We demonstrate that its persuasive power stems not from institutional authority, but from a unique combination of verbal strategies (e.g., direct directives, personal narratives) and non-verbal tactics (e.g., gestures conveying vulnerability, openness, and moral steadfastness). Previous analyses of climate activism rhetoric have focused on linguistic and emotional appeals (e.g., Cox, 2013; Tindale, 2004) but have rarely examined the gestural or multimodal aspect of these appeals.

Likewise, while multimodal pragmatics research has advanced in fields like education and human-computer interaction, its application to social movement discourse remains limited. This study bridges that gap by applying multimodal analysis to activist oratory. We aim to contribute new insights into multimodal persuasion by youth activists – a communicative mode distinct from that of politicians or older advocates. This study is the first to systematically integrate the analytical frameworks of gesture studies (Kendon, McNeill) with classical speech act theory (Searle) within the context of multimodal communication.

The research makes a methodological contribution to multimodal studies by developing and implementing a detailed annotation scheme, provided as a reference in the supplementary materials. This ensures the transparency and reproducibility of the analysis, aligning with the best practices in empirical research.

2. Materials and methodology

2.1. Participants and data sources

The study analyzes the speech behavior of three internationally prominent youth climate activists whose public discourse has gained widespread visibility on platforms such as TED and YouTube. The corpus for this study comprises public speeches by three representatives of the global youth climate movement: Greta Thunberg (Sweden), Jamie Margolin (USA), and Xiye Bastida (Mexico/USA). The purposive selection of these cases is driven by the need to represent the key discursive strategies within this movement. This approach allows us to analyze their speeches not as individual utterances, but as exemplars of the collective rhetoric of this community with its own distinct norms. We proceed from the premise that their communication enacts a specific genre of mass-media environmental discourse – the public speeches of young environmental advocates. This genre is distinct both from the rhetoric of adult politicians, which relies on institutional authority, and from the speech of NGO experts, which appeals to technical solutions. Young environmental ecologists have a unique persuasive challenge: lacking formal authority, they rely on moral appeal and personal ethos. The internal variation within the sample – Thunberg as a confrontational global leader, Margolin as a media- and legally-oriented strategist, and Bastida as an advocate for environmental justice – enables us to identify the invariant, core multimodal patterns of this genre, as well as its context-driven modifications.

The selection criteria also included global recognition, sustained activism, and multimodal public engagement. Each participant represents a distinct sociocultural background, enabling comparative insight across demographic variables.

Greta Thunberg (b. 2003), a Swedish environmental activist, is the initiator of the School Strike for Climate movement and a central figure in the Fridays for Future network. Her addresses to the United Nations and national parliaments have received substantial international coverage. Jamie Margolin, a Colombian-American activist (b. 2001), is the co-founder of Zero Hour, a youth-led climate justice organization. She is known for her direct engagement with legislative bodies and her emphasis on intersectional climate advocacy. Xiye Bastida (b. 2002), of Indigenous Mexican Otomi-Toltec descent, is affiliated with the Peoples Climate Movement and the Fridays for Future initiative. Her activism integrates environmental concerns with Indigenous epistemologies and intergenerational equity.

2.2. Data description

Data were drawn from nine environmental youth activist speeches:

1. Greta Thunberg’s address to the UK Parliament (2019), her UN Climate Action Summit speech (2019), and her speech at the World Economic Forum (Jan 2020).
2. Jamie Margolin’s testimony to the US Congress (2019, her TEDx Youth presentation in Columbia (2019), and her closing remarks at Sun Valley Youth Forum in Idaho (2019).
3. Xiye Bastida’s speech at the “Climate Action: Race to Zero and Resilience” forum in New York (2022), her talk at the Bloomberg Green Festival (2024), and a main-stage Humanity Reimagined TED Conference presentation in Vancouver (2025).

Each speech ranged from 4 to 12 minutes, totaling ~80 minutes of material.

All speeches were transcribed verbatim from publicly available video recordings; the resulting scripts served as the basis for segmenting the discourse into individual speech acts and for identifying co-occurring multimodal cues.

2.3. Analytical framework and data processing

We segmented transcripts into individual speech acts based on predicate-finality boundaries, i.e. points where a main predicate and its complements conclude a proposition (often coinciding with a full sentence or independent clause end). For example, in the sentence "We need to act now because time is running out," we split at the conjunction to get two units: "We need to act now" and "because time is running out," each expressing a distinct proposition/intention. This segmentation criterion ensured that each unit reflected a self-contained instance of speaker intention. Following segmentation, each speech act was subjected to dual-layer annotation: first, at the locutionary level, capturing the formal linguistic structure and rhetorical tactic; and second, at the illocutionary level, identifying the communicative function as defined by the speaker’s intention. While not a standard segmentation in pragmatics, we found that predicate-final segmentation aligned well with the general cadence of speeches, their prosody, breath pauses and shifts in illocutionary expressions.

Illocutionary functions were categorized according to an adapted version of Searle’s (1969) taxonomy, which includes assertives (statements of fact or belief), directives (attempts to influence listener behaviour), commissives (commitments to future action), expressives (revelations of psychological states or attitudes), and declaratives (acts that effect change through utterance) (Barrero Salinas, 2023). This classification provides a stable framework for aligning verbal strategies with pragmatic functions across all speakers (Levinson, 2016).

We adopted Searle’s five illocutionary categories, but given the persuasive context, we treated some categories with nuance. Assertives were defined as statements that present states of affairs as true or factual (e.g. "The climate crisis is happening right now"), directives as utterances that attempt to get the audience to do something (e.g. "We have to come together and know that we can support each other to move forward"), commissives as commitments by the speaker to a future course of action (e.g. "<…> my generation will not give up without a fight"), expressives as verbalizations of psychological states or the speaker’s affective stance (e.g. "Our house is still on fire"), and declaratives as utterances that aim to bring about an institutional change in status by being said (e.g. "I am here before the whole country today announcing that we are instead <...> Generation Green New Deal."), which we expected to be rare given the speakers’ lack of formal institutional authority.

Given the persuasive context, each utterance was coded for the illocutionary type that best captured its dominant rhetorical function. When a speech act could plausibly fit two or more categories (for instance, an assertive that also indirectly invites action), the coders selected a single primary illocutionary act on the basis of the local persuasive strategy, resolving borderline cases through discussion until agreement was reached. This ensured that speech-act labels reflected the main communicative intention of the activists rather than a maximalist accumulation of possible readings.

To ensure the reliability of the speech act annotation, we calculated inter-annotator agreement using Cohen's kappa coefficient. Two trained coders independently annotated a randomly selected sample of 148 speech acts (12% of the total). The resulting Cohen's kappa coefficient was κ = 0.78, which indicates a substantial level of agreement based on standard benchmarks (Landis and Koch, 1977). Subsequently, all disagreements in the training sample were discussed and resolved, and the coders proceeded to annotate the entire corpus independently using the refined guidelines. A critical concern in multimodal research is the subjective nature of annotation. To address this, we established the reliability of our gesture coding scheme through formal statistical testing. Two annotators, after a joint training period, independently coded a stratified random sample representing 15% of the entire dataset. We employed Krippendorff's alpha, a conservative coefficient that accounts for chance agreement and is well-suited for categorical data, for its flexibility with different numbers of coders, sample sizes, and measurement levels (in our case, nominal categories for gesture types). The analysis yielded a coefficient of α = 0.78. Following standard guidelines where values between 0.80 and 0.67 are considered indicators of "substantial" or "good" agreement, our result of 0.78 falls within this acceptable range, confirming that the gesture taxonomy was applied consistently and that the reported frequencies are reliable.

To capture the multimodal dimension of activist discourse, the primary unit of analysis was defined as a multimodal composite: "linguistic expression + gesture." The boundary of each unit was determined by the end of a speech act, enabling one-to-one mapping between verbal and gestural sequences. This alignment permitted analysis of how gestures function in conjunction with verbal content to modulate, reinforce, or redirect communicative intent.

Persuasive tactics were examined through the classical rhetorical triad – ethos (credibility), pathos (emotional appeal), and logos (logical argumentation) – as conceptualized in Aristotle’s "Rhetoric". This triadic model (IxDF, 2016) was used to map the strategic deployment of multimodal elements within the discourse, allowing for integrated analysis of how speech acts and bodily expressions co-articulate persuasive meaning.

This study aims to experimentally combine rhetorical theory and pragmatics: Aristotle’s ethos/pathos/logos model provides a lens on persuasive tactics, while Searle’s speech act theory classifies communicative intentions, and gesture typologies (after Kendon, McNeill, etc.) reveal how physical cues complement those intentions. Departing from traditional syntactic or intonational criteria, this study segmented speech acts using a functional criterion of "predicative completeness", an approach justified by our multimodal framework. A segmentation unit was defined as a speech segment where the propositional core (the predicate and its core arguments) reached semantic and performative finality. This finality was frequently signaled multimodally as well as verbally, often aligning with the stroke phase of a co-occurring gesture (Kendon, 2004; McNeill, 1992). This technique enables the extraction of holistic multimodal utterances, where speech and gesture constitute an integrated pragmatic unit – a feature especially pertinent for analyzing the dynamic and affect-laden discourse of ecological activists.

2.4. Methodological approach

The study employs a combination of comparative analysis, content analysis, computational processing of statistical data, and multimodal analysis of semantic and functional correlations between verbal utterances and gestural behavior. Each method is integrated into a unified framework to investigate how linguistic and non-linguistic modes operate synchronously in the discourse of environmental activists.

Gestures are operationally defined as intentional hand movements that function as communicative signs within the interactional context. To categorize these gestures, the study adopts and refines a typological model grounded in the frameworks of A. Kendon, D. McNeill (Kendon, 2004; McNeill, 1992), and further developed by C. Müller and A. Cienki (Müller, 1998. 2023; Cienki, 2013). This model includes five primary functional categories: pragmatic gestures (including beats or pointing) serve to regulate discourse or emphasize points; thought-formulating gestures (often metaphoric motions) help the speaker conceptualize or package ideas; discourse-structuring gestures mark transitions or organize segments of the talk (e.g., counting on fingers, sectioning space); Representational gestures depict content, subdivided into object-representing (iconic gestures showing objects or shapes) and action-representing (pantomimic gestures illustrating actions).

Our gesture classification merges Kendon’s functional distinctions with McNeill’s categories of representational gestures, refined by Müller’s and Cienki’s work on pragmatic gesture functions; this unified scheme was tailored to fit the rhetorical context of speeches. We focused on hand/arm movements that were clearly timed with speech and appeared to convey emphasis or meaning. Small idle movements or unintentional fidgets were not counted as gestures.

In addition to hand gestures, we noted other nonverbal cues (facial expressions, body posture, eye gaze) qualitatively during the analysis to enrich interpretation, although the systematic coding focused on hand gestures. Notably, when an emotionally charged statement coincided with a facial display (e.g., frown, smile) or a posture shift, we recorded that context in our notes, but these were not quantitatively tallied like the hand gestures. This will be the focus of our next research to provide a more comprehensive multimodal analysis.

Firstly, we conducted this annotation manually, then with the use of specialized software ELAN, given the manageable size of the data. Coding and segmentation were conducted collaboratively by two researchers. First, they jointly discussed and refined the speech act categories and gesture types using a small subset of the data in order to develop a consistent operational approach. Segmentation into speech-act units was then carried out and reviewed jointly, with both researchers working together at the boundary-identification stage. In the subsequent annotation phase, the researchers alternated between roles – at times both coding the material together, and at other times one annotating while the other systematically reviewed and cross-checked the annotations – so that every segment and gesture label was verified by both analysts. Disagreements or ambiguous cases were resolved through discussion.

All nonverbal cues were annotated by combining close video observation with time-synchronized coding. We structured, as an examplar, one speech (described in Table 7 Closing Remarks by Jamie Margolin) as a comprehensive ELAN project to demonstrate the process in detail. In this fully annotated example, the coding is organized into multiple tiers: a base tier with the speech transcript segmented by illocutionary acts; a tier labeling each segment’s speech act type (according to Searle’s categories); a tier indicating the rhetorical appeal of that segment (ethos, pathos, or logos); and additional tiers capturing each multimodal signal – including the speaker’s facial expression, gaze direction, body posture, and each notable hand gesture aligned with the speech. Each gesture instance is further described and classified on a dedicated sub-tier (noting its form and its pragmatic function, such as a discourse-structuring motion or an action-referencing gesture). This ELAN-coded file (provided in the Supplementary materials ^[1]) illustrates how time codes and annotations were aligned (manually or automatically) with the video for precise multimodal analysis (the scheme is also provided in the Supplementary materials ^[2]).

For the remaining speeches in our corpus, we applied the same analytical criteria that were reflected in the examination of the full multi-tier ELAN files. We also employed a manual method prior to using the software – reviewing the videos and aligning notable gestures and other nonverbal cues with the corresponding transcript segments in our notes, which were analyzed according to our annotation scheme. The inclusion of a single, fully annotated ELAN example serves to enhance research transparency and allows for a clear understanding of how the multimodal coding was implemented in practice.

3. Results

3.1. Preliminary analysis

The multimodal analysis of the speakers’ performances is carried out to exemplify how the environmentalists instantiated the moves and performed the persuasive strategy drawing on verbal and non-verbal modes of communication. Specifically, we looked at the activists’ spoken mode, use of non-verbal means (i.e., images and realia), space (i.e., authoritative or interactive), and posture (dynamic or static). The findings reveal that the activists’ speech seem to raise listeners’ awareness and/or familiarize them with a new concept before setting up the activity. In doing so, activists can situate public in the context of the activity and begin to involve them. After completing the activity, the summary move serves to recapitulate, reinforce, and build new knowledge. Activists rely on different images and use a variety of means of expression during their performances in order to influence the audience.

Environmental discourse at the perlocutionary level can be presented by different strategies, in particular by raising awareness, confrontation, persuasion, mobilization, narration. Every strategy is aimed to correlate with certain tactics that are expressed at the locutionary and illocutionary levels. In this study, we focus on the persuasive strategy. Based on Aristotle’s "Rhetoric" we identify three parts reflecting major tactics as essential components of persuasive strategy (Rapp, 2023):

Ethos (credibility). This tactic refers to the character or ethical appeal of the speaker. This tactic establishes the credibility of what is being said and the authority of the speaker. Personal authority is observed when the speaker refers to his status, experience, or moral responsibility to build trust. For example, “I stand before you today representing my entire generation”, which reveals the speaker’s moral responsibility. Ethical appeals are established when the speaker demonstrates honesty or compliance with common values. For example, “I sued my state government in a lawsuit called Piper vs. The State of Washington along with 12 other youth plaintiffs, for contributing to the climate crisis <...>” (Jamie Margolin).

Pathos (emotion). This tactic appealing to the emotions of the audience, using evocative language that expresses the speaker’s sensual attitude. The speaker uses emotionally charged words to evoke feelings. The speaker shares personal anecdotes or stories that evoke an emotional response from the audience. For example, “I also remember when I was driving by El Rio Lerma. It’s the most polluted river in Mexico. And it’s right by my hometown” (Xiye Bastida). The speaker uses personal background to explain the motivation for activism.

Logos (logic). This tactic focuses on reasoning and evidentiary arguments to support a claim, often using facts, data, or logical conclusions. The speaker uses logical reasoning, such as causation or comparisons, to make his arguments. For example, “We must also bear in mind that these are just calculations. Estimations. That means that these ‘points of no return’ may occur a bit sooner or later than 2030" (Greta Thunberg). These inferences that illustrate the urgency and urgency of the problem.

3.2. Speech act analysis

At the illocutionary level we distinguish the following speech acts according to Searle’s model (Table 1): assertives (statements that convey information or describe reality), directives (requests, commands, or suggestions intended to induce action, commissives (commitments to future action), expressives (expressing an emotional state or attitude), declaratives (statements that create or change social reality).

Analysis of Greta Thunberg’s speeches reveals a balanced use of ethos, pathos, and logos, with a slight emphasis on pathos. This emotional appeal is primarily conveyed through assertives and expressives, enabling broad resonance while maintaining persuasive rationality. Correlation between locutionary and illocutionary levels indicates assertives dominate the logos and pathos sections, while expressives feature prominently in the ethos segments.

Jamie Margolin’s rhetoric is heavily saturated with pathos, also realized through assertives and expressives. At the locutionary-illocutionary levels, assertives predominate across all three rhetorical appeals, with expressives exclusively conveying her ethical appeal. This suggests Margolin primarily leverages emotional persuasion to communicate her message.

Xiye Bastida’s speeches are characterized by a strong ethos foundation, established through deeply personal engagement. Expressives, reflecting significant emotional involvement, are the dominant speech act within her pathos and ethos appeals. At the locutionary level, assertives occur more frequently than within her explicit logos appeals.

Collective analysis of all three activists indicates that pathos is the most prevalent persuasive tactic, while logos is the least utilized (Figure 3). This finding suggests that although evidence and argumentation hold importance in environmental discourse, emotional and ethical engagement often take precedence.

Furthermore, analysis at the illocutionary level reveals a clear hierarchy in the speech acts employed. Assertives predominate significantly, indicating that the foundational communicative strategy within this environmental discourse is the presentation of facts, claims, and information transfer. This emphasis on assertives aligns with the activists’ need to establish credibility and ground their urgent messages in perceived reality, particularly concerning scientific evidence and observable impacts of the climate crisis.

Expressives constitute the next most frequent category, serving to articulate the speakers’ personal emotional responses, deeply held values, and moral convictions. Their prevalence underscores how personal narrative and affective resonance are crucial for connecting with audiences and motivating engagement on an emotional level, complementing the factual basis provided by assertives (Figure 4).

Conversely, directives (attempts to get the audience to perform an action) and declaratives (statements that bring about a change in institutional reality) are notably scarce. This scarcity reflects the inherent positioning of youth activists within the broader socio-political environmental. Lacking formal institutional authority or direct legislative power, their primary role is not to command immediate action through directives or effect institutional change through declaratives. Instead, their persuasive power derives from raising awareness (assertives), building solidarity through shared feeling (expressives), and compelling moral witness – a role consistent with their status as activists mobilizing public opinion and demanding accountability from established power structures. The near absence of commissives (commitments by the speaker to future actions) further reinforces this dynamic, highlighting their focus on persuading others (audiences, governments, corporations) to act, rather than pledging actions solely within their own power to undertake.

3.3. Multimodal unit analysis

Applying semiotic principles, we recognize discourse as fundamentally multimodal, integrating both verbal and non-verbal elements. This perspective frames the linguistic personality as a multimodal unit – a subjective construct operating within culture’s meta-discursive space. The core of any discursive field is a synthetic linguistic personality, whose structure and evolution are governed by its cognitive-pragmatic programme (i.e., its knowledge systems and communicative strategies). Thus, the environmental activists’ speech and the synthetic linguistic personality realized in its space is a multi-subtextual formation (synthetic text) formed based on the fusion of verbal, articulatory and visual components. The following is a brief of each of the activists’ visual-functional representations, Greta Thunberg (Tables 2-4), Jamie Margolin (Tables 5-7), Xiye Batida (Tables 8-10).

From the 1230 analyzed speech acts, 172 multimodal units were found containing non-verbal couplings, categorized into multimodal channels.

These couplings were categorized by function as follows: structuring discourse (n=51) emerged as the most frequent, followed by pragmatic functions (n=44), formulating thoughts (n=27), object-representing gestures (n=29), and action-representing gestures (n=21). These findings confirm that gesture functions are contextually activated to reinforce rhetorical appeal. Specifically, the dominant structuring gestures emphasize message coherence and framing, while pragmatic gestures play a significant role in interaction regulation and highlighting speaker agency. Although representational gestures (object and action combined) were the least frequent category, they crucially anchor abstract discourse to concrete referents, significantly aiding audience comprehension.