INTRODUCTION

Parallel with the internationalisation of higher education worldwide, nowadays, growing numbers of students use a second language (L2) for study and research, both in their home country and abroad. A serious consideration for these students is achieving the language proficiency required for academic success. In university settings, for instance, students attend lectures, participate in seminars, and listen to presentations. In other words, they need to be able to communicate effectively in various listening situations in order to successfully pursue their studies. In terms of L2 lecture comprehension, the literature reveals that students who attend English-medium courses, both in English-speaking and non-English-speaking countries, face persistent difficulties in processing the L2 lecture content [11, 29]. With this in mind, research into the variables that affect L2 lecture comprehension becomes increasingly important because it would help theoreticians and practitioners to better understand this process; and, thus, provide insights into how listening to L2 lectures can be facilitated. There have been a number of studies which aimed at identifying the strategies reported by university students in Bulgaria [29, 22]. No previous study, however, investigated the effect of strategy-based instruction (SBI) in an academic context with Bulgarian learners. The impetus for the present study, therefore, came as a response to the perceived need for further research into the area of L2 lecture comprehension. More specifically, this paper aims to address the knowledge gap that exists regarding the effect of listening SBI in the Bulgarian university settings.

LECTURE COMPREHENSION SKILLS AND STRATEGIES

Lecture comprehension

In contrast to hearing, which is an automatic process, in the field of cognitive psychology, listening is generally understood as an intentional, controlled process which is modulated by attentional capacity. Listening further requires the use of self-regulatory resources and involves information processing involving different modalities across the acoustic and visual domain [16]. From a cognitive stand, listening has also been described “as the process of selecting, organizing, and integrating information,” drawing both on internal and external sources for the processing of information [16, p. 98]. A listener does not only take in verbal information: variables as prior knowledge, context information, situational factors, body language, and paralinguistic messages also change and complement the verbal input [16]. As Field notes, the listener acts upon two information sources: “one perceptual, based upon auditory input and one conceptual, drawing upon the listener’s own world knowledge, topic knowledge, and recall of what has been said so far” [8, p. 103]. Process models of listening [10] focus on the direction of information processing and the information sources employed for the interpretation of the message. Thus, bottom-up (“data-driven”) processing draws on the acoustic signal or the perceptual data as an information source, while top-down (“knowledge-based”) processing incorporates activated knowledge structures or contextual sources [7]. Most authors agree that the two processing paths are complementary and occur in interaction [7, 10]. In fact, listening and reading researchers have been interested in the preferred processing route, rather than in establishing which single path has been selected for the construction of meaning from text [7]. Furthermore, the preferred processing direction seems to greatly depend on the individual characteristics of the listener and the specific task she/he has to perform.

The discussion above applies to listening in any and all situations. However, lecture listening has its own distinctive characteristics. Thus, lecture comprehension has been associated with the listener’s background knowledge, including subject matter knowledge; the ability to evaluate information according to its relevance and importance; the ability to concentrate, follow and comprehend extended stretches of discourse while at the same time taking notes; the ability to integrate information derived from the spoken input with other media. Lecture listening also exhibits less turn taking and less emphasis on the interpersonal or illocutionary meaning as compared to conversational listening [9]. Another take on lecture listening comes from Aryadoust, V., Goh, C., and Lee, O. K. [3] who propose a multicomponential construct comprising interrelated elements. The main components in their model are “cognitive processing skills (CPSs): “ability to understand surface (explicitly stated) information and making inferences; and linguistic components and prosody (LCP): vocabulary and syntactic resources” [2, p. 4]. Other variables associated with CPS and LCP are “note-taking (NT): ability to take notes of main ideas and details of the aural message; knowledge of lecture structure (LS): students’ awareness and/or understanding of the framework upon which the structure of the lecture is founded; relating input to other materials (RIOM): ability to form a mental connection between the information transferred through various modes; memory and concentration (MC): ability to keep important parts of the message in mind” [2, p. 4]. This section has briefly summarised the core dimensions underlying lecture comprehension. The next section will highlight perspectives on the strategies for L2 lecture comprehension and previous research into listening SBI.

Strategies for lecture comprehension

The term “strategy”, in the present study, is considered from the perspective of Macaro’s [17] theoretical framework, drawn upon cognitive psychology and information processing, and his and Cohen’s [5] characterisation of learner strategies. Namely, important is the element of choice, and the definition of strategies as conscious mental, goal-directed actions used in relation to language learning or use tasks and which (if appropriately selected) can be effective in maximising existing linguistic resources and language performance. It is worth pointing out that strategies are neutral, i.e. they cannot be intrinsically effective or ineffective. The failure of a learner to accomplish a specific task could not be due to the use of ineffective strategies. A strategy can only be effective if the learner knows when (depending on the task) and how (in orchestration with other strategies) to deploy it. What Macaro views as “skills” in contrast to “a strategy” is “the ability to carry out a language task with relative expertise to a relatively successful degree” [17, p. 321]. He stresses that skills are measurable products (manifestations) of L2 processes and that they can be measured either in isolation (listening, speaking, reading, and writing) or in combination (summaries, reports). Skills can also be measured in terms of successful task completion and with respect to skill acquisition rate. Macaro concludes that the automatisation of strategies through recurrent use of strategy clusters during L2 processes brings about skillful behaviour [17]. Hence, following his line of argument, one can deduce that the automatisation of cognitive and metacognitive listening strategy clusters would lead to students’ increased proficiency in listening to L2 lectures.

A great variety of strategy taxonomies have been constructed by a number of researchers [20, 4, 12, among others].  Among them, the classification proposed by O’Malley and Chamot [20, pp. 197-199] introduces the distinction between metacognitive and cognitive strategies. Further, they divide strategies by function in three groups: metacognitive (advance organisation, advance preparation, organisational planning, selective attention, self-monitoring, self-management, self-evaluation), cognitive (resourcing, grouping, note taking, summarising, deduction, imagery, auditory representation, elaboration, transfer, inferencing), social/affective (questioning for clarification, cooperation, self-talk). However, the present study is built upon the cognitive-metacognitive strategy distinction since it emphasises on two essential operations in L2 language processing. More specifically, on such aspects of cognitive activity as perception, decoding, processing, storage, and retrieval. Metacognitive operations, on the other hand, involve effectiveness planning, monitoring, and evaluation. As Macaro asserts, metacognitive strategies include the affective strategies because the latter draw upon learner’s self-knowledge and monitoring of learning; while, “social strategies are clusters of cognitive and metacognitive strategies that lead to Strategic Plans” [17, p. 328]. Also, the present study refers to strategies as associated with a specific receptive skill area [6] – namely, listening strategies for effective comprehension of L2 lectures.

Recent work in strategy research has investigated the effect of SBI on students’ strategy use and on listening development [19, 24, 14, 28]. Overall, the studies testing the effect of SBI for listening in academic settings produced inconclusive and conflicting findings. There are some intervention studies which reported results in favour of strategy training in academic settings [e.g. 18]. However, a systematic review of available research on SBI reported low weight of evidence that strategy interventions can improve listening [15]. A meta-analysis of 6 studies, conducted in the tertiary foreign/second English, Japanese, and Russian language context, concluded that there is limited evidence for the effect of SBI on listening comprehension. This is so mainly (a) because of methodological flaws of the studies; (b) because of failure of some studies to show significant improvement. These findings resonate with the opinion of other authors who have questioned the viability of listening SBI (esp. with low-proficiency learners) and the effective transfer and use of these strategies in “real-world” situations [8, 21, among others].

In this context, the overall aim of the present study is to empirically test the effect of a SBI programme, based on modelling of cognitive strategies and metacognitive development (focus on metacognitive strategies), on L2 lecture comprehension in the academic discourse.

METHOD

Participants

The participants (N = 59) were first-year students at the average age 19.4 (range 18-27; SD = 1.32). They were all intermediate and upper-intermediate level (CEFR B1-B2) in English proficiency, as determined by the university entrance exams. All of them, studying towards a Bachelor’s degree at the University of Veliko Turnovo and the University of Shumen in Bulgaria. A two-group, experimental (EG) and comparison (CG), design was employed. Thirty-four students comprised the EG and twenty-five served as a CG. With regard to gender, the female students outnumbered (66.102%) the male participants (33.898%). However, gender did not seem to play a role as a confounding factor in the quasi-experiment because the t test produced no significant differences between the mean scores of the female (M = 13.897, SD = 6.00) and the male group (M = 14; SD = 5.380) on the pretest (p = 0.949). The first language of the overall sample was Bulgarian.

Instruments

The lecture listening test

The quasi-experimental design was based on pre-test and post-test data gathered from a set of two lecture listening tests (LL-test) devised for the purposes of the study. They aimed at determining the level of students’ comprehension of the lecture extracts. The two tests comprised of twenty-six multiple-choice items (statements) and asked the respondents to select among three possible options per item. The test items measured the students’ ability to comprehend global and local, as well as implicit and explicit information. The items came in the same order as the information presented in the text. Each correct answer receives 1 mark. 

The test reliability was examined using the Classical Test Theory (CTT) and Item Response Theory (IRT). The models within the IRT paradigm vary according to the test type [26]. For multiple-choice binary items, IRT suggests a 3-PL model with three parameters. This model was utilised in the study and the IRT theta values were used instead of the raw scores in the test evaluation. The Cronbach’s alpha or the internal consistency reliability for the overall LL-test (α = 0.854) was high enough. The item-total correlations (from 0.237 to 0.579) were positive; therefore, all items were of significant positive discriminant power. Finally, the item means (from 0.390 to 0.754) indicated that there were no items of extremely high or low difficulty.

The input texts

The texts were audio-recorded authentic lectures, given by L1 speakers of English. The lecture topics were on the educational system and the health services in the United Kingdom and they were related to the participants’ field of study in English and American Studies. The listening segments were between 17-20 minutes long. Their length reflected a fundamental characteristic of a real-world lecture listening task where the lecture monologues are not “interrupted by interactive exchanges (questions/requests for clarification) more frequently than once every 15 minutes” [23, p. 364], even for more participatory lecturing styles.

Procedures

Data collection

The research was conducted during the participants’ regularly scheduled classes on listening skills development in EFL as a component of the Practical English course. The quasi-experiment took place over 15 weeks (2 classes per week) during the first term when the students start attending English-medium courses. Only the EG received SBI in listening to academic lectures in English, whereas the CG did not systematically focus on strategy development.

Following the introductory sessions, the EG and the CG did the pretest. During the last week of their courses, the posttest was administered, also across the two groups. Before taking the tests the students were provided with instructions that aimed at setting the context orally and in writing (in the test materials). They were briefed on the nature and purpose of the listening task, the topic of the talk, and the speaker. The learners were then asked to preview the questions for 5 minutes before listening. The students were also allowed to take notes on a blank sheet of paper while listening to the lecture. It was emphasised that the notes could be in any language and they would not be marked. Following the lecture listening task, the learners were given 20 minutes to complete the multiple-choice task. They were allowed to use their notes.

The intervention

The intervention focused on the development of listening strategies for L2 lecture comprehension in English. The metacognitive strategies emphasised were selective attention, directed attention, planning, monitoring, evaluation. Cognitive strategies in focus were inference, elaboration, grouping, and summarisation. The training programme was adapted following the CALLA instructional sequence [20], with an emphasis on the explicit training or metacognition about strategies since it allows learners to monitor and self-regulate their listening performance. Thus, the SBI was built around a metacognitive instructional framework incorporating listeners’ metacognitive knowledge (person, task, strategy) and cognitive control (metacognitive strategies for planning, monitoring, evaluating) [13, 27]. The course also aimed to heighten students’ metacognitive awareness of the lecture as a genre in the academic discourse and about lecture listening as a typical task during their university study. Structured class and weekly assignments (e.g. listening log) for self-assessment and peer-assessment were also included in the instructional sequence. The materials were selected to expose the students to naturally occurring speech events and speakers – formal and interactive lectures delivered by L1 and L2 English speakers.

Data analysis

The statistical analysis of the test scores was conducted using the software Statistica 8.0. To contrast the achievements between the pre-test and post-test performance of each group, Student’s t test for dependent samples and the nonparametric Wilcoxon test were utilised. In addition, to contrast the achievements between the EG and the CG, Student’s t test for independent samples and the nonparametric Mann–Whitney U test were performed [25]. The 3-PL theta scores were used in the analysis of the test results. Dispersion analysis for repeated measures (ANOVA) was also conducted to test the main hypothesis which proposes that the intervention programme contributes to higher achievement of the EG in the development of lecture listening in L2. Three null hypotheses were verified in this instance. The most important one was the hypothesis for nonsystematic or random interaction between group type (comparison vs. experimental) and measurement occasion (pre-test vs. post-test). Rejecting this hypothesis would serve as evidence for the existence of intervention effect and as a confirmation of the main hypothesis.

RESULTS AND DISCUSSION

Results

Pre-test and post-test comparisons

Firstly, the t test for dependent samples indicated that the difference between the CG pre-test and post-test scores approached but did not reach statistical significance [t(24) = 1.496; p = 0.148]. The Wilcoxon test produced similar results [Z = 1.251; p = 0.211]. Therefore, the observed difference effect cannot be considered as statistically significant.

On the other hand, a comparison between the pre-test and the post-test results of the EG showed a statistically significant difference in a positive direction after the intervention. This was confirmed by the t test for dependent samples which revealed a strong statistical significance [t(33) = 6.015; p < 0.001] of the difference between the two sets of scores obtained prior to and after the SBI. The same results were obtained using the Wilcoxon test [Z = 4.300; p < 0.001].

Intergroup comparisons

The comparison of the pre-test scores of the EG and CG demonstrated that the groups were equivalent in their L2 lecture listening proficiency before the intervention. More specifically, the t test indicated no statistical significance [t(57) = 0.634; p = 0.529] of the difference between the pre-test mean scores of the EG and the CG. In addition, the F test for variances confirmed that there was no statistical significance [F(33, 24) = 1.268; p = 0.552] of the observed differences between the EG and the CG results (Table 1). Furthermore, the Mann-Whitney U test returned a similar result [Z = 0.736; p = 0.462], with no significant difference between the groups (the average rank for the EG was 31.412 and for the CG was 28.080).

Table 1

Contrast between the Comparison and the Experimental Group for the LL-Test  at Pretest Measurement (t test and Fisher’s F test)

The results of the EG and the CG at pre-test measurement are displayed graphically in Figure 1.    

Figure1 Box and whiskers plot for LL-test at pre-test measurement.

More importantly, at posttest measurement, the mean score of the EG (M = 0.852) was significantly higher than the mean score of the CG (M = -0.610). As Table 2 shows, there is a statistically significant difference [t(57) = 6.916; p < 0.001] between the mean scores of the EG and the CG, while the F test for variance shows no statistical significance [F(33,24) = 1.080; p = 0.824]. The result was confirmed by the Mann-Whitney U test [Z = 5.062; p < 0.001] which revealed that the EG performance was significantly higher than the CG (the average rank for the EG was 39.706 and for the CG was 16.800).

Table 2

Contrast between the Comparison and the Experimental Group for the LL-Test at Posttest Measurement (t test and Fisher’s F test)

The observed differences between the scores of the EG and CG are highlighted in Figure 2.

Figure 2. Box and whiskers plot for the LL-test at post-test measurement.

Finally, the repeated measures ANOVA provided data which strongly supported the main hypothesis that instruction in cognitive and metacognitive strategies leads to improvement in the students’ lecture comprehension skills. In this respect, Table 3 indicates that all three observed effects were statistically significant: the main effect of the “group” [F(1,57) = 16.436; p < 0.001], the main effect of the “measurement occasion” [F(1,57) = 8.669; p = 0.005] and the effect of the interaction “group” x “measurement” [F(1,57) = 26.208; p < 0.001].

 Table 3

   Results of the Repeated Measures ANOVA

Post-hoc analysis was conducted to examine the contrasts in more detail and to highlight exactly where the significant differences were. It is important to note again that there was no statistically significant difference between the scores of the CG and the EG on the pretest [p = 0.488]. In contrast, the difference was highly significant at the posttest [p < 0.001] in favour of the EG. Also, the difference in the performance of the CG between the pretest and the posttest [p = 0.157] did not reach significance.

         The result is illustrated in Figure 3.

Figure 3. Means plot for repeated measures ANOVA.

         To sum up, the results in this section support the main proposal of the study that a SBI programme contributes to a significant improvement of students’ skills for listening to lectures in L2. This finding corroborates with previous research in other L2/FL teaching contexts [19, 14, 18, 28] and differs from the results of the studies which did not obtain conclusive evidence that SBI positively influences L2 lecture listening proficiency [15].

There are several factors in the nature of the programme that could have accounted for the observed effect of SBI. One possible explanation is the informed training in the strategies required for effective processing of the lecture material which differs from the indirect approach, based “on guided practice on the listening process as a whole, through extensive exposure to and practice with the naturalistic oral texts” [28, pp. 487-488]. In this study, the students were made aware of the purposes of the training and the specific strategies they were encouraged to try out and apply, both in the training sessions and in “real-life” lecture listening situations. Explicit SBI was also deemed highly beneficial in the specific context of the study where raising students’ awareness of the characteristic features and conventions of the lecture as an academic genre could be an important knowledge source they could purposefully draw upon while processing a range of academic talks.

It is also important to note that the intervention was based on an integrated model of skill learning and text access through interaction of top-down and bottom-up processes. As stated earlier in this paper, the distinction between these two types of processes is associated with the competition between two types of sources that govern cognitive processing – “the previous context or the present input” [1, p. 4]. Thus, the SBI was devised on the premise that L2 lecture comprehension involves a continuous interplay between bottom-up and top-down processes. Therefore, the SBI incorporated activities that help students manage effectively the two types of processes in text understanding. Hence, alongside inference and elaboration (top-down processes), the students in the EG practiced such “bottom-up” strategies as, for example, focusing on prosodic features; noting redundancy; noticing reduction, ellipsis, and assimilation.

CONCLUSION

The purpose of this paper was to provide empirically-based evidence about the effect of SBI on L2 lecture comprehension. The research was situated within a cognitive framework with the central assumption that cognition and metacognition (both metacognitive knowledge and regulation) have a primary role in the performance of complex tasks – i.e. listening to lectures delivered in a second/foreign language. The results of the study support the main hypothesis that SBI significantly contributes to students’ higher achievement in the comprehension of academic lectures in English. Since the instructional sequence focused not only on cognitive strategies but also on students’ metacognitive development, the results further suggest that monitoring of attention, listener control and consciousness are of critical importance during the processing of oral input during listening to L2 lectures and extended talks. The present research also provides a framework for the exploration future instructional improvements in regard to L2 for academic purposes. In particular, with a view of using SBI to facilitate the L2 lecture comprehension skills of the students and increase their ability to handle the real-world demands of the academic programmes at their home institutions and internationally.