Introduction

The problems of text complexity and text comprehension have been in the focus of numerous researchers for a number of decades. Modern transdisciplinary research into text comprehension views rigorous text leveling based on its complexity as the core of successful reading and learning (Solnyshkina, Harkova, Kazachkova, 2020). Benefits and advantages of leveled reading as a strategy of assigning children to books that match their reading skills have been substantiated by hundreds of studies.  Popular text leveling systems, including Lexile, Guided Reading Level, Developmental Reading Assessment, Accelerated Reader and Scholastic Reading Levels (de-la-Peña, Luque-Rojas, 2021) rely on two main ideas: (1) a text presents challenges related to its informative/ cognitive and linguistic features; (2) a reader employs his reading skills at best when the reading stimuli lie within his zone of proximal development (Vygotsky, 1978). Therefore, reader/text matching algorithms imply assessment of text complexity and readers’ fluency, accuracy and comprehension abilities. While the existing approaches to testees’ reading abilities estimates are primarily based on cloze or open tests and assess how well a testee comprehends levels of meaning, idea, composition and language conventionality (Fox, 2009), concepts on text complexity assessment are plentiful and vary depending on the range of the text features considered. Since manual procedures of text complexity features measurement present numerous difficulties, researchers encourage development of machine analysis and NLP tools, which are expected to provide accurate text features estimates and compete with analysis conducted by humans (Khurana, Koli, Khatter, 2023).

Validated text complexity predictors and their significant correlations build a theoretical foundation for designing sophisticated text profilers which are capable of defining vocabulary levels of reading stimuli, producing customizable word frequency lists[1], identifying text complexity and aligning it with a category of readers[2]. The idea behind a text profiler is the dialogic nature of a text since a text is always addressed to a specific reader and as such can and should be aligned to a category of readers, i.e. ‘profiled’. Once a text is automatically ‘profiled’, software users receive access to textual analyses and, which is more important, its alignment to a category of readers. As for categories of readers, they are typically identified and presented either based on the number of years of formal schooling/ grades (readability formulas) or vocabulary acquired (Lexile.com).

The growing number of studies on text complexity published worldwide, has not exhausted the topic even for the English language (Solnyshkina, Solovyev, Gafiyatova, Martynova, 2022). There are still numerous research niches emerging, one of which we outline as the impact of lexical density on text complexity. In this article we explore to what extent lexical density affects complexity of English academic texts thus testing the well-known belief that lexical density predicts text complexity (Daller, Van Hout and Treffers-Daller, 2003). For this purpose, we outline the following research questions:

- RQ 1: What is the range of lexical density metrics in school Science and Social Studies textbooks across Grades 7-12?

- RQ 2: How do shares of different parts of speech vary in school Science and Social Studies textbooks across Grades 7-12?

- RQ 3: How does lexical density correlate with readability in school Science and Social Studies textbooks across Grades 7-12?

The research is conducted to confirm the hypothesis that lexical density dynamics in school Science and Social Studies textbooks slightly differ, and nominalization index is higher in Social Studies textbooks.

Literature review

Text complexity

As a concept and a notion ‘text complexity’ can be defined as a complex of factors affecting and contributing to text comprehension, in other words, they are a set of features which make understanding of a text challenging for a group of people or a particular person. Quantitative dimensions of text complexity which are typically measured by computer software include multiple features clustered into codependent complexity predictors. (cf. McNamara, Graesser, McCarthy and Cai, 2014). Over decades of intensive studies researchers have proposed numerous text complexity predictors including word and sentence length, word frequency, abstractness, syntactic complexity, (Solovyev, Solnyshkina, McNamara, 2022) lexical diversity or TTR and a number of TTR modifications (Templin, 1957) including Guiraud’s Index (Giroud, 1954), Corrected TTR (Carroll, 1964), Index of Herdan (Herdan, 1960, 1964), Maas (Maas, 1972, Tweedie and Baayen, 1998, Treffers-Daller, 2013); U Notational variant of Maas (Dugast, 1978; 1979), D score (Malvern, Richards, 1997) and MLTD original (McCarthy, Jarvis, 2010).

Flesh-Kincaid readability formula

The first text indices to be selected and derived into a formula able to predict text challenges for readers of different educational backgrounds were word and sentence length (Flesch, 1948: 233). The formula based on these features, the Flesch Reading Ease, became the first readability formula designed to select reading material for people of a certain educational status. Later it was redesigned to convert readability indices into school grade levels.  At present, there are more than 50 readability formulae developed to predict English text complexity (Crossley et al., 2008). The most common and robust formula to measure text readability for mainstream readers is Flesh-Kincaid grade level score. The input parameters in the formula are word length and sentence length:

Flesch-Kincaid Grade Level (FKGL)= 0.39 × (Total Words / Total Sentences) + 11.8 × (Total Syllables / Total Words) − 15.59 (Kincaid, Fishburne, Rogers, and Chissom, 1975).

Scholars of text complexity argue that there are a variety of factors contributing to its dynamics across grades/readability levels and types of discourse (Halliday, 2008). As for specifics of academic discourse, according to Hyland (2006b: 13-4), its typical features can be summed up by high lexical density among two more features, which are, high nominal style, i.e. nominatization, and impersonal constructions.

Lexical density

Lexical density was initially studied to compare spoken and written language (Ure, 1971), interviews and conversations (Zora and John-Lewis, 1989) and oral exam answers (O’Loughlin, 1995) to show the difference in mode or between spontaneous and planned speech. Later, lexical diversity was used to define complexity levels in IELTS writing test papers (To et al., 2013), newspapers, conversation, academic register (Biber, 2021), English textbooks (To, Fan, Thomas, 2013), high school English textbooks (Putra, Lukmana, 2017), textbooks for junior high schools (Mulyanti and Soeharto, 2019), eighth-grade California history textbooks (Schleppegrell et al., 2004).

The notion and the term of ‘lexical density’ was introduced by Jean Ure in 1971 (Ure, 1971) as the ratio of the number of content words per number of running words (tokens). Content carrying words include nouns, verbs, adverbs and adjectives. Prepositions, conjunctions, auxiliary verbs and pronouns are viewed as non-content words.

M. Halliday (1985) advocates lexical density as a text complexity predictor based on the fact that it relates to the text information structure and as such contributes to its complexity. For example, a conversation has lower lexical density compared to the written texts. M. Halliday argues that written language is “more dense” or “semantically loaded”, i.e. lexical density shows “how closely packed the information is” (Halliday, 1985: 62, 66) or “information package” (Johansson, 2008). In his seminal work “The language of science” M. Halliday concludes that “higher lexical density results in higher textual complexity” (2004: 83). D. Biber et al. (2021) claim that linguistic features perform textual tasks of the two major types: marking information structure and cohesion specifying that “text information structure refers to the way in which referential information is packaged or presented within clauses, as well as the way in which clauses are packaged or presented within texts” (Biber et al., 2021: 42). S. Eggins (2004: 94-95) argues that conversation is more dynamic with higher distribution of verbs, linked sequences of clauses while written language tends to have higher distribution of abstract ideas linked by verbs of being in condensed sentences. M. Halliday (1985; 1993), D. Biber and B. Gray (2016) distinguish morphological and syntactic differences affecting complexity of spoken and formal discourse arguing that complexity in conversation is clausal, while academic texts complexity is phrasal and is primarily presented by nominal phrases.

As lexical density refers to statistical indices, researchers suggest different ways of measuring it. J. Ure (1971) calculates lexical density as the ratio of the number of content words per number of running words:

M. Halliday (1989: 67) suggested measuring lexical density by calculating lexical items per total number of clauses. S. Eggins (2004: 97) estimates lexical density as ratio of content-carrying words per total number of words in the text. D.  Biber (2021) also calculates lexical density, i.e. the sum of content words, per text and further normalizes it per thousand words. The notion of content or lexical words shared by many linguists comprises nouns, adjectives, verbs, and adverbs. M. Halliday (1989: 63), O’Loughlin (1995) also proposed to take into account items consisting of more than one word such as phrasal verbs (to catch up on), idioms (to kick the bucket) or contractions (they’re, isn’t).

To the best of our knowledge a comprehensive study of Flesh Kincaid readability and lexical density of school Science and Social Studies textbooks has not been performed. Fang et al. (2006) examined indices of lexical density in the 3rd, 5th, and 10th grades textbooks of different subjects: it was registered as 51% in the fable studied in the 3rd grade, 53% – in the 5th grade Science texts, and 59% – in the 10th grade History texts. Two studies on lexical density progress were performed on Indonesian English textbooks used in junior high school (7th, 8th, and 9th grades) (Mulyanti and Soeharto, 2020) and senior high school (10th, 11th, and 12th grades) (Putra and Lukmana, 2017). The results indicate increase of lexical density across the grades. To, Fan, Thomas (2013) conducted research on four short extracts from reading passages in four English textbooks for elementary, pre-intermediate, intermediate and upper-intermediate levels to challenge the correlation between lexical density, readability (Flesch’s Reading Ease Scale), and text levels. The findings confirmed that the lexical density level may increase steadily with the text levels and their readability, however there was no strong relation identified between them in the reading comprehension texts. To and Mahboob (2019) explored lexical density in 24 texts reading passages from four English textbooks for elementary, pre-intermediate, intermediate and upper-intermediate levels and found increase of lexical density level from elementary to intermediate level and slight decrease in upper-intermediate level. As for the Russian language, the morphological patterns of academic texts reported in (Gatiyatullina et al., 2020; Solnyshkina et al., 2017) demonstrate a significant nominal over verbal dominance with nouns making 40-46%, verbs – 12–17%, adjectives – 14%, adverbs – 4–5% of tokens in the text (Gatiyatullina et al., Solnyshkina et al., 2017: 398). The research also confirmed a stable growth of nouns and decrease of verbs in Biology and Social science textbooks across grades 5-11.

Material and Methods

Material

The research corpus with the total size of 2.715.682 tokens comprises two subcorpora: Science (Biology) and Social studies (Civics, Government) (see Corpus Material). Each consists of textbooks for grades 7 – 12 of American secondary and high schools and published between 2008 and 2020 (cf. Table 1). As the textbooks under study are predominantly written for schoolchildren of more than one grade and as such are studied for two or three school years we divided all the texts into three levels based on the age of the target audience of readers: Level I – aged 12-14, Level II – aged 15-17, Level III- aged 18-19.

Table 1. Corpus Size

Таблица 1. Размер корпуса

For the purposes of the study, we combined texts of the same level and discourse into 6 groups of readability levels: Science I – III and Social Science I – III. Calculations of nouns, verbs, adjectives and adverbs were made both per sentence or per 1000 words (cf. Table 2).

Method

The algorithm of the analysis included 4 stages.

On Stage 1, with the help of TextInspector (https://textinspector.com/), we measured values of the following features in each group of texts: Flesh-Kincaid, verbal elements per sentence, noun elements per sentence, nouns, adjectives, verbs, adverbs, verbs in present tense, verbs in past tense and later normalized each part of speech, i.e. nouns, verbs, adjectives and adverbs, as well as the sum, i.e. lexical density, to 1000 tokens. Thus, the finalized list of the metrics compared and contrasted included the following: Flesh-Kincaid, Verbal elements per sentence, Noun elements per sentence, Nouns per 1000 words, Adjectives per 1000 words, Verbs per 1000 words, Adverbs per 1000 words, Lexical density per 1000 words, Verbs in present tense per 1000 words, Verbs in past tense per 1000 words (see Table 2). Following the universally accepted classification installed in TextInspector (https://textinspector.com/) which we use as a tool to measure text features, we also distinguish between and measured separately grammatical verbs or auxiliaries, and lexical verbs. Phrasal verbs, e.g., account for, were treated as one lexical item, i.e. account, and one grammatical item, i.e. ‘for’.

On Stage 2, we pursued an intra-discourse analysis: compared and contrasted the values across grades separately in Science and in Social Studies subcorpora.

On Stage 3, we contrasted the metrics across discourses.

On Stage 4, we identified the role of each part of speech in lexical density values

across grades and discourses.

Research results

On Stage 1, we measured and normalized to 1000 words those text features which are expected to be confirmed as complexity predictors (cf. Table 2).

Table 2. Linguistic features of texts of three grade levels (I – III)

Таблица 2. Лингвистические параметры текстов трех образовательных уровней (I – III)

The intra-discourse analysis on Stage 2 revealed that readability indices of Social science textbooks are higher than those of Science (line 1) which means that they are more difficult to comprehend as their word and/or sentence lengths are longer.

Noun and verbal ratios per sentence (lines 2, 3) which reflect lexical density per sentence are also higher across the grades in texts of Social science books. We can also see that while indices of nouns and adjectives per 1000 (lines 4, 5) grow slightly across grades, the number of verbs (line 6) declines in texts of both discourses. Metrics of adverbs and lexical density raise marginally in the texts of Social science and Science (lines 7, 8). The lexical density dynamics in Science and Social Studies textbooks slightly differs, and nominalization index is marginally higher in Social Studies textbooks (line 8).

Striking distinctions are observed in the number of verbs in present and past tenses (lines 9, 10): the share of verbs in the present tense is much higher than that of the past which may be viewed as a discourse dissimilarity. The index of “Verbs in past tense per 1000 words” is more than two times higher in the past tense in Social Studies texts than in Science. The opposite trend is observed in the parameter "Verbs in present tense per 1000 words": verbs in the present tense are approximately 1.5 times less common in Social Studies texts than in Science. The obvious reason is disciplines specifics. Science texts present and describe animal species, their habitats, as well as the work of organs and physiological systems of the body. All the above are areas of functioning present tense verbs. The past tense is used less frequently than in the social sciences and is largely related to fewer topics: history and development of biology, theory of animal evolution, and description of extinct species. In Social studies, ratio of past tense verbs is higher, because practically every social problem has its own background, distant past and in some cases even antiquity.

Stage 3. The research shows, that on each level, there are statistically significant differences between the linguistic parameters of educational texts in two disciplines. 5 features on Level I have statistically significant differences: Flesh-Kincaid, Verbal elements/sentence, Noun elements/sentence. On Levels II and III, the number of differences between academic texts increases dramatically.

Figure 1. a) Flesh-Kincaid (Social Studies); b) Flesh-Kincaid (Science)

Рисунок 1. а) Читабельность по Флешу-Кинкейду (Обществознание); b) Читабельность по Флешу-Кинкейду (Естествознание)

а

As we can be seen from Figure 1, text complexity increases from Level I to Level III. However, the dynamics of text complexity rise in two discipline discourses differs. In social studies texts, there is a sharp increase in complexity from Level I to Level II. In Science, the complexity increase is observed on the final level.

Figure 2. a) Lexical density per 1000 words (Social Studies); b) Lexical density per 1000 words (Science)

Рисунок 2. а) Лексическая плотность на 1000 слов (Обществознание); b) Лексическая плотность на 1000 слов (Естествознание)

а

в

Figure 2 demonstrates that readability growth is accompanied with the lexical density increase: this is the pattern characteristic of both, i.e. Social Studies and Science discourses. However, this increase is far from being significant. For example, Lexical density in Science texts increases from 474.89 (on Level 1) to 501.22 (on Level 3), which is only a 6% increase. In Social Studies texts, these changes are even less visible and amount to 3%. Whereas the parameter ‘Adjectives per 1000 words as a part of Lexical density, increased by 23% in Science texts, and by 14% in Social Studies. In this case, this indicates that with an increase in the texts grade level, they begin to use more adjectives, thereby becoming more descriptive and allowing a more complete disclosure of a particular concept typically expressed by a noun.

It can also be assumed that a grade level increase is accompanied by a change in the ratio of parts of speech in the text, while the lexical density increases slightly, since its components dynamics are divergent.

On Stage 4, we focused on the role of each part of speech in lexical density values across grades and discourses. To identify the relationship between the grade level and the number of parts of speech in the text, we employed Spearman's rank correlation analysis. Table 3 below shows Spearman Rank Order Correlations of the parameters and text grade levels investigated.

Таблица 3. Коэффициенты корреляции (Spearman Rank Order Correlations) лингвистических параметров текстов и уровня текста