Large language models (LLMs) are performing an increasing variety of tasks, including generating long texts that aspire to literary quality. The aim of this article is to compare subjective assessments of the quality of literary texts created by large language models with human-written texts in the genre of fan fiction based on the Harry Potter novels. The material for the evaluation consisted of six texts in English: three generated by AI using different approaches, and three written by human authors. The present study utilizes a psycholinguistic experimental design. Eighteen informants evaluated the texts according to eight criteria reflecting key categories of literariness: coherence (integrity), semantic and lexico-grammatical cohesion, narrative rhythm, emotionality, subjective modality of doubt, syntactic complexity, and expressiveness (idiomaticity, creativity, strong textual positions). The data processing included statistical analysis of objective parameters of linguistic complexity in the texts, subjective evaluations obtained from informants, as well as clustering of vectorized retellings of the texts provided by the informants.

The results showed that the gap between human and AI-generated texts varies across different categories. Thanks to specific generation methods, the synthetic texts managed to approach human-written ones in terms of overall coherence and syntactic complexity. However, the most significant differences emerged in the categories of rhythm, semantic cohesion, and the presence of the subjective modality of doubt, which readers rated significantly lower in the synthetic texts. Nevertheless, one of the generated texts was of a quality comparable to that of a human writer in terms of idiomaticity, creativity and the ability to take a strong position on a topic. Clustering of summaries confirmed that this text, unlike the other two generated ones, led to a more uniform understanding of the plot among readers.

It can be concluded that the main challenge for AI is not so much the construction of a coherent, hierarchical text structure, but rather the creation of a dynamic, psychologically authentic and rhythmically organised narrative. The categories of semantic cohesion, rhythm, and modality of doubt currently represent the frontier separating artificial intelligence from human intelligence in the realm of literary creativity.