2313-8912Research Result. Theoretical and Applied Linguistics2313-891210.18413/2313-8912-2024-10-4-0-33674Large Language Models and Prompt Engineering in Linguistics<strong>Prompt injection &ndash; the problem of linguistic vulnerabilities of large language models at the present stage</strong><strong>Prompt injection &ndash; the problem of linguistic vulnerabilities of large language models at the present stage</strong>ZyryanovaIrina N.ZyryanovaIrina N.Irina_zyr@mail.ruChernavskiyAlexander S.ChernavskiyAlexander S.Chernavskiy.com@gmail.comTrubachevStanislav O.TrubachevStanislav O.brandei@yandex.ruOPS Guru LLCFederal State Budgetary Educational Institution of Higher Education «Moscow Pedagogical State University»The Federal State Budget Educational Institution of Higher Education «Baikal State University»202410400

The article examines the phenomenon of &ldquo;prompt injection&rdquo; in the context of contemporary large language models (LLMs), elucidating a significant challenge for AI developers and researchers. The study comprises a theoretical and methodological review of scholarly publications, thereby enhancing the comprehension of the present state of research in this field. The authors present the findings of a case study, which employs a comparative analysis of the linguistic vulnerabilities of prominent LLMs, including Chat GPT 4.0, Claude 3.5, and Yandex GPT. The study employs experimental evaluation to assess the resilience of these models against a range of vector attacks, with the objective of determining the extent to which each model resists manipulative prompts designed to exploit their linguistic capabilities. A taxonomy of prompt injection attack types was developed based on the collected data, with classification according to effectiveness and targeting of specific LLMs. This classification facilitates comprehension of the nature of these vulnerabilities and provides a basis for future research in this field. Moreover, the article offers suggestions for bolstering the resilience of language models against negative manipulations, representing a significant stride towards the development of safer and more ethical AI systems. These recommendations are based on empirical data and aim to provide practical guidance for developers seeking to enhance the resilience of their models against potential threats. The research findings extend our understanding of linguistic vulnerabilities in LLMs, while also contributing to the development of more effective defence strategies. These have practical implications for the deployment of LLMs across various domains, including education, healthcare and customer service. The authors emphasise the necessity for continuous monitoring and improvement of language model security in an ever-evolving technological landscape. The findings suggest the necessity for an ongoing dialogue among stakeholders to address issues pertaining to the prompt injection of funds.

The article examines the phenomenon of &ldquo;prompt injection&rdquo; in the context of contemporary large language models (LLMs), elucidating a significant challenge for AI developers and researchers. The study comprises a theoretical and methodological review of scholarly publications, thereby enhancing the comprehension of the present state of research in this field. The authors present the findings of a case study, which employs a comparative analysis of the linguistic vulnerabilities of prominent LLMs, including Chat GPT 4.0, Claude 3.5, and Yandex GPT. The study employs experimental evaluation to assess the resilience of these models against a range of vector attacks, with the objective of determining the extent to which each model resists manipulative prompts designed to exploit their linguistic capabilities. A taxonomy of prompt injection attack types was developed based on the collected data, with classification according to effectiveness and targeting of specific LLMs. This classification facilitates comprehension of the nature of these vulnerabilities and provides a basis for future research in this field. Moreover, the article offers suggestions for bolstering the resilience of language models against negative manipulations, representing a significant stride towards the development of safer and more ethical AI systems. These recommendations are based on empirical data and aim to provide practical guidance for developers seeking to enhance the resilience of their models against potential threats. The research findings extend our understanding of linguistic vulnerabilities in LLMs, while also contributing to the development of more effective defence strategies. These have practical implications for the deployment of LLMs across various domains, including education, healthcare and customer service. The authors emphasise the necessity for continuous monitoring and improvement of language model security in an ever-evolving technological landscape. The findings suggest the necessity for an ongoing dialogue among stakeholders to address issues pertaining to the prompt injection of funds.

Prompt injectionLarge language modelsLLMLLM vulnerabilitiesLLM jailbreaksecurity of AILinguistic attacks on LLMPrompts securityPrompt injectionLarge language modelsLLMLLM vulnerabilitiesLLM jailbreaksecurity of AILinguistic attacks on LLMPrompts securityСписок литературыChang,&nbsp;Z., Li,&nbsp;M., Liu,&nbsp;Y., Wang,&nbsp;J., Wang,&nbsp;Q. and Liu,&nbsp;Y. (2024). Play guessing game with LLM: Indirect jailbreak attack with implicit clues, arXiv preprint arXiv:2402.09091. https://doi.org/10.48550/arXiv.2402.09091 (In English)Chen,&nbsp;S., Zharmagambetov,&nbsp;A., Mahloujifar,&nbsp;S., Chaudhuri,&nbsp;K. and Guo,&nbsp;C. (2024). Aligning LLMs to Be Robust Against Prompt Injection.&nbsp;arXiv preprint arXiv:2410.05451. https://doi.org/10.48550/arXiv.2410.05451 (In English)Duan,&nbsp;M., Suri,&nbsp;A., Mireshghallah,&nbsp;N., Min,&nbsp;S., Shi,&nbsp;W., Zettlemoyer,&nbsp;L., Tsvetkov,&nbsp;Yu, Choi,&nbsp;Y., Evans,&nbsp;D. and Hajishirzi,&nbsp;H. (2024). Do membership inference attacks work on large language models?, arXiv preprint arXiv:2402.07841. https://doi.org/10.48550/arXiv.2402.07841 (In English)Hines,&nbsp;K., Lopez,&nbsp;G., Hall,&nbsp;M., Zarfati,&nbsp;F., Zunger,&nbsp;Y. and Kiciman,&nbsp;E. (2024). Defending Against Indirect Prompt Injection Attacks with Spotlighting, arXiv preprint arXiv:2403.14720. https://doi.org/10.48550/arXiv.2403.14720 (In English)Khandelwal,&nbsp;U., Levy,&nbsp;O., Jurafsky,&nbsp;D., Zettlemoyer,&nbsp;L. and Lewis,&nbsp;M. (2019). Generalization through memorization: Nearest neighbor language models, arXiv preprint arXiv:1911.00172. https://doi.org/10.48550/arXiv.1911.00172 (In English).Kumar,&nbsp;S.&nbsp;S., Cummings,&nbsp;M.&nbsp;L. and Stimpson,&nbsp;A. (2024, May). Strengthening LLM trust boundaries: A survey of prompt injection attacks, 2024 IEEE 4th International Conference on Human-Machine Systems (ICHMS), 1&ndash;6, available at: https://www.researchgate.net/profile/Missy-Cummings/publication/378072627_Strengthening_LLM_Trust_Boundaries_A_Survey_of_Prompt_Injection_Attacks/links/65c57ac379007454976ae142/Strengthening-LLM-Trust-Boundaries-A-Survey-of-Prompt-Injection-Attacks.pdf/ (Accessed 29.06.2024). DOI: 10.1109/ICHMS59971.2024.10555871 (In English)Li,&nbsp;X., Wang,&nbsp;R., Cheng,&nbsp;M.,&nbsp;Zhou,&nbsp;T. and Hsieh,&nbsp;C.&nbsp;J. (2024). Drattack: Prompt decomposition and reconstruction makes powerful llm jailbreakers.&nbsp;arXiv preprint arXiv:2402.16914. https://doi.org/10.48550/arXiv.2402.16914 (In English)Liu,&nbsp;X., Yu,&nbsp;Z., Zhang,&nbsp;Y., Zhang,&nbsp;N. and Xiao,&nbsp;C. (2024). Automatic and universal prompt injection attacks against large language models, arXiv preprint arXiv:2403.04957. https://doi.org/10.48550/arXiv.2403.04957 (In English)Marvin,&nbsp;G. Hellen,&nbsp;N., Jjingo,&nbsp;D. and Nakatumba-Nabende,&nbsp;J. 2023). Prompt engineering in large language models, Proceedings of the International conference on data intelligence and cognitive informatics, Springer Nature Singapore, Singapore, 387&ndash;402, available at: https://www.researchgate.net/publication/377214553_Prompt_Engineering_in_Large_Language_Models (accessed 29.06.2024). DOI: 10.1007/978-981-99-7962-2_30 (In English)Mudarova,&nbsp;R., Namiot,&nbsp;D. (2024). Countering Prompt Injection attacks on large language models, International Journal of Open Information Technologies, 12&nbsp;(5), 39&ndash;48. (In Russian)Pedro,&nbsp;R., Castro,&nbsp;D., Carreira,&nbsp;P. and Santos,&nbsp;N. (2023). From prompt injections to SQL injection attacks: How protected is your llm-integrated web application?, arXiv preprint arXiv:2308.01990. https://doi.org/10.48550/arXiv.2308.01990 (In English)Piet,&nbsp;J., Alrashed,&nbsp;M., Sitawarin,&nbsp;C., Chen,&nbsp;S., Wei,&nbsp;Z., Sun,&nbsp;E. and Wagner,&nbsp;D. (2023). Jatmo: Prompt injection defense by task-specific finetuning, arXiv preprint arXiv:2312.17673. https://doi.org/10.48550/arXiv.2312.17673 (In English)R&ouml;ttger,&nbsp;P., Pernisi,&nbsp;F., Vidgen,&nbsp;B. and Hovy,&nbsp;D. (2024). Safety prompts: a systematic review of open datasets for evaluating and improving large language model safety, arXiv preprint arXiv:2404.05399.m. https://doi.org/10.48550/arXiv.2404.05399 (In English)Rossi,&nbsp;S., Michel,&nbsp;A&nbsp;M., Mukkamala,&nbsp;R.&nbsp;R. and Thatcher,&nbsp;J.&nbsp;B. (2024). An Early Categorization of Prompt Injection Attacks on Large Language Models, arXiv preprint arXiv:2402.00898. https://doi.org/10.48550/arXiv.2402.00898 (In English)Tavabi,&nbsp;N., Goyal,&nbsp;P., Almukaynizi,&nbsp;M., Shakarian,&nbsp;P. and Lerman,&nbsp;K. (2018). Darkembed: Exploit prediction with neural language models, Proceedings of the AAAI Conference on Artificial Intelligence, 32, 1, 7849&ndash;7854. https://doi.org/10.1609/aaai.v32i1.11428 (In English)Yan,&nbsp;J., Yadav,&nbsp;V., Li,&nbsp;S., Chen,&nbsp;L., Tang,&nbsp;Z., Wang,&nbsp;H. and Jin,&nbsp;H. (2024). Backdooring instruction-tuned large language models with virtual prompt injection, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, Long Papers, 6065&ndash;6086. DOI: 10.18653/v1/2024.naacl-long.337 (In English)Yu,&nbsp;J., Wu,&nbsp;Y., Shu,&nbsp;D., Jin,&nbsp;M., Yang, S. and Xing,&nbsp;X. (2023). Assessing prompt injection risks in 200+ custom GPTS, arXiv preprint arXiv:2311.11538. https://doi.org/10.48550/arXiv.2311.11538 (In English)Yu,&nbsp;Z., Liu,&nbsp;X., Liang,&nbsp;S., Cameron,&nbsp;Z., Xiao,&nbsp;C. and Zhang,&nbsp;N. (2024). Don&#39;t Listen to Me: Understanding and Exploring Jailbreak Prompts of Large Language Models, arXiv preprint arXiv:2403.17336. https://doi.org/10.48550/arXiv.2403.17336 (In English)Zhang,&nbsp;J. (2024). Should We Fear Large Language Models? A Structural Analysis of the Human Reasoning System for Elucidating LLM Capabilities and Risks Through the Lens of Heidegger&rsquo;s Philosophy, arXiv preprint arXiv:2403.03288. https://doi.org/10.48550/arXiv.2403.03288 (In English)