From STM to GPT: A Comparative Study of Topic Modeling Methods for AI in Dentistry
This study presents a comprehensive topic modeling analysis of scientific abstracts in the field of artificial intelligence (AI) applied to dentistry. We compiled and analyzed 3,170 peer-reviewed abstracts published between 2019 and 2025 from the Dimensions and Scopus databases. Three complementary approaches were compared: (1) Structural Topic Modeling (STM), a probabilistic framework incorporating publication year as a covariate to enable temporal trend analysis; (2) embedding-based clustering using the Leiden algorithm on OpenAI text embeddings; and (3) zero-shot GPT-based topic modeling, in which GPT-4o generated topics, descriptions, and keywords directly from batches of abstracts without model training. Topic quality was evaluated using compactness, distinctiveness, silhouette scores, and label redundancy. STM consistently produced the most compact and well-separated topics, embedding-based clustering excelled in identifying discrete semantic groupings, and GPT-based modeling provided interpretable, human-readable labels but exhibited greater thematic overlap. To ensure comparability across methods, we introduced a two-layer alignment framework that integrates topic-level similarity with document-level consensus, enabling robust cross-model comparison. Using this framework, we identified stable topics consistently recovered across methods (e.g., caries detection, radiographic AI diagnostics) as well as method-specific themes. Temporal trend analysis in this shared space revealed a clear shift from foundational AI methods (e.g., image segmentation, image enhancement) toward applied and integrative areas, including large language model applications, patient-facing tools, and AI in clinical education. Our results underscore the value of combining classical probabilistic models with modern large language model (LLM) tools for optimal topic modeling performance. While GPT-4o enhances interpretability, it should not be used in isolation for mapping thematic structures in scientific literature, at least not without pre-screening and prompt experimentation. Overall, our findings demonstrate the importance of hybrid topic modeling for mapping thematic structures in fast-evolving scientific domains, with dentistry serving as a case study.
Figures
Litvinova, T. A., Ippolitov, Y. A., Seredin, P. V. (2025). From STM to GPT: A Comparative Study of Topic Modeling Methods for AI in Dentistry, Research Result. Theoretical and Applied Linguistics, 11 (3), 85–121.
While nobody left any comments to this publication.
You can be first.
Allani, H., Santos, A. T. and Ribeiro-Vidal, H. (2024). Multidisciplinary Applications of AI in Dentistry: Bibliometric Review, Applied Sciences, 14 (17), 7624. https://doi.org/10.3390/app14177624(In English)
Benz, P., Pradier, C., Kozlowski, D., Shokida, N. S. and Larivière, V. (2025). Mapping the unseen in practice: comparing latent Dirichlet allocation and BERTopic for navigating topic spaces, Scientometrics, 10 June 2025. https://doi.org/10.1007/s11192-025-05339-6(In English)
Blei, D. M., Ng, A. Y. and Jordan, M. I. (2003). Latent dirichlet allocation, J. Mach. Learn. Res., 3, 993–1022. (In English)
Büttner, M., Leser, U., Schneider, L. and Schwendicke, F. (2024). Natural Language Processing: Chances and Challenges in Dentistry, Journal of Dentistry, 141, 104796. https://doi.org/10.1016/j.jdent.2023.104796(In English)
Cosola, Di M., Ballini, A., Prencipe F. A., De Tullio, F., Fabrocini, A., Cupelli, P., Rignani M., Cazzolla A. P., Bizzoca M. E., Musella G. (2025). Artificial intelligence in dentistry: a narrative review of applications, challenges, and future directions, Minerva Dent Oral Sci, Jun 18. https://doi.org/10.23736/S2724-6329.25.05217-9(In English)
de Magalhães, A. A. and Santos, A. T. (2025). Advancements in Diagnostic Methods and Imaging Technologies in Dentistry: A Literature Review of Emerging Approaches, Journal of Clinical Medicine, 14 (4), 1277. https://doi.org/10.3390/jcm14041277(In English)
Grootendorst, M. (2022). Bertopic: Neural topic modeling with a class-based tf-idf procedure, arXiv preprint arXiv:2203.05794 [Online], available at https://arxiv.org/abs/2203.05794 (accessed 22.08.2025) (In English)
Hu, J., Miao, C., Wu, Yi and Su, J. (2025). Advances in hydrological research in China over the past two decades: Insights from advanced large language model and topic modeling, Fundamental Research [Online], available at https://doi.org/10.1016/j.fmre.2025.05.002 (accessed 10.05.2025) (In English)
Islam, K. M. S. (2025). Contextual Embedding-based Clustering to Identify Topics for Healthcare Service Improvement, arXiv:2504.14068, https://doi.org/10.48550/arXiv.2504.14068(In English)
Jung, H. S., Lee, H., Woo, Y. S., Baek, S. Y. and Kim, J. H. (2024). Expansive data, extensive model: Investigating discussion topics around LLM through unsupervised machine learning in academic papers and news, Plos One, 19 (5), e0304680. https://doi.org/10.1371/journal.pone.0304680(In English)
Kozlowski, D. (2024). Generative AI for automatic topic labelling, arXiv:2408.07003. https://doi.org/10.48550/arxiv.2408.07003(In English)
Lee, V. V., van der Lubbe, S. C. C., Goh, L. H. and Valderas, J. M. (2024). Harnessing ChatGPT for Thematic Analysis: Are We Ready?. Journal of medical Internet research, 26, e54974. https://doi.org/10.2196/54974(In English)
Lee, Y., Oh, J. H., Lee, D., Kang, M. and Lee, S. (2025). Prompt engineering in ChatGPT for literature review: practical guide exemplified with studies on white phosphors, Sci Rep, 15, 15310. https://doi.org/10.1038/s41598-025-99423-9(In English)
Mahmoud, M., Mashaly, M. and Mervat, A.-E. (2025). Predicting Software Engineering Trends from Scientific Papers with a Combined Framework of Clustering and Topic Modeling, in 2025 15th International Conference on Electrical Engineering (ICEENG), 1–6. https://doi.org/10.1109/ICEENG64546.2025.11031347(In English)
Mathis, W. S., Zhao, S., Pratt, N., Weleff, J. and De Paoli, S. (2024). Mathis Inductive thematic analysis of healthcare qualitative interviews using open-source large language models: How does it compare to traditional methods?, Computer Methods and Programs in Biomedicine, 255. https://doi.org/10.1016/j.cmpb.2024.108356(In English)
Meng, F., Lu, Z., Li, X., Han, W., Peng, J., Liu, X. and Niu, Z. (2024). Demand-side energy management reimagined: A comprehensive literature analysis leveraging large language models, Energy, 291, 130303. https://doi.org/10.1016/j.energy.2024.130303(In English)
Mu, Y., Bai, P., Bontcheva, K. and Song, X. (2024a). Addressing topic granularity and hallucination in large language models for topic modelling. arXiv preprint arXiv:2405.00611v1 [Online], available at https://arxiv.org/html/2405.00611v1 (accessed 22.08.2025) (In English)
Mu, Y., Dong, C., Bontcheva, K. and Song, X. (2024b). Large language models offer an alternative to the traditional approach of topic modelling. In LREC-COLING 2024, ELRA Language Resource Association, 2024, 10160–10171. (In English)
Ogunleye, B., Lancho Barrantes, B. S. and Zakariyyah, K. I. (2025). Topic modelling through the bibliometrics lens and its technique, Artif Intell Rev, 58, 74. https://doi.org/10.1007/s10462-024-11011-x(In English)
Pham, C. M., Hoyle, A., Sun, S., Resnik, P. and Iyyer, M. 2024. TopicGPT: A Prompt-based Topic Modeling Framework, in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2956–2984, Mexico City, Mexico. Association for Computational Linguistics. (In English)
Reuter, A., Thielmann, A., Weisser, C., Fischer, S. and Säfken, B. (2024). Gptopic: Dynamic and interactive topic representations, arXiv preprint arXiv:2403.03628 [Online], available at https://arxiv.org/abs/2403.03628 (accessed 22.08.2025) (In English)
Riaz, A., Abdulkader, O., Ikram, M. J. and Sadaqat, J. (2025). Exploring topic modelling: a comparative analysis of traditional and transformer-based approaches with emphasis on coherence and diversity, International Journal of Electrical and Computer Engineering (IJECE), [S.l.], 15 (2), 1933–1948. http://doi.org/10.11591/ijece.v15i2.pp1933-1948(In English)
Roberts, M. E., Stewart, B. M. and Tingley, D. (2019). Stm: An R package for structural topic models, J. Stat. Softw. 91 (2), 1–40. https://doi.org/10.18637/jss.v091.i02(In English)
Şakar, S. and Tan, S. (2025). Research Topics and Trends in Gifted Education: A Structural Topic Model, Gifted Child Quarterly, 69 (1), 68–84. https://doi.org/10.1177/0016986224128504(In English)
Sbalchiero, S. and Eder, M. (2020). Topic modeling, long texts and the best number of topics. Some Problems and solutions, Qual Quant, 54, 1095–1108. https://doi.org/10.1007/s11135-020-00976-w(In English)
Shapurian, G. (2024). Large Language Models and Knowledge Graphs for Astronomical Entity Disambiguation, arXiv:2406.11400 [Online], available at: https://arxiv.org/pdf/2406.11400 (accessed 22.08.2025) (In English)
Sharma, A., Wallace, J. R. (2025). DeTAILS: Deep Thematic Analysis with Iterative LLM Support, in Proceedings of the 7th ACM Conference on Conversational User Interfaces (CUI '25). Association for Computing Machinery, New York, NY, USA, Article 28, 1–7. https://doi.org/10.1145/3719160.3735657(In English)
Shirani, M. (2025). Trends and Classification of Artificial Intelligence Models Utilized in Dentistry: A Bibliometric Study, Cureus, 17 (4): e81836. https://doi.org/10.7759/cureus.81836(In English)
Silveira, L. (2024). Cone Beam Computed Tomography and Artificial Intelligence. ¿Where We Are? Rev Cient Odontol, 12 (4), e214. https://doi.org/10.21142/2523-2754-1204-2024-214(In English)
Tarek, A., Mahmoud, M., Afifi, B., Mashaly, M. and Abu-Elkheir, M. (2024). Query-Based Topic Modeling and Trend Analysis in Scientific Literature, in 2024 International Conference on Microelectronics (ICM), Doha, Qatar, 2024, 1–6. https://doi.org/10.1109/ICM63406.2024.10815706(In English)
Torres, J., Mulligan, C., Jorge, J. and Moreira, C. (2025). PROMPTHEUS: A Human-Centered Pipeline to Streamline Systematic Literature Reviews with Large Language Models, Information, 16 (5), 420. https://doi.org/10.3390/info16050420(In English)
Wu, X., Nguyen, T. and Luu, A. T. (2024). A survey on neural topic models: Methods, applications, and challenges, Artif Intell Rev 57(18): 1–30. https://doi.org/10.1007/s10462-023-10661-7(In English)
Xie, B., Xu, D., Zou, X. Q., Lu, M. J., Peng, X. L. and Wen, X. J. (2024). Artificial intelligence in dentistry: a bibliometric analysis from 2000 to 2023, J Dent Sci., 19, 1722–33. https://doi.org/10.1016/j.jds.2023.10.025(In English)
Zatt, F. P., Rocha, A. O., Anjos, L. M., Caldas, R. A., Cardoso, M. and Rabelo, G. D. (2024). Artificial intelligence applications in dentistry: a bibliometric review with an emphasis on computational research trends within the field, J Am Dent Assoc., 155 (9), 755–764. https://doi.org/10.1016/j.adaj.2024.05.013(In English)
The study is supported by Russian Science Foundation, grant number 23-15-00060.