Enhancing Medical Text Summarization using Transformer-Based NLP Models for Clinical Decision Support
Downloads
Medical text summarization plays a crucial role in clinical decision support by enabling healthcare professionals to quickly access essential information from vast amounts of unstructured medical texts. With the rapid advancements in Natural Language Processing (NLP), transformer-based models have emerged as powerful tools for generating high-quality summaries. This paper investigates the effectiveness of state-of-the-art transformer models, such as BERT, GPT, and T5, in summarizing medical texts while preserving critical information. We conduct comprehensive evaluations using benchmark datasets and assess the performance of these models in terms of coherence, relevance, and readability. The experimental results demonstrate that transformer-based models significantly outperform traditional extractive and abstractive summarization techniques, offering more accurate and contextually meaningful summaries. Furthermore, we highlight the importance of domain-specific pretraining and fine-tuning to enhance model performance in medical applications. This study provides valuable insights into the practical deployment of transformer-based summarization models in healthcare settings, ultimately contributing to improved clinical workflows and informed decision-making.
SABBIR, M. A. I. Data-Driven Healthcare: Exploring Biomedical Text Mining Through NLP Models.
Ghosh, A., Tomar, M., Tiwari, A., Saha, S., Salve, J., & Sinha, S. (2024, August). From sights to insights: Towards summarization of multimodal clinical documents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 13117-13129).
van Zandvoort, D., Wiersema, L., Huibers, T., van Dulmen, S., & Brinkkemper, S. (2023). Enhancing summarization performance through transformer-based prompt engineering in automated medical reporting. arXiv preprint arXiv:2311.13274.
Sun, D., He, J., Zhang, H., Qi, Z., Zheng, H., & Wang, X. (2025). A LongFormer-Based Framework for Accurate and Efficient Medical Text Summarization. arXiv preprint arXiv:2503.06888.
Singh, A. (2025). Automated Patient History Summarization using Generative AI. Available at SSRN 5184805.
Balouch, B. A. K., & Hussain, F. (2023, July). A transformer-based approach for abstractive text summarization of radiology reports. In International Conference on Applied Engineering and Natural Sciences (Vol. 1, No. 1, pp. 476-86).
Kalusivalingam, A. K., Sharma, A., Patel, N., & Singh, V. (2021). Leveraging BERT and LSTM for Enhanced Natural Language Processing in Clinical Data Analysis. International Journal of AI and ML, 2(3).
Vale, L., & Lee, A. (2024). Advancing Medical Diagnosis: Enhancing Sentiment Analysis in Electronic Medical Records with Transformer Models. Journal of Computer Technology and Software, 3(7).
Arokodare, O., Wimmer, H., & Du, J. (2024). Clinical Text Summarization using NLP Pretrained Language Models: A Case Study of MIMIC-IV-Notes. In Proceedings of the ISCAP Conference ISSN (Vol. 2473, p. 4901).
Sun, Z., Liu, J., Chen, X., Wang, G., Zhang, L., & Zhou, M. Leveraging Pretrained Transformers for Medical Text Summarization.
Cruz Montealegre, M. C. (2023). Enhancing medical diagnosis through NLP in clinical texts for informed healthcare decisions.
Khan, F. A. (2024). ENHANCING ELECTRONIC HEALTH RECORDS SYSTEMS AND DIAGNOSTIC DECISION SUPPORT SYSTEMS WITH LARGE LANGUAGE MODELS (Doctoral dissertation, Purdue University Graduate School).
Chaves, A., Kesiku, C., & Garcia-Zapirain, B. (2022). Automatic text summarization of biomedical text data: a systematic review. Information, 13(8), 393.
Cho, H. N., Jun, T. J., Kim, Y. H., Kang, H., Ahn, I., Gwon, H., ... & Ko, S. (2024). Task-Specific Transformer-Based Language Models in Health Care: Scoping Review. JMIR Medical Informatics, 12, e49724.
Shah-Mohammadi, F., & Finkelstein, J. (2024). Extraction of Substance Use Information From Clinical Notes: Generative Pretrained Transformer–Based Investigation. JMIR Medical Informatics, 12, e56243.
Ahmed, U., Iqbal, K., Aoun, M., & Khan, G. (2023). Natural language processing for clinical decision support systems: a review of recent advances in healthcare. J Intell Connect Emerg Technol, 8(2), 1-17.
Van Veen, D., Van Uden, C., Blankemeier, L., Delbrouck, J. B., Aali, A., Bluethgen, C., ... & Chaudhari, A. S. (2023). Clinical text summarization: adapting large language models can outperform human experts. Research square, rs-3.
Hossain, M. R., Mahabub, S., Al Masum, A., & Jahan, I. (2024). Natural Language Processing (NLP) in Analyzing Electronic Health Records for Better Decision Making. Journal of Computer Science and Technology Studies, 6(5), 216-228.
Rani, S., & Jain, A. (2024). Optimizing healthcare system by amalgamation of text processing and deep learning: a systematic review. Multimedia Tools and Applications, 83(1), 279-303.
Yang, F., Wang, X., Ma, H., & Li, J. (2021). Transformers-sklearn: a toolkit for medical language understanding with transformer-based models. BMC Medical Informatics and Decision Making, 21, 1-8.
Prakash, C., Shaikh, M., Mutha, P., & Parimi, P. K. A Systematic Analysis of Accuracy and Trust in Clinical Decision Making with NLP.
Kumar, A., Sharma, A., & Sangwan, S. R. (2025). Transformer-Based Abstractive Summarization for Depression Detection Literature for Enhanced Medical Insights. Available at SSRN 5029494.
