Transformer-Based Natural Language Processing Model for Automated Clinical Diagnosis and Electronic Health Record Summarization

Abstract

Electronic Health Records (EHRs) hold vast amounts of unstructured clinical notes that would be hard to efficiently analyze within traditional healthcare information systems. Correct interpretation of the clinical notes is crucial for prompt diagnosis, adequate treatment planning, and decrease in physician documentation workload. Yet clinical texts may present certain terminology that is ambiguous, some abbreviations, and some different writing styles, all of which pose challenges to automated medical text processing. This research aims to develop an automated clinical diagnosis prediction framework and an EHR summarization framework based on Natural Language Processing (NLP) and transformer. The proposed model uses contextual embedding, multi-head attention mechanisms and multitask learning to model disease classification and to summarize a patient's raw records into a concise description. Benchmark healthcare datasets such as the Medical Information Mission to Intelligent City (MIMIC-III) clinical notes were used for experimentation, and the clinical notes were preprocessed using tokenization, normalization, and medical entity extraction. The framework was assessed on a set of diagnosis prediction metrics (accuracy, precision, recall, F1-score, and AUROC) as well as summarization metrics (ROUGE and BLEU scores). Experimental results show that the model outperforms the current deep learning models in terms of the increased accuracy of prediction and the quality of summarization. The proposed framework has good potential for intelligent clinical decision support and healthcare documentation automation.