FA-Transformer-MRMR: A Hybrid Feature Selection Framework Combining Firefly Algorithm, Transformer Networks, and Minimum Redundancy Maximum Relevance for Medical Data Analysis

Abstract

The increasing dimensionality, heterogeneity, and temporal complexity of intensive care unit (ICU) data pose significant challenges for developing accurate and interpretable predictive models. Conventional feature selection methods may fail to simultaneously address high-dimensional search spaces, temporal dependencies, and redundancy among clinical variables. To overcome these limitations, this study proposes FA-Transformer-mRMR, a hybrid feature selection framework integrating the Firefly Algorithm (FA), Transformer Networks, and Minimum Redundancy Maximum Relevance (mRMR). FA performs global exploration of the high-dimensional feature space to identify promising feature subsets, while the Transformer captures complex temporal dependencies and interactions within sequential clinical observations. Subsequently, mRMR refines the candidate features by maximizing their relevance to the target outcome while minimizing redundancy among selected variables. Thus, the proposed framework combines global optimization, temporal representation learning, and relevance-based feature refinement to obtain a compact and informative feature subset. The framework is evaluated using the Medical Information Mart for Intensive Care III (MIMIC-III) dataset and compared with conventional feature selection approaches, including Logistic Regression with L1 regularization (LR-LASSO) and Decision Tree with Gini Index (DT-Gini). Experimental results demonstrate improved feature-selection effectiveness and predictive performance compared with the baseline methods. The selected features also provide clinically meaningful insights into factors associated with patient outcomes, demonstrating the potential of FA-Transformer-mRMR for improving predictive modeling and supporting interpretable clinical decision-making in critical care.