Learning Under Uncertainty in Sparse and Noisy Data Environments

Abstract

Learning from real-world data is challenging because sparse observations, stochastic noise, and uncertainty often reduce the reliability and predictive capability of conventional machine learning models. Existing approaches mainly focus on either noise mitigation or sparse data handling and often fail to simultaneously model aleatoric and epistemic uncertainties, thereby leading to unstable predictions in uncertain environments. To address this limitation, this research proposes an uncertainty-aware learning framework based on a Chaotic Dragonfly Algorithm-driven Efficient Deep Neural Network (CDA-EDNN) for accurate prediction in sparse and noisy data environments. Research utilizes the Uncertainty Sparse Noisy Signal Dataset, containing signal characteristics associated with noise, sparsity, uncertainty, and prediction quality. Data preprocessing is performed using Min-Max Normalization to scale feature values and reduce magnitude variations. Subsequently, Independent Component Analysis (ICA) is employed for feature extraction to separate latent source signals and enhance signal representation. The CDA optimizes network parameters, while the EDNN learns robust feature patterns under uncertainty conditions. Experimental results demonstrate that the proposed model achieves superior stability with an Area under the Precision-Recall Curve (AUPRC) of 0.96, an accuracy of 0.98, and an F1-Score of 0.97 compared to standalone machine learning and traditional interpolation methods. The model is implemented using Python 3.10, TensorFlow 2.x, NumPy, Pandas, and Matplotlib on a Graphics Processing Unit (GPU)-enabled computing platform. Overall, the proposed CDA-EDNN framework provides a scalable and reliable solution for prediction tasks involving sparse, noisy, and uncertain data.