Automating Business Intelligence Insights Using Deep Learning and NLP Algorithms

Abstract

Modern Business Intelligence Systems are constantly facing large volumes of both structured and unstructured information, such as financial statements or customer feedback. In this paper, an automatic business intelligence tool using deep learning algorithms combined with natural language processing methods is suggested to make more accurate and effective decisions. A combination of recurrent neural networks (or long short-term memory neural networks) and convolutional neural networks to manage structured time series data and convolutional neural networks to manage structured spatial data are implemented using a hybrid neural network architecture that incorporates both methods to simultaneously treat both kinds of data. Sentiment analysis of customer reviews, natural language processing, and latent Dirichlet allocation have also been utilized to perform text-based sentiment analysis of customer reviews. The developed model is trained using a data set of historical performance metrics and approximately 7250 customer reviews at a 70% to 15% to 15% ratio, respectively, among training, validation, and test datasets. The CNN model resulted in 94.2% accuracy with F1-score = 0.92, and the RNN/LSTM reached 95.6% accuracy with F1-score = 0.94. The hybrid model achieved the best results (97.1% and an F1-score of 0.96). Sentiment analysis indicated that 62.4% of the reviews were positive, while 25.7% were neutral and 11.9% were negative. Additionally, the comments addressed the following topics: Product Quality (40%), Delivery Times (30%), and Customer Service (20%). Comparison to previous studies showed the proposed model is significantly superior to the earlier models, with an accuracy of 85%-96%. Therefore, the results demonstrate that using deep learning algorithms in conjunction with Natural Language Processing has produced a more rapid, accurate, and scalable BI solution. Future research should investigate the use of transformer models and multi-modal datasets.