Meta Optimizer Algorithms for Automated Fine-Tuning of Massive Multimodal Models

Abstract

The recent breakthroughs in artificial intelligence have helped to develop massive multi-modal models that can process heterogeneous data like text, images, audio, and video. Despite their impressive results in areas like healthcare diagnostics, autonomous systems, robotics, and vision-language learning, fine-tuning these models is computationally expensive and requires considerable hyperparameter tuning efforts. In this paper, a new meta optimizer framework is proposed for automatic fine-tuning of massive multi-modal models. The main objective is to increase the efficiency of the optimization process, ensure stability, and make the model more scalable. The new framework incorporates advanced features like adaptive learning, dynamic hyperparameter tuning, feedback-based optimization, and parameter-efficient fine-tuning to help achieve the scalability of a multi-modal learning environment. The experimental evaluation of the meta optimizer has been performed on multi-modal benchmark datasets and has been compared with the state-of-the-art optimization techniques such as SGD, Adam, AdamW, RMSProp, and Lion Optimizer. The results showed that the proposed meta optimizer achieved the best accuracy and F1-score values, i.e., 96.4% and 0.95, respectively, with the lowest number of convergence epochs being 16 as opposed to 42 for SGD and 31 for Adam. In addition, the proposed framework was able to reduce the training time to 5.1 hours, along with minimizing the GPU memory requirement to 19GB. It can be seen that the training process is quite efficient computationally and scalable as well.