Advanced Hybrid Reasoning and Transfer Learning on Multimodal Data with Transformers

Reasoning is a vital process in machine learning (ML), involving making infer- ences and drawing conclusions based on data. This capability is important for developing intelligent systems which can understand and predict complex pat- terns. The study investigates reasoning through two distinct methodologies: multimodal reasoning and transfer learning based reasoning. In the first approach, multimodal reasoning is used with a semi-supervised method to label unlabeled datasets. In the second approach, transfer learning has been used to transfer knowledge of data from one model to another. Both approaches are demonstrated using unlabeled vehicular telemetry data. During processing, three sets of teleme- try data are used to extract features separately through the autoencoder. These features are then clustered and aligned to create labelled and unlabeled datasets. The eXtreme Gradient Boosting (XGBoost) algorithm achieved over 98% test accuracy when applied to the labelled datasets and was then used to predict labels for the unlabeled datasets, which were later added to the labelled dataset to form three datasets for further processing. In transfer learning, a transformer model specifically designed to handle continuous features is developed. Labelled datasets are applied to the transformer model, one after the other, resulting in three final models, with each model achieving over 80% accuracy. The model’s prediction confidence is also validated using conformal learning, where the final models achieved over 80% accuracy. The transformer model is also separately trained on the datasets and compared with traditional machine learning mod- els, outperforming the others by achieving an accuracy of 98%. By building on the groundwork laid by this study, future research can push the boundaries of what is possible with reasoning approaches, opening up new paths for scientific exploration and practical applications in different fields