Download PDFOpen PDF in browserStepwise AI Interpretive Approach for Mutimodal Data FusionEasyChair Preprint 139127 pages•Date: July 10, 2024AbstractIn recent years, Artificial Intelligence technology has excelled in various tasks and is taking the world by storm. However, the various transformations in neural networks make it difficult to make sense of the reasons why decisions are made. For this reason, trustworthy AI techniques have started gaining popularity. To this end, this study uses AI interpretability as an anchor point to point to the field of data fusion for multimodal AI for in-depth insights. The paper proposed a Stepwise AI Interpretative (SAII) approach using different pairing methods of 'one-to-one' and 'many-to-many' in an attempt to illustrate/demonstrate the interpretability of the process of pairing images and text. A counterfactual instantiation method was used to compare the whole-local relationship between a set of images and their associated descriptive text. The approach was evaluated via ‘task performance’. Keyphrases: Trustworthy AI, data fusion, multimodal
|