Multimodal Reasoning is an emerging research area that aims to enable intelligent systems to reason and learn from information obtained from various modalities, such as language, images, videos, and sensor data. This workshop focuses on exploring different aspects of multimodal reasoning, including combining multimodal learners, language models, and attention mechanisms, evaluating the effectiveness of transfer learning and pre-training in multimodal reasoning, and examining the impact of data augmentation techniques. Additionally, we will explore how multimodal reasoning can be used to improve educational outcomes, healthcare outcomes, and other information processing tasks.

Workshop Description

This workshop is a follow-up to our IJCAI ‘19, 20, 21 AIMA4EDU workshops. It expanded the use of LLM and other foundation models to enable a more integrated intelligent system not only for education and healthcare. One day workshop. Program includes invited talks, presentations, discussion, final panel discussion, and interactive sessions.

Paper Submission and Publication

Authors should submit papers in IJCAI-2023’s format as PDF (Springer Format) on easychair (easychair.org/) To be updated, or send to richard.tong@ieee.org

Scope and Objectives

The objectives of this workshop are to:

• Provide a forum for researchers and practitioners to discuss the latest developments, challenges, and opportunities in multimodal reasoning.
• Exchange ideas and insights on using multimodal reasoning techniques to address real-world problems in education, healthcare, and other information processing fields.
• Identify potential future research directions and applications of multimodal reasoning.
• Identify and discussion standardization, deployment, operation and responsible AI practices

Relevant topic areas include (but not limited to)

• Develop new architectures and models for multimodal reasoning.
• Utilize multimodal reasoning for intelligent agents in education and healthcare settings.
• Combine multimodal learners with LLM reasoners using language symbols to enable information exchange through chain of thought.
• Develop multimodal representation in theory of mind and other hidden contexts Create a GAN framework to enable self-learning using multimodal generation. Cross-validate the model grounding by utilizing different modalities and combining different reasoning strategies.
• Investigate the use of attention mechanisms in multimodal reasoning.
• Explore the role of pre-training and fine-tuning in multimodal reasoning.
• Evaluate the effectiveness of transfer learning in multimodal reasoning.
• Examine the impact of data augmentation techniques on multimodal reasoning performance.
• Investigate the ethical implications of multimodal reasoning, including the impact of bias and fairness. Explore ethical considerations and standardization issues in the use of multimodal reasoning, especially with the IEEE AI Standards Committee and IEEE Learning Technology Standards Committee.
• Introduce and design new implementation approaches such as prompt engineering, local fine-tuning, integrated reasoning from LLM base, and incremental learning.

Important Dates

• Paper submission ultimate deadline: June 20th, 2023
• Notification of acceptance/rejection: July 4th, 2023
• Camera-ready version deadline: August 3rd, 2023
• Conference 19-25 August, 2023

Organizing Committee

• Richard Tong (IEEE AISC Chair), richard.tong@ieee.org
• Yiqiang Chen (Institute of Computing Technology), yqchen@ict.ac.cn
• Dr. Zitao Liu (Dean of Guangdong Institute of Smart Education, Jinan University)
• Joleen Liang (Squirrel AI)
• Jiahao Chen (TAL)

Members of program committee

• Carles Sierra (IIIA, Spain)
• Guodong Long (University of Technology Sydney)
• Ken Forbus (Northwestern University)
• Dapeng Oliver Wu (City University of Hong Kong)
• Zachary Pardos (UC Berkeley)
• Lingfei Teddy Wu (Pinterest)
• Tom Mitchell (Carnegie Mellon University)
• LP Morency (Carnegie Mellon University)
• Kang Lee (U. of Toronto)
• Jonathan Rowe (North Carolina State University)
• Jing Jiang (University of Technology Sydney)
• Shirui Pan (Monash University)
• Marcelo Worsley (Northwestern University )
• Dor Abrahamson (University of California, Berkeley)
• Paulo Blikstein (Columbia Teacher’s College)
• Bertrand Schneider (Harvard University)
• Sen Wang (University of Queensland)
• Lina Yao (University of New South Wales)
• Lujie Karen Chen (University of Maryland Baltimore County (UMBC))