| ||||
| ||||
![]() Title:From Concepts to Evidence: Literature-Grounded Skin Lesion Diagnosis with Vision–Language and Retrieval-Augmented Models Authors:Martin Heckmann, Gabriel Santos Martins Dias, Willian Amorim, Pedro Henrique Bugatti, Cid Adinam Nogueira Santos and Priscila Tiemi Maeda Saito Conference:IEEE CBMS 2026 Tags:Dermatological Diagnosis, Evidence-Grounded Medical AI, Retrieval-Augmented Generation and Vision-Language Models Abstract: Artificial intelligence has shown strong potential for dermatological diagnosis, but clinical adoption requires not only accurate predictions but also transparent and verifiable reasoning. Recent concept-based approaches improve interpretability by exposing clinically meaningful dermoscopic features predicted from images; however, the diagnostic explanations produced by large language models (LLMs) remain internally generated and are not explicitly grounded in external biomedical evidence. We propose a literature-grounded diagnostic framework that integrates dermoscopic concept extraction with retrieval-augmented generation (RAG) over a dermatology-focused PubMed corpus. The framework combines vision–language models for lesion concept prediction, dense biomedical literature retrieval, multi-stage reranking strategies, and LLM-based reasoning to generate diagnoses accompanied by explanations supported by retrieved scientific evidence. Beyond introducing this architecture, we conduct a systematic analysis of retrieval design choices—including diversity-based retrieval, cross-encoder reranking, and natural language inference filtering—and examine how these strategies influence both diagnostic performance and explanation grounding. Experiments on Derm7pt, HAM10000, and PH² under the same binary setting used in prior concept-based work (nevus vs. melanoma) achieve balanced accuracy up to 0.792, 0.735, and 0.831, respectively. Our results reveal a trade-off between retrieval diversity and semantic precision: diversity-oriented strategies improve diagnostic performance, while precision-oriented reranking yields explanations more strongly supported by biomedical evidence. By explicitly linking predictions to retrievable scientific literature and enabling claim-level grounding analysis, the proposed framework supports auditable and evidence-grounded dermatological AI systems. From Concepts to Evidence: Literature-Grounded Skin Lesion Diagnosis with Vision–Language and Retrieval-Augmented Models ![]() From Concepts to Evidence: Literature-Grounded Skin Lesion Diagnosis with Vision–Language and Retrieval-Augmented Models | ||||
| Copyright © 2002 – 2026 EasyChair |
