DRDD: AI-Powered Multi-Modal System for Diabetic Retinopathy Disease Detection

EasyChair Preprint 15956

Abstract

Diabetic Retinopathy is a disease due to diabetes which destroy the blood vessels supplying the retina’s light sensitive tissue and resulting in vision loss and complete blindness. This disease primarily affects people of working age and increases the socioeconomic burden on individuals and the healthcare system. The automated detection for diabetic retinopathy using machine learning techniques with the help of retinal images and structured data of the patient helps to enhance the accuracy rate and efficiency of diagnosis.

 This work proposes the Transformer - Based Fusion model that integrates deep learning and structured clinical data analysis to improve diabetic retinopathy detection. This model combines EfficientNet-B3, Vision Transformer, which is used in feature extraction and detect changes in retinal images, and TabNet, used to analyze structured clinical data of a patient to improve the accuracy of the result. Generative AI model is also integrated with the help of GPT-4 to give a personalized medication suggestion to patients according to their health condition and medical history.  This fusion method significantly demonstrates the accuracy of 94.7% in disease detection.

Keyphrases: Diabetic Retinopathy, Gen AI., Retinal fundus image, Structured patient data, Transformer-Based Fusion model

@booklet{EasyChair:15956,
  author    = {Sofia A Sathya and Devi D Soundarya and Shree A Viniksha and E Yashika and K Yuvashree},
  title     = {DRDD: AI-Powered Multi-Modal System for Diabetic Retinopathy Disease Detection},
  howpublished = {EasyChair Preprint 15956},
  year      = {EasyChair, 2025}}