Diabetic retinopathy (DR) is the leading cause of blindness in the working-age population of the developed world. It is estimated to affect over 93 million people worldwide. Unfortunately, DR is not reversible and current therapy can only prevent further deterioration. Early detection of DR enables preventive treatment and vision loss. DR diagnosis is a time-consuming and expensive procedure of manual examination of retina fundus images carried out by ophthalmologists.

Moreover, in the absence of a well-defined diagnostic metric. i.e., a “gold standard,” it is prone to misdiagnosis. In this talk, I will present a semi-supervised multimodal machine learning model for early detection and diagnosis of DR using a combination of retinal images and medical records, where only a small subset of the data is labeled. Our unique solution is based on a deep neural network ensemble of multimodal (both image and tabular data). We developed a special semi-supervised iterative algorithm to handle this use case effectively. Our model is based on only a few thousand patients’ records and yet achieves state-of-the-art early DR detection based solely on the patient’s medical records. Periodical application of our model on patients’ records allows for early detection of DR and can save patients’ vision. This research collaborates with the data science group of Ben Gurion University and ophthalmologist experts from Soroka hospital.