IBM Research demonstrates impressive 86% accuracy in classifying diabetic retinopathy severity from eye scans – April 24, 2017

Yesterday, IBM Watson announced new topline data demonstrating impressive accuracy in classifying diabetic retinopathy severity using Watson’s deep learning and visual analytics technology. Using 35,000 eye images from the EyePACS database, the classification tool was trained to identify indicators of retinal blood vessel damage to assess the presence and severity of retinopathy, characterized by five categories: no retinopathy, mild, moderate, severe, and proliferative. Very impressively, the tool demonstrated 86% accuracy in classifying severity of the disease, far exceeding the performance of clinicians (who typically do well at yes/no presence of retinopathy, but have a harder time with minutely classifying retinopathy stages – we believe this is partly because some eye doctors do not want to deliver bad news). It only takes about 20 seconds for Watson to analyze an image, another plus. In a call, Dr. Joanna Batstone, Vice President and Lab Director of IBM Research Australia, underscored that IBM Watson’s technology is differentiated in its ability to make these subtle classifications in retinopathy severity with such high accuracy; other comparable deep learning platforms only have the capacity to identify the binary “yes/no” absence of presence of diabetic retinopathy.

In the company announcement, IBM positioned this cloud-delivered, software-as-a-service technology as a method of expanding access diabetic retinopathy screenings, particularly in low-income communities and in developing countries, though notably this is still a research project with no concrete commercial timeline as of yet. The company emphasized that a third of people living with diabetes have never had an eye exam and suggested that access to specialists is limited. Thus, IBM envisions this technology as a screening tool to help clinicians (including, presumably, primary care physicians) identify patients who would benefit from referral to specialists. This echoes comments from IBM Watson Health General Manager Ms. Deborah DiSanzo at JPM 2017, when she positioned Watson as “augmented intelligence” to improve physician workflow and boost productivity. While many have wondered whether Watson will eventually replace doctors altogether, IBM clearly sees Watson as a tool to make doctors more productive, rather than a physician replacement. Dr. Batstone explained that this retinal imaging technology “provides additional insight to the clinician,” but it is ultimately up to them to “incorporate this data into the patient’s broader story.” Indeed, as physicians are increasingly pressed for time with patients, we hope that tools like Watson can cut down on some of the more process-oriented tasks. Ideally, this would free up more time for the physician to spend speaking with and truly listening to their patients, strengthening the doctor-patient relationship. We’ll be interested to see how the positioning and deployment of Watson changes over time – even now, we might not recommend radiology to aspiring medical students looking to work on eye disease, as Watson is pretty darn good at image analysis.

For diabetes technology, this news shows yet another application of IBM Watson (image analysis), and continued commitment to diabetes from the tech giant.

• Google is also working on an automated deep learning algorithm to detect retinopathy and macular edema from retinal fundus photographs. The data, published in JAMA in December 2016, is impressive, with the algorithm demonstrating over 90% sensitivity and specificity. However, IBM told us Google’s algorithm focuses on the yes/no detection of retinopathy and macular edema, rather than the more granular classification of stages diabetic retinopathy that Watson performs. Well, then!
• Diabetic retinopathy continues IBM Watson’s work in diabetes, most notably through the high profile Medtronic-partnered Sugar.IQ app. According to Medtronic, Sugar.IQ is expected to launch this May-October. The app identifies hidden patterns based on CGM and pump data and logged meals – it has been quite delayed, and we wonder if it is waiting to launch with Guardian Connect (standalone mobile CGM). IBM Watson is also partnered with Novo Nordisk, though details on what the partnership might bring forth have been scarce since its announcement in December 2015. Same for the partnership with the ADA announced last year. Novo Nordisk and Glooko also signed a partnership in January, and while their initial efforts will not leverage IBM Watson per se, we believe all three companies will ultimately work together.
• IBM Watson’s diabetic retinopathy project could benefit mightily from the company’s expertise in medical image analysis for other disease states. Dr. Batstone explained that some fundamental deep learning techniques, such as border analysis or segmentation of key features, are transferrable from one domain to another – for example, image analysis for breast and skin cancer screening could be applied to diabetes and vice-versa. This is yet another strength of IBM Watson’s huge data vault (see its JPM 2017 presentation), big team, and massive algorithm/software expertise.  

-- by Helen Gao, Abigail Dove, Adam Brown, and Kelly Close