August 15-16, 2018; Reston, VA; Full Report - Draft

Greetings from Reston, Virginia! The JDRF/Helmsley Big Data Workshop was a mile-a-minute, featuring a number of insightful talks (we report on 13 below), three parallel break-out sessions on hot topics in diabetes, and a “Roundtable” of all 90+ attendees.

As Co-Chair Dr. Frank Doyle astutely noted, there’s rarely a meeting with this kind of diversity in skill and background, but it’s clear that this is exactly what it will take to begin to address the data generation, aggregation, standardization, and utilization demands of an autoimmune disease with complex pathogenesis and clinical management. Hope you enjoy the full report as much as we did the meeting!

Talk Highlights

1. Tidepool Patient Donated Data (n=3,750 Donors) and Clinical Study Platform Being Used by 11+ Partners for RT Compliance Monitoring, Virtual Trials, R&D, Trial Recruiting, etc.; HCT-Funded Loop Observational Trial to be Coordinated by Jaeb

Tidepool’s Mr. Brandon Arbiter shared a number of exciting updates from Tidepool’s Big Data Donation Project (up to 3,750 donors), including ongoing projects and the latest anonymized donor statistics (age, estimated A1c, hypoglycemia). At this point, there are at least 11 entities leveraging Tidepool’s aggregated data or Clinical Study Platform. Lilly (advance development of insulin delivery ecosystem), Dexcom (decision support for MDIs and pumpers), Jaeb Center (real-time compliance monitoring), T1D Exchange (virtual clinical studies), Evidation Health (recruiting from Tidepool’s user base), OHSU (hypoglycemia prediction), DreaMed (improve DreaMed Advisor algorithm), USF, Rice University, Dr. Anne Peters, and Stanford (Dr. David Maahs and team). Dr. Peters’ work is particularly interesting. She has a number of geriatric patients on CGM, and receives automatic daily alerts from Tidepool each day containing information about any hypoglycemia – she can then intervene as needed on a daily basis. Also of note, JDRF/Tidepool’s first Early Career Research Award went to a group of grad students under the supervision of Dr. Maahs. Aided by CGM tracings provided by Tidepool’s Big Data Donation Project, Dr. Maahs and his team noticed a common glycemic signature (see below), in which patients experience “pretty good” time-in-range abruptly followed by weeks of hyperglycemia “all the time.” Apparently, this happens all throughout Stanford’s pediatric clinic, the pattern was just never detected – Mr. Arbiter didn’t say exactly what was causing the trend, but we’re surely looking forward to finding out (our team spent a solid five minutes crowded around a screen last night and we don’t think we were able to get to the bottom of it – we guessed it might be endocrinologist visit cadence and sports/school/home schedule). More importantly, the grad students are now automating a mechanism to identify this particular glycemic signature and send alerts to clinics, allowing intervention on a weekly basis as opposed to waiting three months (at least!) until a patient’s next appointment. It’s great to see Tidepool step into the role of standardized data aggregator and disseminator.

• Tidepool’s database now includes an impressive 3,750 real-world data donors, one billion CGM data points, 1,250 complete, longitudinal datasets with insulin pump and CGM data, and 1,000 years of patient data. As Tidepool shared late last year, estimated A1c of Tidepool data donors is slightly lower at every age group than that recorded in the T1D Exchange, though it follows a similar pattern. However, Mr. Arbiter was surprised to see that those between the ages of 30-39 years-old spend the most time in hypoglycemia “by far” – on average, over one hour and 20 minutes every day <70 mg/dl.  Mr. Arbiter characterized this finding as truly “shocking,” given these are “fairly engaged users” (many may also be busy parents, taking care of older relatives, and have intensive work schedules). 

• Mr. Arbiter briefly mentioned the Helmsley-funded Loop Observational Study, first introduced at a Vatican Healthcare meeting by Mr. David Panzirer. Mr. Arbiter today noted that the study will be run by Jaeb, Tidepool, and Stanford Children’s Health, emphasizing that he is “really excited” to collect clinical evidence validating the open source system.

• Mr. Arbiter (who has type 1 diabetes) was extremely positive about his experiences with both DIY closed loop and adjunctive SGLT-2 therapy – and he had the data to back it up. Colleague Dr. Ed Nykaza built a graph depicting Brandon’s time-in-range going back to January 2016. As Mr. Arbiter noted, the graph of his data serves as a “pretty nifty decision support system,” as the clearly rising trend (of Time in Zone) allows him to “feel good about some of the decisions” he’s made. In other words: Starting to use a DIY Loop system. Citing Dr. Jeremy Pettus’ comment that he’s gotten an hour back in every day not dealing with diabetes tasks since starting Loop, Mr. Arbiter said that he simply changes his infusion site every day and knows his diabetes won’t be something to worry about. Another compelling story was that Mr. Arbiter had a mean glucose of 120 mg/dl two weeks ago while he was traveling in Europe “eating pasta and having a great time”; upon returning home, he had a mean glucose of 170 mg/dl. Why? He stopped taking the SGLT-2 inhibitor Jardiance! “I was on an SGLT-2 for a year. People said, ‘stop taking it,’ I’m not sure why, so I decided to stop taking it, and oh my god, my blood sugars just went crazy! Highs and lows like never before! I was living for 10 days like this and couldn’t get in to see my endocrinologist until late October, so two nights ago I started again.”

• Mr. Arbiter referenced solving meals as another important application of decision support. We were pleased to see Tidepool’s Nutshell app featured on Mr. Arbiter’s slide, albeit listed as “not yet available.” Tidepool last spoke publicly about Nutshell, its app intended to easily track past meals and improve mealtime dosing, at DTM 2016. Earlier that year at ATTD, Tidepool’s Mr. Howard Look mentioned that he hoped a launch would occur “within the next quarter” – obviously the timeline has been meaningfully delayed, but this is going to be a compelling platform to pair photo-logged meals with CGM data. Neither development status nor launch timing was shared today. There has been some progress on the diabetes meal front in the meantime, from Medtronic (iPro2 with Nutrino Foodprint; Sugar.IQ), Onduo meal-logging app… We especially think sorting meal photos into categories of Bright Spots (glucose stayed in range) and Landmines (glucose went out of range) would be compelling – simply seeing the meal photos gathered together in a pattern could drive behavior change, even if the software cannot recognize what’s actually in the meal.

• Mr. Arbiter shared several examples of “glanceable displays” that he has used or currently uses to support his own diabetes. He highlighted the ability to see his CGM data on his Apple Watch with the help of Night Scout as critical. With Dexcom’s Watch app, Mr. Arbiter explained, this display can finally “go to the masses.” Other glanceable displays featured in Mr. Arbiter’s presentation include a Hue lightbulb hacked to change colors in accordance with his glycemic excursions overnight (his wife didn’t find this helpful as she wanted actual blood glucose numbers) and a tablet displaying Mr. Arbiter’s blood glucose, as well as the weather, time, and bus schedule (“the four pieces of information a person with diabetes needs before leaving for work in the morning”). This is awesome, and a clear sign of the incredible combinations that interoperable systems can enable.

• Mr. Arbiter argued that innovations like Dr. Maahs’ (see CGM trace above) is only possible given open access to data – Apple Health, Dexcom’s APIs, and Tidepool were all necessary. The main detractors, he explained, are companies with proprietary data silos. Mr. Arbiter called for open access ecosystems, claiming that such silos may actually be breaking the law under the 21^st Century Cures Act. He applauded Dexcom for “paving the way” to interoperability with iCGM, and noted that next steps include the development of iPumps and iAlgorithms. Excitingly, Mr. Arbiter mentioned that in his own conversations with pump companies, “the entire landscape has changed their discussions – many companies are going in the direction of iPump.” We’re excited to hear this commitment in the field. At ADA, we learned that SOOIL plans to launch an open protocol, smartphone-controlled insulin pump in the US by ADA 2019. We’d have to imagine Tandem, Insulet, Lilly, and Beta Bionics are all thinking about it too, given Dexcom’s G6 iCGM approval.

2. JDRF-IBM T1D Big Data Project Has “Several” Manuscripts In Progress, One Year In; Preliminary Work in Predicting T1D Disease Onset (very cool)

Nearly a year after JDRF and IBM announced a partnership to explore type 1 diabetes risk and onset – and with “several” manuscripts in the works – IBM Research Manager of Health Analytics Dr. Kenney Ng provided a progress report. Preceding a very interesting Q&A featuring the likes of Drs. Des Schatz and Marian Rewers (see below), Dr. Ng shared the status of the collaboration’s four main aims: (i) data aggregation and curation; (ii) phenotyping type 1 diabetes classes; (iii) type 1 diabetes onset prediction; and (iv) type 1 diabetes progression modeling.

• Data aggregation and curation: The data set has swelled to 81,000+ total clinic visits, 22,000+ subjects, and 650+ cases of type 1 diabetes, and includes auto-antibodies, clinical, physical, genetic, family history, socio-demographic, and environmental data. As a reminder, the data comes from data sets from previous type 1 diabetes trials: DAISY, DiPiS, DEW-IT1/DEW-IT2, and the most recent addition, DIPP. Not surprisingly, Dr. Ng noted challenges in aggregating the data sets due to differences in place, time, patient population, method of classifying features, etc. – this was a main conversation point during Q&A.

• Phenotyping classes: Researchers have identified three clusters of individuals that have vastly different probabilities of developing type 1 diabetes. In the graph below, the blue line (very few auto-antibodies) has a low risk of developing type 1 diabetes (4% at 10 years); meanwhile the green and red groups, both of which have more auto-antibodies, have much greater risk of developing type 1 (59% and 83%, respectively, at 10 years). These clusters are highly associated with type 1 development, with very strong sensitivity of 90% and specificity of 85%. Western Michigan’s Dr. Craig Beam suggested that these measures of sensitivity/specificity are great, but wondered if they were clinically relevant (i.e., whether there should be a higher bar. How does this clustering affect clinical practice?). IBM-JDRF’s work is still early stage, but we can easily imagine how stratifying individuals by their probability of developing type 1 diabetes early on could impact clinical care and research.

• Researchers also sub-typed type 1 diabetes by auto-antibody transition patterns (below). The algorithms found, unsurprisingly, that the most significant predictor of type 1 diabetes was persistent presence of antibodies to insulin and GAD65. The second most significant predictor was persistence of antibodies to GAD65, protein tyrosine phosphatase, and Zinc Transporter Antibody 8…and so forth. When the model was trained on transition patterns in the DAISY data set and tested on the DiPiS data set, the ROC area under the curve was a very strong 86%. In the flipped scenario, when the model was trained on the DiPiS set and tested on DAISY, the ROC area under the curve was also a strong 81%.

• Onset prediction: IBM’s model (“RankSvx”) accurately predicted time from seroconversion (auto-antibodies appear in the blood) to type 1 diabetes onset and correctly ranked the relative risks - high importance of auto-antibodies and genetic risk, and low importance of factors like age and gender.

• Progression modeling: Work is ongoing here, so there are no results to share. However, the goal is to answer population-level questions (e.g., “What are the underlying progression states for the disease? What are the most/least likely progression pathways?) and patient-level questions (e.g., What is the current state for a patient? What is the patient’s most likely progression pathway going forward?).

Selected Questions and Answers

Dr. Desmond Schatz (University of Florida): Regarding data aggregation and analysis challenges – different formats, different bits of data – do you actually have the raw, longitudinal data just to make sure that it’s accurate and valid? Hopefully you can validate in TEDDY and other sets. I struggle with the data accuracy. How confident you are in the models you’re producing?

Dr. Ng: There are definitely a lot of challenges as you can imagine, and we’ve been working closely with the clinical experts from the sites, multiple calls per month to try to resolve those issues. We’re relying on a collaborative effort to validate that the data we get is the data the clinicians expected and that the analyses are consistent with their understanding. We try to make sure that data is faithfully received and analyzed by us. We also have a collaborative workshop where we share initial results to make sure interpretation and use of data is consistent.

Dr. Schatz: But that’s not the raw data?

Dr. Marian Rewers (Barbara Davis Center): Excellent question. Our study (DAISY) provided the raw data at the highest level of resolution we could provide. But there are issues. Starting with HLA classification, each study had different classification. Three-quarters used DR, whereas one used DQ as the primary classifier. It took us three months to figure this out and agree on categories depending on application. For auto-antibodies – anyone who participated in antibody standardization can see how difficult it is – but we agreed for the time being to use “positive” and “negative.” I have to say that for all of the studies it was a very nice experience. It’s moving very well, we’ve outlined five manuscripts so far.

Prof. Chantal Mathieu (KU Leuven): I was thinking the same, with the definition of diabetes, when you try to predict the time – is the definition of diabetes the same? Are some studies using OGTT, random glycemia? Is that also taken care of?

Dr. Ng: I think onset was provided by each of the sites. I didn’t share this work, but we did actually look at defining the development of antibodies with different definitions, performed an extensive sensitivity analysis, with different definitions of similar events along the progression of type 1 diabetes. Does it really matter? At some level it’s too fine-grained, but could be interesting to know. There are a lot of interesting questions in this space.

Dr. Craig Beam (Western Michigan University): A number of different approaches will be taken to deal with big data. I’m wondering if it’s not essential for us, early in the development stage, to come up with a standard way of assessing success and clinical relevance. Sensitivity and specificity have to be clinically relevant at some point – is there a clinician- and health outcomes-based way to derive standards for proof?

Dr. Ng: You’re totally right in terms of the early research vs. clinical deployment phases. There are a lot of challenges there.

Q: Will you need additional data types to get the answers you’re looking for?

Dr. Ng: That’s definitely a yes. This is a baseline. We’re starting with common data features from the data sets. Some of the work we’re doing now is to enlarge the features space to include as much information as we have available – the challenge is they don’t all have all of the features.

Q: You don’t feel that the data you have now will be limiting with regards to preliminary answers?

Dr. Ng: I think there are insights we can still extract from the existing data. There’s enough there to lead to some more interesting work.

3. Verily’s Dr. Howard Zisser: AI helps us stop “treating to the mean”; Busting 3 AI Myths; Google’s Healthcare AI Publications

Verily’s Dr. Howard Zisser showed off his AI expertise – a new hat after years driving AID forward – in a 30-minute talk, dispelling three myths in AI, highlighting two recent Google publications, and reviewing Verily’s work with automated retinopathy detection (no updates on this front). He also shared some comical examples of where AI is not perfect: (i) A dinosaur picture with a scale below that a computer labeled “a dinosaur on top of a surfboard”; and (ii) a Google Translate translation of a Spanish sign to “Recent attack of shark” – was the shark attacked, or did it attack a human? But at the end of the day, he said, using AI in healthcare helps us to stop “treating to the mean” – it allows us to look at a big population, stratify based on risk, and then to derive insights at the level of the individual. “What can I ask that will help me take care of patients better? What can help me to look at populations? Who should be getting what and when? But also when talking to individual, to make predictions based on big data sets…That’s what patients want, it’s more precise, and it’s better for populations as well.”

• Dr. Zisser explained why three myths about AI are in fact not true: 

  • Myth #1: “Artificial Intelligence = Machine Learning.” Dr. Zisser has previously dispelled this idea – machine learning is in fact a subset of artificial intelligence, and deep learning is a subset of machine learning. The two are often used interchangeably, but are not in fact the same thing.

  • Myth #2: “AI is a monolith.” AI is often a top layer built on a heuristic infrastructure. In other words, AI is part of the system, not the whole system. For example, with an algorithm that identifies retinopathy and tells the provider whether to refer the patient to an ophthalmologist, the system includes the payer, the patient, the doctor, to name a few – not just the algorithm. Workflow matters, which is why Verily was undertaking workflow-based studies for AI retinopathy screening in India (as of November 2017).

  • Myth #3: “AI doesn’t need a human.” Noted Dr Zisser, “It needs supervision in design, training, evaluation, iterations, and often when addressing errors. Basically at every single step. All models are wrong, but some are useful.  [The latter is a famous quote from statistician George Box]”

• Dr. Zisser referenced two recent Google papers in the Nature family, including one published just two days ago (which attracted a fair amount of media attention)! In March, Poplin et al. (including Verily Head of CV Innovations Dr. Michael McConnell) showed that an algorithm could predict age, gender, smoker status, A1c, BMI, and blood pressure with high accuracy just from retinal fundus photographs. The DeepMind paper published on Monday – picked up by outlets such as STAT, Business Insider, and The Inquirer – showed that an algorithm could accurately detect 50 types of eye disease just by looking at optical coherence tomography scans. Taken together, the larger Google healthcare family seems to be driving toward harvesting as much health information as possible from retinae – we’re envisioning a single image diagnostic panel that could, in real-time, inform next steps with respect to all sorts of eye disease and CV risk. This is also makes us harken back to the no-longer-updated Verily/Novartis glucose-sensing contact lens – is it still in development?

• Apart from the vision to have automated retinopathy screening at PCP offices and in pharmacies and to “have a diagnosis by the time you end a screen,” there were no updates on Verily’s work in this arena. This part of the talk was basically identical to that Dr. Zisser delivered at DTM 2017 – he discussed shortages of eye specialists (particularly in India), a 2016 JAMA paper speaking to Verily’s algorithm’s strong sensitivity/specificity, and a general overview of the algorithm and its features.

4. Helmsley/Cyft Collaboration Update: Algorithms Can Predict Who’s Likely to Have 1.0% A1c Increase (Telehealth Intervention Upcoming), Who’s Likely to Cancel Appointment Late ($250k Savings/Year?)

Joslin’s Dr. Sanjeev Mehta provided an update on the clinical (Children’s Mercy Kansas City) and operational (Joslin) tracks of the Helmsley Charitable Trust/Cyft collaboration to proactively identify, stratify, and manage at-risk type 1s. This partnership was originally announced a year ago, and we’re glad to hear of initial progress. (i) Cyft and Children’s Mercy have developed models that can predict which individuals will have a clinically-significant rise in A1c (+1.0%) at their next visit with strong accuracy (next step: telehealth intervention to see if the prediction yields fruits); (ii) Cyft models can predict with high accuracy individuals likely to cancel appointments within 72 hours – 50 phone calls per week to these “high-risk” individuals is projected to save Joslin $250,000 per year. Overall, Dr. Mehta was very enthusiastic about the collaboration, suggesting that in his tenure as CMIO for Joslin, this is “the most comprehensive exchange of clinical administrative data we’ve ever done – if there was anything we could give, we did it. We gave everything. Hundreds if not thousands of data points on each patient.” He also said that he is not thinking about the three-year Helmsley grant period, but really thinking about how to implement changes at Joslin for the long term.

• Cyft and Children’s Mercy tested a number of EHR- and CGM-based models to predict the likelihood of a significant rise in an individual’s A1c at the next visit, arriving at a number that have a “hit rate” of ≥50%. In other words, ≤two patients would have to be targeted – in order to accurately predict a clinically significant rise. Wow! That compares to the null model, which would require targeting eight individuals to catch one who would experience a significant increase in A1c. We were elated to hear that the next phase will be a telehealth intervention with the aim of reducing or preventing the A1c increase prior to the next visit. We assume the model will suggest that the Mercy Children’s team reach out to a number of patients – theoretically ~twice the number who are actually expected to see A1cs rise – in attempts to address drivers of poor diabetes management. This data will be very informative for Helmsley and the broader type 1 diabetes care field – it could really redefine care models and help prioritize resources more effectively.

• Joslin’s 50 call per week operational plan to recover lost revenue due to late cancellations is not the clinic’s first initiative in line – it’s behind modifying the EHR and appointment-making processes – but the models will be validated in Q1 or Q2 of 2019. Cyft’s models proactively identify patients most likely to cancel, so they can reach out well prior to appointments and give patients a reminder/opportunity to cancel earlier if necessary. According to Dr. Mehta, Cyft’s machine learning algorithm identifies late cancellations with ~60% accuracy – 20% better than simple regression and 40% better than random prediction. Though the main argument here is to reduce waste in the system and boost revenue, this could also result in improved patient care by way of maximized provider time and reduced frustration.

5. BDC’s Dr. Marian Rewers Calls for Increased Clinical Utility and Interoperability of Diabetes Technology, Applauds Glooko and Tidepool

Barbara Davis Center’s Dr. Marian Rewers brought a much-needed clinical perspective to the day’s big data discussion, urging industry players, particularly those in CGM, to focus on clinical utility and interoperability. He expressed frustration with the lack of interoperability between different manufacturers’ software, which he says complicates efforts to identify overall clinic trends and patterns in CGM data (for purposes of quality improvement). Restricting CGM data to device-specific interfaces, he explained, limits the value of data to clinicians and patients. To this end, he pointed to Tidepool and Glooko as “extremely important” (management from both teams were in the room). However, software interoperability is but one component of the big data bottleneck for clinicians – EMR compilation, registry integration, and business information also contribute to the data profile of each individual patient. Together, these factors create a complicated, inefficient system, which Dr. Rewers hopes can be streamlined to allow for patient care.

• As Dr. Rewers astutely explained, “data reduction” is fundamentally in opposition to “big data” efforts, yet it remains integral to the improvement of diabetes technology. He asserted that the software interfaces of CGMs are often thorough to a fault, including data that are of no use to clinicians or patients; alternatively, he would prefer for dashboards to be reduced to simple, easily understood figures that highlight trends. He suggested that Dexcom has made headway in this regard, showing a Clarity display on a slide following one from a previous generation – indeed, we find Clarity’s main display to be very intuitive and a strong step forward from Dexcom Studio. Offering an alternative account, Dr. Chantal Mathieu suggested following the talk that Dr. Rewers’ opinion on CGM dashboards is not necessarily generalizable to all patients and clinicians. In particular, she finds that many of her younger patients find value in the charts and graphs provided in CGM dashboards, which makes us wonder if customizable software (e.g. the option to select a “simple,” “classic,” or “detailed” dashboard) may be useful. Of course, adding a bunch of customizability also adds complexity, so this is a tough balancing act. The standard one-page AGP (Ambulatory Glucose Profile) – now licensed by all major CGM companies – has made increasing strides in CGM standardization as a simple, base view for all CGM display, while each manufacturer also offers other displays of varying granularity and content. However, we’d note that AGP does not do pattern recognition – e.g., “high pattern between 3-8 am” – which we believe is the most clinically actionable piece of CGM data. (When is someone going high or low, why is that occurring, and what can be done about it?) While the AGP does display the modal day chart and someone can easily see where highs or lows are occurring, having a prioritized list of patterns by time of day would be quite useful to many. (Dexcom Clarity and Medtronic CareLink both do this. We haven’t seen Abbott’s LibreView software in quite a while, but this FAQ suggests it also gives similar patterns.)

• Dr. Rewers provocatively (and “tweet-ably”) said, “I really would like artificial pancreas and artificial intelligence to replace me and my colleagues.” In his experience, the current healthcare system in which patients pay up to $800 for a one-hour outpatient visit is both outlandish and inaccessible, especially when time constraints allow for only “limited insights.” While we agree that the cost of care needs to come down, we expect that most endocrinologists and primary care providers would jump at the opportunity for a one-hour session – at one of Dr. Irl Hirsch’s  AACE 2018 talks, the majority of attendees indicated that they are allotted 15 minutes or less with their patients. Dr. Rewers hopes that AI-based technology can reduce burden on the healthcare system while conferring better outcomes for patients. TO that, we’d add better use of the system’s resources – spend in-person time on the complicated cases, and use more remote monitoring and telemedicine for simpler cases. In the mid/long-term, we think this is inevitable, given the trends in endocrinologists and PCPs vs. people with diabetes. However, it is important to note that hybrid closed loop and AI is still in the infancy stage, so training and implementation will likely add burden before they decrease it.

• According to Dr. Rewers, unless big data can be used to reduce burden (including the high cost of healthcare), it remains an academician’s pipedream. He cited a 2014 article from the New York Times, in which one woman said she paid ~$4,000 in out-of-pocket costs (~$26,000 without insurance) to manage her type 1 diabetes for just one year. Dr. Rewers said this number would be 30% higher today. In his view, big data has an opportunity to reduce patient burden on the healthcare system, the savings from which must ultimately be passed down to patients; we wholeheartedly support this sentiment, and believed the companies that best collect and use data to cut overall costs will certainly capture a lot of value in the system. While he noted AID as one example, this is not yet a classic application of big data, as most of the decision-making is n=1 based and fairly moment to moment. (over time, however, systems should improve to higher-level insights informed by populations – “patients like you.”). Decision support tools, hypoglycemia risk predictors, and other prevention strategies also come to mind as exciting options.

6. DreaMed Advisor Pro “Definitely” Going to Go Direct to Patients When Shown Safe with HCPs

Schneider Children’s Prof. Moshe Phillip said that the recently-FDA cleared DreaMed Advisor Pro clinical decision support software for optimizing insulin pump settings based on CGM data would “definitely” go direct to patients “when we show safety with physicians.” As a reminder, Advisor Pro recommends pump setting changes for physicians, who approve them before they are sent to a patient’s Glooko app. Given that the system was both CE marked and FDA-cleared this year and has been slated to launch with partner Glooko at the end of summer 2018 in both geographies, we imagine that a study shuttling pump setting updates and general insights directly to the patient could begin in the very near future. Prof. Phillip’s exciting comments raised a number of questions in our minds about “Advisor” (we assume DreaMed will simply cut “Pro” from the title for the patient-facing version): (i) How will the insights be passed to the patient (also through the Glooko app? A different app? Text message?); (ii) Will advice be pushed to patients, pulled by patients, or a mix of the two?; (iii) Will “Advisor” go through the same “Insulin Therapy Adjustment Device” FDA pathway as “Advisor Pro”?; and (iv) What could the business model be for “Advisor” (Out-of-pocket payment? Health plans/employers pay for beneficiaries on per-member per-month basis? Shared savings?). Regardless we’re glad to hear these insights may eventually not require HCP oversight on an ongoing basis – like basal titration apps, once they are pre-configured by an HCP, they can run on their own. We first heard Prof. Phillip allude to a patient-facing Advisor at AACE 2017. Today, he alluded to the multi-center Helmsley Charitable Trust-sponsored time-in-range trial of Advisor Pro (n=112), which as of June had enrolled 73 participants, and results from the first three months of the study were to hopefully read out by the end of 2018 or into 1Q19. In other positive news, pilot work on the Advisor Pro – presented at ATTD 2017 – was formally published in the June edition of Diabetes, Obesity and Metabolism (Nimri et al.).

• Prof. Phillip also reviewed DreaMed’s work in automated insulin delivery, most notably the NIH-funded FLAIR study, which is set to begin in 4Q18. As a reminder, the trial is comparing MiniMed 670G to Medtronic/DreaMed’s “Advanced hybrid closed loop,” which adds automatic correction boluses, including the DreaMed Diabetes algorithm. At Keystone, Medtronic’s Dr. Fran Kaufman shared that the goal of this next-gen system is to deliver >80% time-in-range, 100% time in closed loop, lower glucose targets, and a better meal experience.

• Prof. Phillip’s definition of big data 10 years ago: “Too much data for me to handle alone.” To handle an influx of data from clinical trials and practice, he took two main steps: (i) He and colleagues created the ATTD meeting; and (ii) he hired an engineer to work in his pediatric endocrinology clinic. “I felt it was too much data to digest, handle, interpret, and we were missing an opportunity to use it.”

Questions and Answers

Dr. Craig Beam (Western Michigan): Why do you strive for the Advisor to be as good as human clinicians? Why don’t you try to be better? And have you considered giving the experts the same case back again to see how often they disagree with themselves? Isn’t that the advantage of the machine?

Prof. Phillip: Second question first: No, we didn’t try to do it. It’s a great idea, we just didn’t think of it. First question: Why do we aim for non-inferiority … all those experts are my friends! Just kidding, if it turns out to be superior, no one would blame us.

Mr. Brandon Arbiter (Tidepool): I’m a huge fan of what you’re doing at DreaMed. What are the hurdles of getting this software directly to patients, and what are the risks of doing so?

Prof. Phillip: Regulatory hurdles. We had a strategy to approach FDA with the Advisor for healthcare providers first, and when we show safety with physicians, we will definitely go direct to patients.

7. Helmsley-Funded T1D Trial Progress: 34 Participants Enrolled in POInT, T1-DEXI Data Set for 2021, Handle-I and Handle-P Trials

Helmsley’s Dr. Gina Agiostratidou provided updates on three of the organization’s funded type 1 endeavors: GPPAD’s Primary Oral Insulin Trial (POInT), T1-DEXI, and Handle-I. Dr. Agiostratidou emphasized that data from each of these trials will be shared openly with the public as part of Helmsley’s recently launched data-sharing initiative.

• The POInT Trial. Earlier this month, we heard that enrollment for GPPAD’s primary prevention trial for oral insulin had just begun, with the aim to screen >300,000 pregnant mothers and newborns across the five GPPAD countries (Germany, UK, Belgium, Poland, and Sweden), to eventually enroll 1,040 genetically at-risk babies. According to Dr. Agiostratidou, ~30,000 mothers and babies have already been screened, and 34 at-risk infants have been enrolled in the trial. Several huge updates on the trial’s timeline were also given. Dr. Agiostratidou shared that genetic data on these first 34 participants will be available as soon as December 2018 or January 2019, and data from the entire cohort will be available in 2025 (see below), in line with the scheduled completion date of January 2025. This is certainly a long game, and we’re glad to see the incredible investment here.

• T1-DEXI. Most importantly from this update, we learned that Helmsley’s trial (n=800-1,000) investigating the effect of exercise on glucose control in type 1s – described yesterday – is set to start in early 2019, with data expected to be available in 2021 (see below).

• Handle-I. This trial is new to us. According to Dr. Agiostratidou, Handle-I is designed to help researchers understand the development of the non-type 1 immune system in the hopes of garnering insights into the pathogenesis of type 1. No details were given on the design, participant population, or outcomes of this trial. However, Dr. Agiostratidou did note that a similar program focusing specifically on pancreatic development, “Handle-P,” is also in the works.

8. Update on HCT/Jaeb T1-DEXI Exercise Study: Hope to Launch Observational Study in “Early 2019” with N=800-1,000 (Ages 14-70)

Helmsley Charitable Trust’s Ms. Deniz Dalton reiterated plans for a Jaeb-coordinated, multi-site, n=800-1,000 (ages 14-70) clinical observational study in type 1 diabetes to gather data on exercise (T1-DEXI). The hope is to commence the study in “early 2019.” This study is expected to have an initial four-week observational period, followed by a break and another four-week period (total data collection period of eight weeks). Main outcomes monitored in the study will be insulin delivery, exercise, and CGM (for participants that do not use CGM at baseline, a CGM will be provided). All participants will also receive exercise videos “to be able to characterize certain exercise types” and a T1-DEXI study app that collects information about activity and food intake. Aligning with Helmsley’s ever-strengthening focus on data quantity, quality, access, and interoperability, the goal of this project is to broadly share data with the community, which will then allow researchers to test existing hypotheses and create new ones. Put simply: “A lot of data will be coming out of this project.” We’re delighted to see a growing focus on data aggregation around exercise in type 1 diabetes. Notably, the expected “n” of 800-1,000 shared today by Ms. Dalton is significantly larger than that of “300-500” previously shared at ADA. Presumably this could lead to some meaningful decision support around exercise, which remains challenging in type 1 diabetes.

• The T1-DEXI pilot study, described by OHSU’s Dr. Jessica Castle at ADA, is currently underway. This pilot study will enroll 60 individuals between ages 15-70 with type 1 diabetes, and collect one month of insulin, CGM, food, and physical activity data. Data will be collected with Dexcom G5, DiabNext’s Clipsulin dose capture device (an unconventional choice), a Garmin activity tracker, and a custom app developed at OHSU for food photos and exercise logging. Participants were randomized to complete two in-clinic and four home sessions of either aerobic, anaerobic, or high intensity interval exercise. Every seven days, providers review CGM and insulin data, and make insulin dose recommendations. This pilot will eventually inform the larger T1-DEXI study.

• Helmsley also recently launched a data-sharing initiative – going forward, whenever it funds a clinical trial, it will embed a data-sharing policy in the agreement. Ms. Dalton also alluded to the organization’s investment in CDISC to develop type 1 diabetes-related data standards, something we’ll hear more about tomorrow. Hats off to the T1D team at Helmsley for really driving forward the data ecosystem in type 1!

• We noted more than 15 mentions of the word “data” in this very short (~5-minute) talk – this certainly seems to be a bigger and bigger Trust priority. We agree this is a place where it can have real leverage and drive the ecosystem in a coordinated direction.

9. CDISC’s Data Standards for Type 1 Diabetes Expected to Complete in 2020, With Public Comment Period Prior; Topics to Include DKA, Insulin, Pumps, and CGM

The Clinical Data Interchange Standards Consortium’s (CDISC) Dr. Diane Wold provided updates on the Helmsley-funded initiative to create data standards specific for type 1 diabetes. CDISC received a 2.5-year, $1 million grant from the Helmsley Charitable Trust in April to develop type 1 diabetes data standards aiming to enable data sharing, cross-study comparisons, and meta-analyses with the ultimate goal of accelerating efficient type 1 diabetes research and discovery. Since identifying four key areas requiring data standardization (pediatrics, devices, prevention, and exercise), Dr. Wold shared that CDISC expects to work on data surrounding DKA, insulin, pumps, and CGM, with more topics “coming.” CDISC has submitted these focus areas to Helmsley for consideration, after which CDISC will continue development, progressing through rounds of internal and public review. Dr. Wold emphasized that the public review stage allows “anybody” to comment, as CDISC is determined to have “as many eyes on the standards” as possible. Given the rigor with which these standards are created, Dr. Wold cautioned that she doesn’t expect the project to be complete until 2020. CDISC standards are required for pre-clinical and clinical research for all new electronic drug submissions in the US and Japan, so we appreciate the care taken here and look forward to seeing how these standards drive greater interoperability and innovation.

• Dr. Wold mentioned that certain data standards already exist in diabetes, dating back to 2014. Standards surround diabetes lab results and hypoglycemia events (established in 2014) as well as renal function and kidney failure (established in 2016). We haven’t seen these and are trying to track them down.

10. Genetic Risk Score for Type 1 Diabetes (Based on HLA Allele Copy) is Useful for Research but May Not Be Ready for Population Screening

University of Exeter’s Dr. Richard Oram emphasized that use of a continuous genetic risk score (GRS) for type 1 diabetes poses a strong advantage for investigators, but may not be suitable at an individual level. The GRS is calculated using the number of HLA allele copies present – HLA is a common genetic variant associated with type 1 diabetes. The GRS alone was shown to be “strongly discriminatory” for the diagnosis of type 1 diabetes and type 2 diabetes. Moreover, Dr. Oram found that when GRS is combined with age of diagnosis, BMI, and presence of autoantibodies into a single continuous variable, the discrimination is “close to perfect.” He sees the GRS being particularly effective in mitigating the costs of early intervention trials, as it is critical to restrict them only to very high-risk populations. As an example, he determined that identifying those at 10% vs. 5% risk of multiple autoantibodies could halve the cost of an early intervention trial. On the other hand, if a population screening approach is taken, he suggested using GRS to identify the 10% of the population at greatest risk of developing type 1 diabetes, as it contains 78% of cases.

• Dr. Oram detailed ongoing work leveraging combined modeling to improve type 1 diabetes prediction in those with first degree relatives with type 1 diabetes. At part of the TrialNet Pathway to Prevention program, ~7,000 individuals with first degree relatives with type 1 diabetes are being followed, ~1,000 of which have already been genotyped. After stratifying by single or multiple autoantibodies, the results indicated that GRS can identify type 1 diabetes risk more effectively than the number of autoantibodies. In fact, Dr. Oram noted that some individuals with a single autoantibody had the same type 1 diabetes risk as those with multiple autoantibodies (we would hazard that these single-autoantibody individuals were typically on their way to developing multiple).

• Audience members were very interested to hear Dr. Oram’s thoughts on whether the GRS could be effectively implemented for population screening. As one attendee pointed out, genetic variants will differ based on ethnicity, and the GRS in question was generated using Caucasian genomes. Furthermore, a specific risk threshold would have to be determined in identifying when to actually approach parents with the result of a “high genetic risk score.” Dr. Oram asserted that the GRS is not yet suited for population screening, maintaining that “at the moment it’s a research tool.” He acknowledged that there is a need to test the GRS in non-Caucasian populations, although he’s been surprised by the solid prediction accuracy demonstrated in initial analyses investigating different ethnicities. Still, he noted a tradeoff – is it better to make 50 different GRSs for unique situations or one broader, simplistic metric? The former is obviously more sensitive, but it could be less translatable. Dr. Oram also described the complexities surrounding counseling, explaining that if disease information is provided, so, too must appropriate information – and this is further complicated in cases of polygenic risk, where progression to disease is even more uncertain, than in diseases concerning one gene. Determining “the right amount of risk” at which to intervene is also difficult as it’s currently unknown what kind of impact clinicians might have for high-risk patients. Interventions to delay onset of type 1 diabetes, such as oral insulin, require more work to determine reproducibility and value.

11. The Subjectivity of Diabetes Prevention: Why Algorithmic Prediction Requires Human Insight, and the Connection to WWII Fighter Planes

In a highly technical talk, University of Washington’s Dr. Shuai Huang underscored the potential and pitfalls of rule-based algorithms for type 1 diabetes prediction. Our understanding is that Dr. Huang’s algorithms predict outcomes (such as type 1 diabetes development) based on a set of rules (linear combinations of risk factors) derived from biomarker data. In this way, data patterns and insights are built into the machine learning processes used for prediction at a fundamental level, forcing algorithms to be adaptive (e.g., risk is different for patients with HLA-DR3 vs. HLA-DR4) and conditional (e.g. HLA risk changes depending on whether autoimmunity is present). The rules that inform these algorithms must be formulated in a very cautious manner, incorporating human insights. Dr. Huang provided a useful parable, hearkening back to a statistician named Abraham Wald, who was tasked with placing armor on WWII military planes. He couldn’t coat the entire aircraft with armor due to (i) resource scarcity and (ii) weight. He parked himself on the military base’s runway and observed the location of bullet holes in the plane. At first glance, one would think to reinforce the spots that attracted the most fire – as did most people in the audience. However, Dr. Huang argued, this rationale is not necessarily right in this case: One should instead reinforce the parts of planes that do not have any bullet holes, as planes that did receive fire in these areas never made it back to the base (i.e., they were shot down). Dr. Huang drew a connection to algorithmic prediction: In order to provide the greatest benefit, machine learning must be married to data insights, which are often not intuitive and require humans to contextualize.

12. Generating Personalized Disease Risk Profiles from Health Records; Algorithm Predicts Likelihood and Rate of Developing T2D Complications (Under Review)

Notre Dame’s Dr. Nitesh Chawla detailed his team’s work leveraging patients’ medical histories to predict their individual risk of developing specific diseases. The results of this effort, called CARE (Collaborative Assessment and Recommendation Engine) were published in 2013 in the Journal of General Internal Medicine – an impressive feat, especially given the absence of MDs in the byline (though perhaps a broader reflection on the entry of “non-traditional players” into healthcare). CARE functions via a Netflix- or YouTube-like, “collaborative filtering method,” which identifies patient similarities and generates personalized disease risk profiles for individuals. Importantly, these similarities stretch far beyond traditional factors. As Dr. Chawla noted, healthcare (i.e., access to care, quality of care) is responsible for only ~20% of an individual’s overall health and wellness state. To account for the myriad other social health determinants, the algorithm sorts through shared diseases, symptoms, family histories, lab results, urban/rural residencies, occupation, demographics, and more. While a disease risk profile is certainly a step in the right direction, Dr. Chawla acknowledged that if such a solution were to ever actually be implemented in the clinic, meaningful contextualization would be required. To this end, Dr. Chawla and his team provided physicians with a list of the top 20 diseases for which a given individual is likely at risk. Impressively, follow-up visits confirmed that the CARE model was able to capture ~51% of future diseases in the top 20 list. Dr. Chawla admitted that 51% may not seem like a strong proportion, but as he pointed out, if even a small portion of these diseases can be addressed in a timely fashion, savings in cost and life could prove very effective. We do wonder to what degree (if any) these savings might be offset by any preventative measures taken regarding diseases that were inappropriately flagged.

• Dr. Chawla just wrapped up work using CARE to predict the likelihood of developing complications (e.g., myocardial infarction, heart failure, kidney disease, liver disease, retinopathy, stroke) for individuals diagnosed with type 2 diabetes ­­– results are currently under review for publication. Critically, this algorithm will not only describe whether a given complication is likely to occur, but also the speed at which progression might take place.

• To enhance the CARE model, Dr. Chawla is working to integrate medication data. For example, an individual moving to a different state may not establish a new health record, but it is likely that the individual’s prescription will be transferred and thus recorded. To this end, Dr. Chawla found that when health record and medication data were combined, CARE accurately captured 66% of diseases (up from 62% with health record alone).

• Dr. Chawla emphasized the ability to translate a research paper into a useful tool as the “biggest challenge.” He underscored the importance of social scientists and design thinkers, cautioning that “silos will fail us.”

• This approach to predicting disease is similar to that employed by GNS Healthcare, Cardinal Analytics, and Base Health – read about them all here.

13. Dr. Søren Brunak on the Potential for Big Data in Primary Prevention; Modeling Connections Between Diseases to Predict Risks and (When Diseases Will Arise)

Opening the meeting’s official agenda, University of Copenhagen’s Dr. Søren Brunak highlighted the utility of big data in understanding relationships between diseases and predicting diagnoses. According to Dr. Brunak, current systems of acute diagnosis are inefficient and reactive whereas predictive models can help raise awareness of associations (genetic, epidemiological, etc.) between diseases, alerting patients and clinicians to risks before diagnosis. One such model – the focus of Dr. Brunak’s work – connects diseases in stratified trajectories, demonstrating how each is connected so that clinicians and patients can take proactive steps to prevent further diagnoses if possible. Moving forward, Dr. Brunak said that data from the healthy domain (e.g., A1c trends before diabetes diagnosis) would help to complete the diagnostic picture, improving models and providing insights for primary prevention.

• Dr. Brunak suggested that advances in big data could make primary prevention possible, stating that his group is even beginning to understand when patients will be diagnosed with diseases based on trajectories. We wonder how understanding an individual’s time to diagnosis might be valuable in diabetes – for type 2 diabetes, could the timescale inform the intensity and/or selection of pharmacotherapy? Dr. Desmond Schatz expressed his belief that primary prevention is an attractive prospect for type 1 diabetes last month at Friends For Life, and GPPAD’s currently-recruiting POInT trial is the first primary prevention trial for oral insulin in the type 1 diabetes population. Regarding diabetes-related CV complications, Lilly’s highly-anticipated REWIND CVOT (for GLP-1 agonist dulaglutide) is the first such study to include a large (69%) primary prevention cohort, which will provide excellent insight into the CV primary prevention capabilities of the once-weekly GLP-1 agonist.

• Dr. Brunak underscored the need for standardized, longitudinal data to construct accurate disease trajectories, but noted that such a database is rare with clinical big data still in its infancy. However, he  touted data from his home country of Denmark as the next best thing, explaining that the personal identification number (PIN), 20-30 years of digital EMR data, and single-payer system of Denmark create the perfect (good) storm for comprehensive, linkable data. Together, the standardized medical and socioeconomic information associated with each PIN allow for a holistic analysis of associations between diseases, including the idiosyncrasies attributed to social health determinants like race, income, and education – factors which we know to be significantly associated with diabetes and obesity.

Full-Group Roundtable

Data Sharing by Researchers, Patients Stressed; In-the-Weeds on Data Format, Location; Consensus Meeting on TIR at ATTD?

In a one-hour general roundtable – an ambitious but successful experiment! – all ~90 attendees went back and forth on the biggest gaps in big data, with major emerging themes of “data sharing” and “data aggregation.” This was perhaps best captured by Dr. Desmond Schatz: “Right now we’re in left field, you’re in right field, and we don’t have any consistency, and I’m not sure we can overlap the data with any confidence. It’s an incredible opportunity for us and organizations to make a statement to urge each and every member of this group here to share data and create a consistent format.” Data sharing was, indeed, the source of widest agreement…in that it should be done by researchers, and it would be great if it were done by patients. NIDDK’s Dr. Guillermo Arreaza-Rubin noted that NIH already requires data sharing by grantees – “it’s one of our priorities” – though “maybe could put more emphasis on it” in type 1 diabetes. Similarly, JDRF and the Helmsley Charitable Trust have been working closely to move to a platform that ensures there is a plan in place, as well as a requirement, to share data from funded trials before the study even begins! This platform, “Vivli”, actually launched a mere three weeks ago, but already includes 2,500 data sets across diseases, mostly from industry (the hope is to make a type 1 diabetes vertical eventually). From the patient side, Tidepool’s Mr. Brandon Arbiter emphasized that now is the time to enable seamless data sharing – “all of these device-makers are now creating new data platforms and APIs, if there was ever a time to put a little checkbox on every API to say ‘Donate data to JDRF; and have that agreement in place with every company, that is now. It’s zero cost, minimal effort, and would give JDRF access to all of the observational type 1 diabetes data in the world.” According to KU Leuven’s Dr. Chantal Mathieu, it would not be difficult to get patients to tick the box because they are “so angry with companies.” We truly hope to see this initiative pushed forward – in fact, perhaps the default could be an “opt-out” system where a checked box is the norm, like it is for app developers who collect diagnostic data. Issues around privacy, consent, and GDPR were also brought up, but were generally regarded as addressable if we get started now.

• An interesting discussion ensued on location and format of big data, but Dr. Schatz requested everyone take a large step back: “I do want to emphasize the challenges at the interface of research, data aggregation, and clinical care. We’ve all got to agree, we need a standardized format, and we need to validate it. We can’t just say ‘this is what it is.’ We have our own custom Epic (at University of Florida), and I don’t know how shareable those data are. And once we’ve set up a format and validated it, then I challenge all of you on the analytical side, you develop techniques, then we validate those as well.” BDC (Barbara Davis Center)’s Dr. Marian Rewers believes that Epic gives the best chance in the US collect big clinical downloads, including diabetes-related downloads, since it is so pervasive. However, in order for that to happen, he believes “we need to get organized.” He shared that BDC spends ~$100 million per year (!) on customization of Epic, an expense many other centers surely incur. Dr. Rewers suggested that all of these clinics band together and push Epic to create a “robust and acceptable module for diabetes as part of its basic package,” which would give it a competitive advantage and simultaneously allow the field to collect large amounts of data and develop standards for outcomes and quality improvement. This point was well taken, though University of Copenhagen’s Søren Brunak balked at the idea of selecting Epic before the data scope and mission have been defined – perhaps Epic is the right medium to store and share data, but perhaps it is not (as a side note, he called attention to Epic’s obsolete programming language and some of his colleague’s less-than-ideal experiences with it). Children’s Mercy’s Dr. Mark Clements pushed back against the idea of working with EHR vendors themselves, arguing instead that we need to standardize clinical care data across all EHRs, to enable real-time documentation and analysis.

• Janssen’s Dr. Joe Hedrick and Critical Path Institute’s Dr. Inish O’Doherty commented on the importance of involving regulators early and considering what they will need from big data efforts. Dr. Hedrick encouraged JDRF to take the lead here, driving toward a critical path innovation meeting to talk about the impact of big data. On a practical note, Dr. O’Doherty mentioned that having data standards in the FDA-preferred CDISC format takes out the need for later curation and formatting – a clear win!

• Attendees raised a couple of caveats around the insights derived from big data/AI: (i) USF’s Dr. Jeffrey Krischer cautioned about sample bias and misapplication – “I would be happy to come up with a model with a great AUC (prediction). In any other room, I can predict who has type 1 diabetes – all I have to do is say nobody has it. That’d have a tremendously high AUC. Those making decisions about and evaluating these models have to be trained in order to make effective decisions”; and (ii) Western Michigan’s Dr. Craig Beam (as on day #1) emphasized that standards must be tied to clinical endpoints. In other words, strong predictive/sorting power of an algorithm must be assessed in the context of economics or patient care.

• Some of Dr. Mathieu’s clinician friends view big data and decision support as a threat – they’d prefer to “keep the steering wheel in their hands” and not give control back to the patients. (So depressing but we appreciated her candor.) Therefore, it becomes important to get clinicians on board, convincing them new tools both add value and enhance their work, rather than replacing them. We believe DreaMed has gotten the messaging down quite well here, positioning its Advisor Pro clinician decision support as “just another member of the team” (only you don’t have to bother anyone or say thank you). Ultimately, we hope overburdened providers see value in decision support tools and AID once they realize that (i) the tools should make their lives easier and save/make them money; and (ii) they will still be needed to administer optimal in-person care. However, it may change the skillsets and certainly the nature of the work – will young endos be well trained for this environment of decision support and CGM?

• Dr. Kowalski exclaimed “it’s incredible that we’re 12 years into CGM and it’s still treated as a novel technology – that’s unacceptable in my mind.” Mayo Clinic’s Yogish Kudva expounded on this point, noting that although there are 240 positions for endocrinology fellowships, very few of the programs are looking at competence with CGM analysis and CGM-based decision-making – if not at the early provider education level, then where will CGM competence develop? Dr. Kudva, fortunately, noted that organizations like the Endocrine Society, AACE, AADE, and JDRF are taking the lead in this area.

• Prof. Moshe Phillip stated the goal of holding a consensus meeting on time-in-range – presumably benchmarking goals – at ATTD in Berlin. This would come two years following a consensus meeting on outcomes beyond A1c, later published in Diabetes Care and after a broad stakeholder meeting with FDA in August, 2016.

Takeaways from Meeting Chairs/Hosts

Drs. Doyle, Kowalski, Mathieu, Dunne: Standardization, Big Data “A Means to an End”; Type 1 Patient Voice Perhaps Lacking at Gathering; Big Data Demands Diversity to Tackle, Results Must Add Value

Harvard’s Dr. Frank Doyle remarked in his summary comments how struck he was by the diversity of the audience, noting that the impressive cross-section of talent, background, and skill-sets reflects “the array of backgrounds needed to tackle these problems.” He expressed excitement regarding the new methods demanded by big data analysis. Indeed, KU Leuven’s Dr. Chantal Mathieu extolled the many “fresh approaches” to big data that were brought up throughout the meeting, highlighting the potential of big data to optimize processes and make clinical care more valuable. However, in order for the many prospects that were brought up at the meeting to become realities, Dr. Mathieu noted that we must overcome the nontrivial issue of data standardization – a pervasive theme throughout the meeting. To JDRF Chief Mission Officer Dr. Aaron Kowalski, standardization means prioritizing and having a laser-focus of where we’re heading; in his words, “it’s about the end, not the means.” The end is clearly improved clinical care and outcomes for people with or at risk of type 1 diabetes. However, Dr. Kowalski noted that this particular meeting could’ve better integrated patients’ perspectives. In his view, there was lots of discussion around outcomes and the clinical view, but not enough integration of what patients want and need – in his opinion, “clinical care aside, we need to hear that voice loud and clear.” As for future steps, Dr. Kowalski identified leveraging big data to move JDRF’s programs forward and his organization’s partnership with Helmsley as integral to progress; Dr. Doyle pointed to all of the high-impact action items identified in the breakout sessions. JDRF Research Director Dr. Jessica Dunne echoed Dr. Doyle’s comments, emphasizing that the breakout sessions generated “a lot of great things to build upon.” She urged attendees to reach out with ideas for implementation, reminding them that although JDRF is “very patient-focused,” it will be necessary to engage all stakeholders. Dr. Mathieu took a more granular approach in her closing comments, stressing that the greatest challenge of big data is translating it into value for patients.

• Dr. Doyle referenced Dr. Marian Rewers’ day #1 quote: “I really would like artificial pancreas and artificial intelligence to replace me and my colleagues.” While Dr. Doyle stipulated that, “we’d all agree we don’t want to replace doctors,” he does hope such advances will enable clinicians to perform “more effective work and perhaps more importantly have a broader impact.” We certainly agree that AI poses the massive potential to free up physicians’ time by taking on some of the more rudimentary tasks, allowing them to better tend to the medical and other factors that truly matter to patients. Many do also believe that in addition to “rudimentary tasks,” AI clearly (based on DIY and Medtronic and other data) doses insulin incredibly well – it’s not just about saving time for doctors but about giving many patients better management.

• In Dr. Mathieu’s opinion, privacy concerns remain a significant barrier to progress, noting that “consent and re-consent” are required with the GDPR in Europe before data can be shared. That was really depressing to hear. Baked into this idea is the concern surrounding big data and “big brother” – some patients are understandably worried that their personal information might be nefariously sold without their consent. Dr. Mathieu shared a particularly unnerving story, in which a colleague told her she could be identified based on only four factors: her age, sex, place of residence, and occupation.

Breakouts

1. Algorithms for AID: What Can We Do with the Data We Have Now? (JDRF RFA? Kaggle Competition for Glucose Prediction?); What Other Data Do We Need?; Would Scientific Journals Require Data Sharing?

A breakout including Drs. Eyal Dassau, Roy Beck, Yogish Kudva, Guillermo Arreaza-Rubin, and Frank Doyle zeroed in on data architecture – doing more with the data that we have and figuring out which data that we don’t have but would be useful – as the next step with respect to improving AID algorithms. As Helmsley’s Mr. Scott Kahn pointed out, one of the quick wins has to do with building translators between different data sets, echoing calls throughout the meeting for harmonization. JDRF’s Dr. Daniel Finan set the stage of the first “phase” of the breakout, leading with an implication that JDRF may launch an RFA as a follow-up to the Big Data Workshop and asking what can be done to improve algorithms’ adaptability and/or personalization with the CGM and insulin infusion data that we have at our fingertips today. According to Dr. Doyle, what distinguishes AID from other big data applications is time-dynamism: “We maybe collect data in the form of 280 CGM measurements per day over several months from a number of patients. Whereas with biomarkers for PTSD – another area of my research work – we collect a million data points in one drop of blood.” The consideration of trends makes glucose regulation a really tough computational problem. Further, we don’t know what the ROI would be for adding more data – how much data would it take to make strides in machine learning algorithms – and that’s something Dr. Doyle deemed worthy of looking into. Dr. Beck asserted that there will never be enough clinical trial data to learn enough about populations to tailor algorithms at the level of an individual, and that’s the real limit – instead, the goal must be to capture a lot of data from a lot of people at a reasonable cost. “Type 1 diabetes in some respect is a really hard challenge for AI. On the other hand, it’s probably like no other disease where you have so much individual information captured over a long time. Thinking about how we can get data from 25,000, 50,000, 100,000 people in some reasonable way to then be able to do a little signal:noise work. I’d say that’s a ‘major project.’” In a very well-received proposition that seemed to pick up some steam during the following lunch and report-backs, Tidepool’s Mr. Brandon Arbiter posed a “kaggle competition” for glycemic prediction. If this were to come to fruition, Tidepool would publish anonymous glucose, insulin, and carb data from ~500 patients, divvy the data up into one model “training data set” and one model “testing set,” and see which data scientist can build the best glucose prediction model (better predict future excursions => better prevent future excursions). Ideally, JDRF or another organization would provide a financial reward. As Dr. Doyle put it, “there are a lot more bright young minds out there we can turn loose on this problem.” We truly love this idea, given the incredible ingenuity in the open source/DIY community. There are also many entities including Medtronic/IBM Watson, One Drop, potentially Dexcom, and presumably others developing glucose prediction models. Bringing in outside minds and developing/testing models on large, complete data sets could be fruitful.

• “What additional data inputs could be useful (additional sensors, meta-data, contextual data), and how could they be used in the context of big data, personalization, and contextualization? What’s missing?” – Dr. Dassau. A number of attendees pointed out that collecting new data is limited to what people will wear (Dr. Beck suggest incorporating sensors into CGMs/pumps), and Illinois Institute of Technology’s Dr. Ali Cinar said he has “all” the additional inputs he needs to handle everything but meals: Accelerometer, PPG, GSR, and skin temperature. (Admittedly, that sounds like a lot of devices.) Others proposed analytical methods of arriving at the answer to “what’s missing.” Mr. Dave Schneider (daughter has type 1) recommended amassing the data we have, and then making predictions of what is needed based on gaps (something the Department of Defense does “all the time”).  “Once you understand what you’re getting, now you can look at what we need, perhaps a sensor that can analyze sweat.” Tidepool’s Dr. Ed Nykaza supplied one way of going about this: Build an algorithm based on population-level statistics, then test on the individual. Next, leverage machine learning at an individual level, adding in contextual data to rank the importance of the variables (sounds similar to a sensitivity analysis to us), at which point those variables can be applied at the level of the population. Drs. Nykaza and Beck agreed that it may be possible to give patients “real-time pings” during/after a glucose excursion, affording them the opportunity to provide contextual information. The algorithm could then use this information to tailor to the individual and impute missing data. This is a different application of a technique used in a Lilly/Joslin poster we noticed at ATTD.

  • Close Concerns’ Mr. Brian Levine, “in the interest of being provocative,” wondered why the discussion hadn’t turned to reducing burden on the algorithm itself. Referencing an earlier talk from Mr. Arbiter: “Brandon mentioned he’s getting near 90% time-in-range, with a mean glucose of 120 mg/dl on Loop. These kinds of outcomes are possible with the algorithms we have today. But what is the role of patient education, maybe around low-carb approaches to meals or SGLT-2 inhibitors? Perhaps funding a study with those adjuncts to closed loop would be valuable, because great outcomes are clearly possible today. What would adding additional input sensors do to Brandon’s 90% time in range and minimal hypoglycemia?” Of course, not everyone is an early adopter nor of a socioeconomic status that would allow them to maintain a healthy low-carb diet, and SGLT-2s are currently not indicated for type 1 diabetes (under FDA/EMA review), but there is something to be said for figuring out what behaviors work with an algorithm in tandem with figuring out what algorithm features work for the individual. Adam has seen similar data on Loop and continues to wonder about this too – given all the AID field’s focus on covering meals, it’s important to note that current basal modulation algorithms CAN handle low-carb meals without announcement. While all users may not be willing to take such an approach, certainly more could benefit than currently are. Adding meal-specific boluses to closed loop algorithms would go a long way, since most people tend to eat the same things. (“I’m about to eat my usual Mexican food dinner.”) Beta Bionics has arguably the best approach to meal boluses we’ve seen so far – small, typical, large meal – and we suspect systems will move to this simplified, adaptive approach over time.

• Funding organizations were vocal throughout the two days about implementing data sharing policies for grantees, but Dr. Doyle called journals “the hidden factor,” urging them to establish similar requirements. This would keep industry, who can often self-fund, from falling through the data-sharing cracks, since they still want to be published. Dr. Beck noted progress, since starting July 1^st, major international journals (e.g., NEJM, Annals of Internal Medicine, JAMA) require authors to either (i) make their data sets available; or (ii) explain why they are not doing so.

• NIH’s highly-respected Dr. Guillermo Arreaza-Rubin participated in the breakout, picking the room’s collective brain on a host of topics. It was great to see a leader from the biggest US scientific funding organization present and so engaged in dialogue. See some quotes/ideas below:

  • “In order to design an algorithm, you need data to feed that. At the same time, algorithms integrate data and may generate novel data. How do you harmonize that process and use it more effectively so that at the end, you have more personalized adaptable algorithms to generate data that’s valuable for healthcare providers and patients?” (In sum, we believe he was asking about adaptive algorithms that can assess their own performance, make tweaks to parameters, and then make that data clear to patients/HCPs. For instance, it would be valuable to know if an algorithm’s settings were not permitting it to deal with early-morning hyperglycemia.)

  • “NIH has a so-called Big Data to Knowledge program, but that’s mainly based on use of data in basic research (genomics,…). That’s multiple terabytes of data. Ambitious. It is based on FAIR principles – findable, accessible, reusable, and interoperable. Those could be used as reference for what we’re doing here. How do we apply those principles to the generation of data by the methods we use? The other thing, we are discussing how to put together Big Data to Knowledge, but now it’s looking more like Big Data to Value to Healthcare Providers and Patients. That’s not only based on the use of technologies, but also on the use of other tools in the day-to-day clinic, hospital, outpatient setting. Those are things we’re discussing continuously inside NIH.”

  • Dr. Kudva lamented a missed opportunity to begin studying the 670G sooner after approval in the post-market setting than was realized. Dr. Arreaza-Rubin took an interest, asking what sort of study Dr. Kudva had in mind for future closed loop systems, but Dr. Kudva didn’t have time to respond.

• A type 1 patient with an expansive background in economics had the following to say on Dexcom’s weekly Clarity reports: “I feel like the report I get from Dexcom could be so much better. It’s a very reactive document, historical data, the recommendations are lame. I don’t need Dexcom to tell me I’m high from 3-5 AM, I can see that. I want them to help me understand why I’m high Monday 3-5 AM, but not Wednesday 3-5 AM.” We imagine this could be a common sentiment, since it’s the ultimate goal of pattern detection with CGM – the WHY. At the same time, there certainly are some patients who will appreciate various patterns being pointed out, patients who may not even be otherwise looking at the data. More fully-featured decision support is going to be a huge value-add to CGM, that is for sure.

Prioritization Matrix – Algorithms for automated insulin delivery

2. Improvement in Clinical Care/Decision Support Tools: Dr. Aaron Kowalski on Diabeter, “Ripping Off the Band-Aid” for Value-Based Care; Dr. Chantal Mathieu Advocates for Redefined Patient Outcomes

The Improvement in Clinical Care/Decision Support Tools breakout session, led by Dr. Mark Clements (Children’s Mercy Kansas City), fostered lively debate surrounding value-based care, desired outcomes, the ability of pilots to engage stakeholders, and data standardization. Participants struggled to contend with a fundamental tradeoff – should solutions be considered that fit within the confines of the current fee-for-service healthcare system, or, as Dr. Aaron Kowalski urged, should we “rip off the Band-Aid” and envision a new value-based system that allows for the right solutions to be implemented? Perhaps Dr. Moshe Phillip captured the mood best when he succinctly stated: “I would suggest changing the world.” Read on for some of the major themes from the discussion, and see a list of action items below ranked on impact and effort axes.

• On value-based care: Dr. Kowalski pointed to Diabeter, which he suggested many look to as a model. As a reminder, Diabeter is a Dutch type 1 diabetes clinic, which negotiates directly with payers for a lump sum of money with which to treat patients. (Medtronic acquired it in April 2015.) Dr. Kowalski characterized Diabeter as “amazing” and said it is “ludicrous” that attendees even had to consider whether payers would reimburse a tool like DreaMed’s Advisor Pro – optimizing basal rates based on CGM data in less time with automated algorithms provides such a clear benefit to patients and providers. If healthcare does not move towards a value-based system, Dr. Kowalski explained, we are simply “band-aiding” with solutions, instead of “driving full-tilt” at the system so that when it comes time to plug in solutions, the right ones can be selected for the right reasons. Dr. Chantal Mathieu agreed, claiming that “the way to go is talking immediately to payers” and that “the clever ones will figure it out.” That may be optimistic, particularly in the US. While Dr. Sanjeev Mehta shared Dr. Kowalski’s vision for a value-based system, he countered that it is important to “face the reality of the system we operate in.” He noted that when he asked 10 CFOs and COOs of various health systems to describe what is valuable to them, they didn’t cite factors like patient outcomes, clinical experience, or even the duration of visit; rather they talked about things like CMS star ratings. Dr. Mehta’s comments served as a sobering reminder that the real keeper of the keys have very different ideas regarding value proposition, and at least for now, consideration of these differences will be critical. Who can blame them in the current system and given that they don’t necessarily see patients face-to-face every day?

• On outcomes and incentives: Dr. Clements urged participants to consider what it will take to incentivize institutions and clinics to adopt decision support tools like DreaMed’s Advisor Pro. Dr. Phillip asserted that tools should show some benefit, offering improvements in A1c, time-in-range, or reduced hypoglycemia. (To that, we would add improving the clinician experience!) However, Dr. Chantal Mathieu argued for the “need to reflect on what we define as outcomes.” She readily admitted that she “really likes” DreaMed’s Advisor Pro, but finds it to be “very glucose- and insulin-centered.” Instead, she’d like to see more of an emphasis on the value being experienced by the patient, claiming that patients who receive solutions tailored to their own experiences will be more likely to recommend their clinic, ultimately benefiting the institution. Dr. Mathieu personally would like to see a solution that could phenotype her newly diagnosed type 1 patients based on the degree to which they’ll require support. More “personalized” and predictive decision support systems are surely coming, but insulin titration based on CGM was a low-hanging fruit and a clear first step. Dr. Phillip claimed that “most of the care we’re doing now can be done by patients themselves given the right advice,” but Dr. Mathieu disagreed, asserting “I’ve been wrong so many times. Will this person do well by himself, or is this the guy that my team will have to be around?” One Drop’s Dr. Dan Goldner put a bow on the discussion: “The better the tools, the easier it will be to be an engaged patient. Right now, it’s hard.”

• On the utility of pilots: While participants were eager to see pilots conducted investigating new methods of healthcare delivery, many questioned whether such studies are the best way to demonstrate value to payers. As T1D Exchange CEO Ms. Dara Schuster noted during the summary discussion, a lot of payers have evaluated pilots, but there haven’t been many translated into actual practice (a phenomenon we’ve heard referred to as “pilotitis”). For this reason, the breakout group decided to delineate pilots to the “low-impact, low-effort” quadrant in the grid below. Ms. Schuster also suggested that already-conducted pilots be considered to identify gaps. It is a fascinating catch 22 for payers: Pilots are conducted as a means to identify which solutions could be successful without taking on too much risk, but in order to be successful, a solution will generally have to be scalable.

• On data standardization and sharing: While all participants agreed on the need for data standardization, Dr. Mathieu pointed out that the actual formatting of the EHR doesn’t need to be identical; rather the data must be transmitted in a standardized way. She explained that in Belgium, there used to be seven or eight different EHRs until the Ministry of Health “turned the discussion around” and said: “we want you to be able to shoot out your data in a standardized way.” Essentially, the system itself can differ so long as there is a software capable of transforming the data according to standards. One wonders, what would prompt a result like this in the US? UVA’s Dr. Harry Mitchell noted the need to break down silos between the clinic-side of institutions and research so as to facilitate better data sharing. Still, as Ms. Schuster pointed out, this kind of concerted effort is “like moving a dinosaur down a stream of water.” She highlighted the importance of advocacy, proposing: “You can’t underestimate the value of champions pushing for these changes.”

Prioritization Matrix – Improvement in Clinical Care/Decision Support Tools

3. Actionable Ideas Towards Prediction and Diagnosis in Prevention: “Data Dictionary” and Standardization = Quick Wins; Stratification and Connection to Other Autoimmune Diseases = Major Projects

In a fascinating breakout session on prediction and diagnosis in prevention led by Drs. Desmond Schatz and Dr. Jessica Dunne, emphasis was placed on cataloging big data currently available and standardizing future data collection. These insights were developed into “quick wins” (high impact/low effort) and “major projects” (high impact/high effort), summarized in the bullets and table below.

• A “data dictionary” cataloguing available big data: The purpose of this dictionary is two-fold: (i) to provide a resource from which researchers can find and use big data; and (ii) to characterize different data collection and storage methods to serve as inspiration for standards. Participants agreed that organization on the front-end of the big data movement could save a lot of hassle on the back-end. Moreover, aggregation of all big data from major type 1 diabetes datasets (DIPP, DiPiS, DAISY, TEDDY, etc.) could help to identify gaps in current knowledge, paving roads for future research. This said, there were mixed feelings as to whether more data is needed before progress can be made, and we think that a catalog could serve as a good starting point in determining an answer to this question. The field will not stop amassing data (and nor should it), but the point is that there should be a clear methodology moving forward to ensure that the data collected is relevant and usable among itself and with existing data; and there will continue to be simultaneous efforts to clean and harmonize that existing data. See below for a few quotes that we feel captured the conversation behind this “quick win:”

  • “We have a bunch of information, but we don’t know if we need more or if we can do something with what we have.” – Dr. Leandro Balzano Nogeuira (University of Florida)

  • “This morning we said, ‘let’s add more data!’ Wait a minute – we’re still figuring out how to analyze the data we already have.” –  Dr. Jeffrey Krischer (University of South Florida)

  • “I think there are cautionary flags that need to be raised before adding in data that will ultimately obfuscate our ability to draw conclusions.” – Dr. Krischer

  • “For me, it would be useful to know what data is actually out there, and to have it be well characterized.” – Dr. Ana Conesa (University of Florida)

• A template for data collection in future studies: As it currently stands, big data sets are difficult to combine – NIDDK’s Dr. Beena Akolkar asserted that even simple variables such as sex can be coded differently between studies, making data aggregation time-consuming at best and virtually impossible at worst. If coding even a binary variable like sex is complicated, we can only imagine how complex continuous medical metrics can be (and multiplied by however many variables there are over however many time points there are…instant headache!). Accordingly, a pervasive theme throughout the breakout session was how to best standardize big data to facilitate aggregation and analysis of data sets. For example, JDRF could establish a list of 10 characteristics to be collected in every type 1 prevention study with specific directions for collection methodology. While creating data standards is certainly compelling, we’re not quite sure if it constitutes a “quick win.” Creating a template for data collection is likely to require significant investment and time – Dr. Diane Wold’s presentation on CDISC’s efforts to develop type 1 diabetes data standards (see below) suggested otherwise. However, most agreed that this investment is well worth it.

• Identify what is meant by a "responder" to type 1 therapies: All participants seemed to agree that type 1 diabetes is unlikely to have a one-size-fits-all solution, given its heterogeneity of pathogenesis and reactions to tested therapies (such as ATG and oral insulin). The spectrum of reactions to therapies currently lacks a definitive cutoff against which researchers can identify those who respond well to a therapy. Once this cutoff is set, the genotypical and phenotypical commonalities of those who responded well could be determined to create a potential target population for a given drug. Janssen's highly respected Dr. Joe Hedrick suggested three clear cutoffs for type 1 therapies based on the stages of type 1 diabetes development: (i) Does the intervention stop those at risk from developing autoimmunity?; (ii) Does the intervention stop the progression from one autoantibody to two?; and (iii) Does the intervention delay the progression from dysglycemia to diabetes? Of course, any “responder” cutoff would likely be a moving target as interventions improve through better population targeting, timing of intervention, and molecule/delivery.

  • Stratification of the type 1 population into homogeneous subgroups composed of individuals with the same biomarkers and phenotypes was suggested as the next step forward (see “major projects” in the table below). As the pot of gold at the end of the rainbow, these targeted populations could then be exposed to the therapy to which they are most likely to respond. This said, identifying these populations becomes increasingly difficult as the scope is narrowed. We believe Dr. Krischer captured both of these ideas well:

    • “Data may be relevant for one form of type 1 diabetes but not for another, which can create a lot of noise and make it difficult to understand pathogenesis. In my opinion, the better way is to stratify groups and understand them in a narrower scope. We want to separate the data, not combine it.”  – Dr. Krischer

    • “I’ve screened a thousand people just to find one that has two antibodies for type 1 diabetes.”  – Dr. Krischer

• Dr. Hedrick also suggested expanding the scope of big data to draw insights from the commonalities of various autoimmune diseases – a prospect which was met with some controversy. According to the UVA’s Dr. Steve Rich, type 1 diabetes has similar genetic risk factors to other autoimmune diseases, such as Juvenile Idiopathic Arthritis (JIA), which can enter remission. However, the commonalities in genetics may not necessarily translate into clinically relevant insights, as the heritability of response to therapies (i.e., the degree to which an individuals’ genetics impacts their response to a given therapy) is not well understood. This means that costly investigations of disease connections may prove fruitless, especially given that type 1 diabetes appears to be more complex and heterogeneous than many autoimmune diseases. That said, Dr. Hedrick noted that perfection can be the enemy of progress, particularly in the case of type 1 diabetes where little progress has been made thus far, and there was enthusiasm from the group that the effort would be worthwhile:

  • “What I’m suggesting is that all of these [autoimmune diseases] have intricacies and complications to them but not necessarily unique characteristics. If each one of us in this room had type 1 diabetes, we could all have unique types, but what is the practical knowledge that we need to make a difference? Using big data, we can say, ‘These individuals with type 1 look a lot like these patients with JIA,’ and we can try to apply concepts between diseases. If we wait until we have a perfect picture, we’ll be having this conversation 50 years from now.” – Dr. Hedrick

Prioritization Matrix – Prediction and diagnosis in prevention

-- Brian Levine, Peter Rentzepis, Maeve Serino, Adam Brown, and Kelly Close