Just as a building needs secure foundations to stay strong and stable, healthcare innovations should be underpinned by solid clinical evidence showing that they are safe and make a difference to people’s lives.
For conventional medicines, this evidence base is built from long-established building blocks of preclinical data, clinical trials and regulatory approvals that can take years to assemble. But in the booming world of digital health, the same rules don’t always seem to apply.
A recent study of more than 200 US digital health startups1, led by digital health investors Rock Health, revealed that nearly half had no clinical data or FDA filings to back them up, with just 20% of companies making up the bulk of the evidence provided.
Meanwhile in Europe, the Medical Device Coordination Group (MDCG), which advises the EU on the regulation of novel digital health technologies, has warned that more than 90% of the provisional approvals2 for medical devices granted on the basis of minimal clinical data are due to expire in 2023-24. More than half of the full regulatory applications for these products are currently incomplete, suggesting that sufficient evidence is yet to be gathered.
That’s a big problem, not only for the patients and clinicians who rely on these technologies, but for the health tech sector as a whole. Unless companies step up and produce solid clinical data demonstrating the foundational safety and effectiveness of their products, there’s a risk that the market for digital health - together with public confidence and trust - could start to fall apart.
Let’s get rigorous: measuring clinical robustness
It’s easy to ask that digital health products demonstrate evidence supporting their claims - often referred to as clinical rigour or robustness - but much harder to know what this should look like in practice.
In their study3, the Rock Health team combined the number of registered clinical trials and FDA filings from each company together to produce a measure of clinical robustness, which they could then compare against the number of health claims being made.
Another approach would be to look at the number of peer-reviewed papers published about a particular company’s product or technology. However, publication isn’t necessarily a guarantee of research quality or reliability, and the increasing tendency to ‘salami slice’4 one study into multiple publications can also inflate the tally.
More formal frameworks, such as CONSORT-EHEALTH5, aim to provide a more comprehensive checklist for reporting and assessment of the evidence supporting digital health tools. The World Health Organization takes a broader approach when it comes to assessing digital health interventions6, suggesting that regulators should look at the overall strength of confidence in the data about a particular technology and make adjustments according to the scale and scope of the product.
Reassuringly for Huma, of the roughly 40 peer-reviewed studies we have published about our technology, the majority are rated as good according to the WHO criteria - the second-highest ranking available. This reflects the fact that the highest rating is only awarded to multi-centre randomised controlled trials, which can be the most expensive and time-consuming study types.
Embedding clinical rigour into digital health at Huma
Here at Huma, we take the clinical robustness of our platforms seriously. The two of us have recently stepped into new senior positions dedicated to generating evidence and ensuring clinical rigour, as Global Clinical Directors of Evidence Generation & Information Governance (Arrash) and Patient Safety & Quality (Peter), bringing more than 25 years of clinical experience between us.
We’ll be working together with clinicians and patients to ensure that their voices are heard by our technical development team so we continue to create digital health solutions that truly connect people to the care they need.
We’re fortunate to have our in-house clinical colleagues on hand, and are developing plans to set up patient and clinical advisory groups to get broader insights. We’ll also be keeping a close eye on the scientific literature and changing landscape around digital health regulation, to make sure we’re getting it right.
Huma is committed to evaluating our projects and sharing what we learn with the people who need to know. This is something we bake in from the start of every study, creating a prospective evaluation plan for how we’ll gather evidence and assess whether something is working before a single patient is recruited.
It’s a strategy we’ve used to good effect for our ‘hospital at home’ platform. Rather than simply give our partners the technology to run virtual COVID-19 wards in Germany and let them get on with it, we agreed with them how we would measure success.
We were able to show that hospitals using the platform had up to four times lower mortality rates, 40% shorter hospital stays and the technology reduced patients’ need for intensive care, generating robust data that was then published in a peer-reviewed journal7.
The future of digital health is evidence-based
Digital healthcare is a relatively new and fast-growing industry in an ever-changing technological and regulatory world. The decisions that we make now will shape our industry for years to come, for better or for worse.
As the sector grows, we have a duty of care to do our part to ensure that it is as healthy as we wish our patients to be. At Huma, we’re committed to generating robust clinical evidence that builds trust in these transformative tools and technologies, and to sharing our results with the digital health community and the wider world to help people live longer, fuller lives.
Day S, Shah V, Kaganoff S, Powelson S, Mathews SC. Assessing the Clinical Robustness of Digital Health Startups: Cross-sectional Observational Analysis. J Med Internet Res. 2022 Jun 20;24(6):e37677. DOI:10.2196/37677.
Directorate-General for Health and Food Safety 2022 MDCG 2022-11 - MDCG Position Paper: Notice to manufacturers to ensure timely compliance with MDR requirements [Online] Available at: https://health.ec.europa.eu/system/files/2022-06/mdcg_2022-11_en_0.pdf [Accessed on: 28 July 2022]
Day S, Kaganoff S, Ryan R 2022 Stacking up against the clinical robustness index: A primer [Online] Available at: https://rockhealth.com/insights/stacking-up-against-the-clinical-robustness-index-a-primer/ [Accessed on: 28 July 2022]
Elsevier Factsheet: Salami Slicing 2019 [Online] Available at: https://www.elsevier.com/__data/assets/pdf_file/0011/653888/Salami-Slicing-factsheet-March-2019.pdf [Accessed on: 28 July 2022]
Eysenbach G CONSORT-EHEALTH Group CONSORT-EHEALTH: Improving and Standardizing Evaluation Reports of Web-based and Mobile Health Interventions J Med Internet Res 2011;13(4):e126. DOI: 10.2196/jmir.1923
World Health Organization 2016 Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment [Online] Available at: https://apps.who.int/iris/bitstream/handle/10665/252183/9789241511766-eng.pdf?sequence=1 [Accessed on: 28 July 2022]
Lim, A. et al. An Outpatient Management Strategy Using a Coronataxi Digital Early Warning System Reduces Coronavirus Disease 2019 Mortality Open forum infectious diseases (2022) 9(4), ofac063 DOI: 10.1093/ofid/ofac063