Photo Source: Dragon Images/Getty Images
Many venture-backed digital health startups aren’t clinically robust, as measured by the number of regulatory filings and clinical trials, according to a study published in JMIR.
The analysis examined digital health companies using the Rock Health Digital Health Venture Funding Database, FDA data on 510(k), De Novo and premarket approval filings, and the number and type of clinical trials listed on ClinicalTrials.gov. Researchers then assigned a “clinical robustness” score for each company, the sum of the number of regulatory filings and clinical trials weighted equally.
Of the 224 startups included in the study, 98 had a clinical robustness score of 0, while 45 received a five or higher. The average score was 2.5, with 1.8 clinical trials and 0.8 regulatory filings, and the median score was one.
Companies that aimed to diagnose conditions had the highest average score at 2.8, then treatment-focused companies, at 2.2, and finally prevention startups, with a score of 1.9.
Startups that sold their products to employers received an average clinical robustness score of 3.1, compared with 2.7 for providers, 2.2 for consumers and 2.0 for payers.
Researchers also examined public claims made by the companies, or qualitative statements about clinical, economic and engagement outcomes. The average number of claims made by the companies was 1.3, with 43% of companies making zero claims. The study also noted that startups that sold to employers made more clinical, economic and engagement claims compared with companies that sold to other customer types, like consumers, providers and payers.
Overall, the study didn’t find any correlation between clinical robustness and the number of claims, robustness and total funding, or clinical robustness and startup age.
“Despite the hundreds of digital health companies targeting the myriad of needs across the care continuum, clinical robustness and public communication of claims remains low across much of the sector,” the study’s authors wrote. “These results highlight a significant opportunity for companies to differentiate themselves and for customers to demand greater validation for the products and services they purchase.”
WHY IT MATTERS
The study’s authors, some of whom are members of Rock Health’s research team, noted a few limitations to the analysis, like the potential to miss filings or clinical trials, and the decision to include only venture-backed companies that had raised more than $2 million.
They suggest future studies could use condition-specific metrics for effectiveness that could be standardized across clinical areas to provide a clearer picture of impact.
Though the analysis found many startups lacked clinical robustness, it did find 20% of startups received a score of five or higher, suggesting a core of highly tested products. Still, the number of companies with low scores demonstrated that many venture-backed startups lacked clinical evidence.
“Although this subpopulation may portend progress, the lack of meaningful clinical validation for nearly half of digital health companies (44% had a clinical robustness score of 0) highlighted a major gap in healthcare technology today,” researchers wrote.
“The lack of overall correlation between a company’s total venture funding and its clinical robustness score similarly highlighted a significant asymmetry in how companies are potentially valued in today’s marketplace (ie, no correlation between clinical impact and funding). However, it is possible that funding amounts reflect future anticipated value rather than current value.”
