External validation of a commercially available deep learning algorithm for fracture detection in children
- PMID: 34810137
- DOI: 10.1016/j.diii.2021.10.007
External validation of a commercially available deep learning algorithm for fracture detection in children
Abstract
Purpose: The purpose of this study was to conduct an external validation of a fracture assessment deep learning algorithm (Rayvolve®) using digital radiographs from a real-life cohort of children presenting routinely to the emergency room.
Materials and methods: This retrospective study was conducted on 2634 radiography sets (5865 images) from 2549 children (1459 boys, 1090 girls; mean age, 8.5 ± 4.5 [SD] years; age range: 0-17 years) referred by the pediatric emergency room for trauma. For each set was recorded whether one or more fractures were found, the number of fractures, and their location found by the senior radiologists and the algorithm. Using the senior radiologist diagnosis as the standard of reference, the diagnostic performance of deep learning algorithm (Rayvolve®) was calculated via three different approaches: a detection approach (presence/absence of a fracture as a binary variable), an enumeration approach (exact number of fractures detected) and a localization approach (focusing on whether the detected fractures were correctly localized). Subgroup analyses were performed according to the presence of a cast or not, age category (0-4 vs. 5-18 years) and anatomical region.
Results: Regarding detection approach, the deep learning algorithm yielded 95.7% sensitivity (95% CI: 94.0-96.9), 91.2% specificity (95% CI: 89.8-92.5) and 92.6% accuracy (95% CI: 91.5-93.6). Regarding enumeration and localization approaches, the deep learning algorithm yielded 94.1% sensitivity (95% CI: 92.1-95.6), 88.8% specificity (95% CI: 87.3-90.2) and 90.4% accuracy (95% CI: 89.2-91.5) for both approaches. Regarding age-related subgroup analyses, the deep learning algorithm yielded greater sensitivity and negative predictive value in the 5-18-years age group than in the 0-4-years age group for the detection approach (P < 0.001 and P = 0.002) and for the enumeration and localization approaches (P = 0.012 and P = 0.028). The high negative predictive value was robust, persisting in all of the subgroup analyses, except for patients with casts (P = 0.001 for the detection approach and P < 0.001 for the enumeration and localization approaches).
Conclusion: The Rayvolve® deep learning algorithm is very reliable for detecting fractures in children, especially in those older than 4 years and without cast.
Keywords: Artificial intelligence; Deep learning algorithm; Fractures; Pediatric; Radiographs.
Published by Elsevier Masson SAS.
Similar articles
-
External validation of an artificial intelligence solution for the detection of elbow fractures and joint effusions in children.Diagn Interv Imaging. 2024 Mar;105(3):104-109. doi: 10.1016/j.diii.2023.09.008. Epub 2023 Oct 8. Diagn Interv Imaging. 2024. PMID: 37813759
-
Comparison of diagnostic performance of a deep learning algorithm, emergency physicians, junior radiologists and senior radiologists in the detection of appendicular fractures in children.Pediatr Radiol. 2023 Jul;53(8):1675-1684. doi: 10.1007/s00247-023-05621-w. Epub 2023 Mar 6. Pediatr Radiol. 2023. PMID: 36877239
-
Artificial Intelligence for Detecting Acute Fractures in Patients Admitted to an Emergency Department: Real-Life Performance of Three Commercial Algorithms.Acad Radiol. 2023 Oct;30(10):2118-2139. doi: 10.1016/j.acra.2023.06.016. Epub 2023 Jul 18. Acad Radiol. 2023. PMID: 37468377
-
Validation of a Deep Learning Algorithm for the Detection of Malignant Pulmonary Nodules in Chest Radiographs.JAMA Netw Open. 2020 Sep 1;3(9):e2017135. doi: 10.1001/jamanetworkopen.2020.17135. JAMA Netw Open. 2020. PMID: 32970157 Free PMC article.
-
Is Deep Learning On Par with Human Observers for Detection of Radiographically Visible and Occult Fractures of the Scaphoid?Clin Orthop Relat Res. 2020 Nov;478(11):2653-2659. doi: 10.1097/CORR.0000000000001318. Clin Orthop Relat Res. 2020. PMID: 32452927 Free PMC article.
Cited by
-
Artificial intelligence in commercial fracture detection products: a systematic review and meta-analysis of diagnostic test accuracy.Sci Rep. 2024 Oct 4;14(1):23053. doi: 10.1038/s41598-024-73058-8. Sci Rep. 2024. PMID: 39367147 Free PMC article.
-
Detecting pediatric appendicular fractures using artificial intelligence.Rev Assoc Med Bras (1992). 2024 Aug 30;70(9):e20240523. doi: 10.1590/1806-9282.20240523. eCollection 2024. Rev Assoc Med Bras (1992). 2024. PMID: 39230068 Free PMC article.
-
Elbow trauma in children: development and evaluation of radiological artificial intelligence models.Res Diagn Interv Imaging. 2023 Apr 29;6:100029. doi: 10.1016/j.redii.2023.100029. eCollection 2023 Jun. Res Diagn Interv Imaging. 2023. PMID: 39077546 Free PMC article.
-
Improving traumatic fracture detection on radiographs with artificial intelligence support: a multi-reader study.BJR Open. 2024 Apr 25;6(1):tzae011. doi: 10.1093/bjro/tzae011. eCollection 2024 Jan. BJR Open. 2024. PMID: 38757067 Free PMC article.
-
Artificial intelligence improves resident detection of pediatric and young adult upper extremity fractures.Skeletal Radiol. 2024 Dec;53(12):2643-2651. doi: 10.1007/s00256-024-04698-0. Epub 2024 May 2. Skeletal Radiol. 2024. PMID: 38695875
MeSH terms
LinkOut - more resources
Full Text Sources
Miscellaneous