Clinical Studies Overview
Two clinical studies were conducted in 2020 and 2021 involving a total of 25 radiologists double blind reading mixed (MRI vendor and field strength) sets of 150 MRI cases each with biological ground truth points acquired with 3D in-bore MRI guidance or FUSION guidance. These truth points were used to measure the performance of detection by both radiologist and the AI software, ProstatID individually and as a group score using current acceptable statistical methods (MRMC). In the FDA study, the radiologists interpreted the MRI data sets first without the aid of the software (CADe, CADx), and after a thirty-day memory wash-out, they read them again with the aid of ProstatID.
Performance of both radiologist, group and software was measured via the area under the ROC (receiver operating curve) or AUROC which is a measure of sensitivity vs specificity. This standard method measures how well one identifies a cancerous patient without calling all patients cancerous – meaning that it considers the entire confidence matrix of true positives, false positives, true negatives and false negatives.
The highest AUROC number (grid method – out of 1.00) attained by the radiologist was 0.71 while that of the software was 0.936.
Another method of measurement employed the free-response ROC or FROC. The performance of the algorithm to detect true positives was assessed using the free-response ROC (FROC) curve. The FROC analysis is a method for evaluating the performance of both detection and classification in a free-response system. In this case, the free-response system is the localization and classification of cancerous lesions in prostate MRI. A single case may have none, one, or multiple cancerous lesions. The FROC curve is a plot of sensitivity versus false positives per patient. FROC analysis needs two main components for evaluation: a designation of “detection” and a measure of “confidence”. For ProstatID, the level of confidence at a particular point is the value of the ProstatID index at that point. Detections associated with a positive biopsy were classified as true positives. Detections that did not match a positive biopsy (i.e., a negative biopsy or a detection in normal tissue) were classified as false positives.
Standalone Detection Accuracy (FROC Analysis)
ProstatID demonstrated a detection performance with a sensitivity of 80% at a rate of ONE false positive per patient compared to a 54% sensitivity score by the radiology group. This implies a significant improvement in detection accuracy of the software.