Development of Artificial Intelligence-Based Real-Time Automatic Fusion of Multiparametric Magnetic Resonance Imaging and Transrectal Ultrasonography of the Prostate

Objectives: To report the development of artificial intelligence (AI)-based software to allow for the autonomous fusion of transrectal ultrasound and multiparametric magnetic resonance images of the prostate to be used during transperineal prostate biopsies. Materials and methods: This study was approved by the Institutional Review Board (protocol ID3167CESC). The automatic software development for fusion biopsy involved three steps: 1) Developing an AI component to segment the prostate during ultrasound; 2) Developing the component to segment anatomical structures in magnetic resonance images using labeled datasets from the Cancer Imaging Archive and in-house scans; 3) Developing the fusion component to register segmented ultrasound and magnetic resonance images using a three-step process: pre-alignment, rigid alignment, and elastic fusion, validated by measuring the lesion distance between modalities. Statistical analysis included descriptive statistics and the Mann-Whitney U test, evaluating outcomes with Dice scores and average precision metrics. Results: The ultrasound component showed a Dice score of 0.87 with a test set of 75,357 images and 28,946 annotations. The magnetic resonance component achieved a Dice score of 0.85 on a test set of 2,494 images and annotations. It also demonstrated a mean Average Precision of 0.80 for bounding boxes and 0.88 for segmented objects, both measured at a 50% intersection over union threshold. The fusion AI component reduced the median magnetic resonance-ultrasound lesion distance from 8 mm (IQR 6-9) after rigid fusion to 4 mm (IQR 3-5) after elastic fusion (p<0.001). Conclusion: A data processing pipeline and AI were created to allow for the autonomous fusion of ultrasound and magnetic resonance images to be ideally used during transperineal prostate biopsies.