Effective transfer of tumor annotations from hematoxylin and eosin to fluorescence images of breast and lung tissues.

Significance: Accurate transfer of annotations from histological images to fluorescence images is essential in developing deep learning (DL)-based optical imaging systems for intraoperative assessment of tumor margins. Manual annotation is time-consuming, prone to interobserver variability, and impractical for large-scale datasets. Aim: We present a semi-automated method that can effectively transfer tumor annotations from pathologist-annotated hematoxylin and eosin (H&E) images to fluorescence images captured using microscopy with ultraviolet surface excitation (MUSE). This method is not intended for intraoperative use but rather to facilitate the creation of annotated datasets for DL model development. Approach: Our semi-automated method consists of nonrigid image registration, outline extraction and refinement, and annotation transfer. The method was applied to H&E and MUSE image pairs from 35 breast and lung tissue samples. Manual annotations in MUSE images were used as the ground truth for evaluation. Results: The proposed method achieved a Dice score coefficient of 0.87 ± 0.07 , convolutional-neural-network-based feature similarity of 0.94 ± 0.04 , and a normalized Hausdorff distance of 0.15 ± 0.06 across the dataset. Conclusion: These results demonstrate that the method provides a fast and accurate solution for generating annotated MUSE datasets necessary for training DL algorithms for intraoperative tumor margin detection.