A transformer-based pathomics model using endoscopic biopsy WSIs for predicting pathological complete response to preoperative immunotherapy in colorectal cancer.

[BACKGROUND] Immune checkpoint inhibitors have improved the survival of colorectal cancer (CRC) patients. Patients who respond well to preoperative immunotherapy and achieve a complete response may potentially avoid surgery. Leveraging whole slide images (WSIs) from endoscopic biopsies, this study aims to construct a predictive model using a deep learning approach to identify potential pathological complete response (pCR) in CRC patients undergoing preoperative immunotherapy.

[METHODS] CRC patients who received preoperative immunotherapy from West China Hospital, Sichuan University were retrospectively included and randomized into the training and the validation sets at a 3:1 ratio. Using pathological outcomes as the reference standard, a predictive model was developed based on the H&E-stained WSIs of endoscopic biopsy, incorporating the Swin Transformer architecture and a self-attention mechanism augmented by convolutional neural networks (CNNs). And the Clustering-constrained Attention Multiple Instance Learning (CLAM) framework was used to optimize pathological image analysis. Through attention-based visualization, the top 5 % of patches most influential for determining tumor response to preoperative immunotherapy were identified.

[RESULTS] The pCR rate was 30.6 % (22/72) in the training cohort and 30.4 % (7/23) in the validation cohort. The predictive model yielded an area under curve (AUC) of 0.830. Attention-based visualization in the validation set revealed that among the top 5 % of patches contributing to the prediction, 62.25 % were tumor patches and 37.75 % were non-tumor patches.

[CONCLUSIONS] A deep learning-based pathomics model, which distinctly focuses on both tumor and non-tumor regions, has the potential to predict tumor response to preoperative immunotherapy in CRC patients.