Virtual NBI image synthesis using stable diffusion for enhanced recognition of early gastric cancer: a technical validation study.

1.
Introduction
According to the study, about 20 million new cases of malignant tumors were diagnosed globally in 2022, of which gastric cancer (GC) was the fifth most common malignant tumor in terms of both incidence and death rate [1]. Early gastric cancer (EGC) is defined as tumor tissue confined to the mucosa and submucosa, with or without metastases [2]. The overall prognosis of GC patients is poor, with 5-year survival rates of only 14.0–45.6% for patients with pathologic stages III–IV, whereas EGC patients not only have a 5-year survival rate of more than 90% after endoscopic treatment [3,4]. This shows the importance of timely detection of EGC.
Currently, white light endoscopy (WLE) is the most commonly used diagnostic procedure for EGC [5]. However, some EGC lesions are difficult for endoscopists to detect in WLE images due to objective reasons such as small extent, flat morphology, and shallow depth of invasion of some EGC lesions [6–8]. Related studies have shown that GC has a misdiagnosis rate ranging from 2.0% to 40.0% in WLE examination [9–12]. This indicates that the diagnostic effect of WLE on EGC is not satisfactory. In recent years, image enhanced endoscopic technology has made rapid progress. Narrow-band imaging (NBI), a representative optical enhancement technique, utilizes specialized bandpass filters to exclusively transmit 415 nm (blue) and 540 nm (green) wavelengths. These selected spectral bands correspond to hemoglobin’s absorption maxima while demonstrating limited tissue penetration depth, thereby providing high-contrast visualization of superficial mucosal microvasculature and fine morphological features [13]. The vessel plus surface (VS) classification system is a method to judge whether malignant lesions occur by evaluating the changes of mucosal microvessels and microstructure. NBI can help endoscopists diagnose through VS classification system by virtue of its advantages. Compared to WLE, NBI has been proven to have superior performance in identifying EGC [14–17]. However, due to reasons such as technological property rights and economic costs, endoscopists in many underdeveloped areas or small medical institutions are unable to use NBI to observe and detect lesions. Therefore, there is a current need to invent a technology that can convert WLE images into NBI images to help endoscopists better observe lesions and reduce the risk of missed diagnosis of EGC.
CycleGAN is an artificial intelligence (AI) technology that can perform image transformation [18], it has played an important role in the application of digestive diseases. Yuan et al. used CycleGAN to convert WLE images of colorectal polyps in dynamic videos into Virtual indigo carmine (VIC) images for observation [19]. A research team from Japan used CycleGAN to convert WLE images into VIC images for 3D reconstruction of the stomach [20]. The VIC images generated by Suzuki et al. using CycleGAN can help determine the subtypes of gastric tumors [21]. Although CycleGAN performs well in the field of image translation, the quality of the generated images is often limited, the details are not detailed enough, and may be accompanied by artifacts and pattern collapse issues [22].
In contrast, stable diffusion, as a generative model based on stepwise denoising reversion, can more accurately capture data distributions and generate images with high levels of detail and authenticity. Additionally, stable diffusion’s robust training process mitigates the instability commonly seen in adversarial models, particularly in complex medical image tasks, offering significant advantages. Currently, the stable diffusion model has been widely applied in various medical imaging tasks, including cross-modal image translation, image reconstruction, image generation, denoising, and anomaly detection [23].
To the best of our knowledge, there is currently no research on using stable diffusion for endoscopic image conversion. In this study, we utilized stable diffusion, a model with strong conversion capabilities, to generate virtual-narrow band imaging (Vir–NBI) images from WLE images. The aim is to enable more endoscopists, especially those without hardware support, to use virtual NBI images for detailed observation and timely detection of EGC during endoscopic examinations. Finally, we compared the efficacy of Vir–NBI images with WLE and NBI images in identifying lesion structures and diagnosing EGC.

2.
Methods
2.1.
Patients and materials
We retrospectively collected medical records and endoscopic images of 325 patients with gastric mucosal lesions who underwent endoscopic submucosal dissection (ESD) treatment at the First Affiliated Hospital of Soochow University from January 2016 to June 2024. According to the revised Vienna Classification of Gastrointestinal Epithelial Diseases [24], the pathological types of lesions are diagnosed as EGC in categories 3 (high-grade intraepithelial neoplasia), 4 (intramucosal carcinoma), and 5 (submucosal carcinoma), while categories 1 (negative for neoplasia) and 2 (low-grade intraepithelial neoplasia) are diagnosed as noncancerous lesions. All WLE and NBI images were captured by standard endoscopes (GIF-Q260J, GIF-HQ290, GIF-H290Z Olympus Corporation, Tokyo, Japan) and EVIS LUCERA ELITE endoscopic system (CV-290, Olympus Corporation).
This study adopts a retrospective design, and the use of research data has been submitted in writing to the Science and Technology Department of the First Affiliated Hospital of Soochow University, and has been approved by the Ethics Committee after review (Ethical Approval Number:2024-649).

2.2.
Dataset preprocessing and experimental design
A doctoral student excluded patients with poor image quality (bleeding, halo, blur, and defocusing) and those who did not simultaneously retain WLE and NBI images of the lesion site. The filtered WLE and NBI images were saved in BMP or JPEG format, uniformly cropped, with excess black background removed, retaining only the images within the endoscopic field of view, and then saved in PNG format. 273 NBI images from 218 patients were selected to fine-tune stable diffusion, and Vir–NBI images were generated from the WLE images of another 107 patients to validate the effectiveness of stable diffusion in the form of WLE, NBI, and Vir NBI image pairs (Figure 1).

2.3.
Generate Vir–NBI images through stable diffusion conversion
Stable diffusion, as a deep learning based generative model, is capable of generating high-quality images based on text prompts or other conditional inputs. In this study, we utilize stable diffusion to convert WLE images into NBI images in gastrointestinal endoscopy.
Our virtual NBI synthesis framework is built upon the stable diffusion model and consists of two main stages: LoRA-based fine-tuning and self-attention injection sampling. The framework is illustrated in Figure 2. In the fine-tuning stage, we adapt the U-Net backbone of the stable diffusion model using a small curated dataset of real NBI images, guided by the text prompt ‘narrow-band imaging, NBI’. The model is optimized by minimizing the mean squared error between the predicted noise and the actual Gaussian noise, enabling it to capture the visual appearance and modality-specific semantic patterns of NBI images.
In the subsequent sampling stage, we propose a self-attention injection mechanism to guide the generative process. Specifically, during DDIM-based sampling, the self-attention maps of the LoRA fine-tuned model are dynamically replaced with those from the original pre-trained stable diffusion model applied to the corresponding WLE image. This process starts at the first sampling timestep and is maintained throughout generation, allowing the model to preserve the semantic structure of the WLE input while accurately mapping it to the NBI modality.
All experiments were conducted using Python 3.8 and PyTorch 1.11.0, with acceleration provided by CUDA 11.3 on an NVIDIA RTX 4090D GPU with 24 GB VRAM. LoRA was applied to fine-tune the attention layers of the U-Net with a rank of 16. The model was trained for 100 epochs using the AdamW optimizer with a learning rate of 2 × 10−4 and weight decay of 0.01. During inference, DDIM inversion with T1 = 1000 steps is first used to recover the latent representation of the WLE image, followed by T2 = 50 guided sampling steps using the same text condition as in fine-tuning. On average, the generation of a single Vir–NBI image takes total 27.8 s, with 20.6 s for DDIM inversions and 7.2 s for sampling costs (Figure 2).

2.4.
Compared with CycleGAN
To quantitatively validate the superiority of the proposed stable diffusion-based method over traditional GAN-based approaches, we conducted a comparative study using CycleGAN as a representative baseline. We evaluated the quality of generated virtual NBI images using four metrics: Fréchet Inception Distance (FID), Learned Perceptual Image Patch Similarity (LPIPS), Artifact-aware FID (ArtFID), and Structural Similarity Index Measure (SSIM).
The results demonstrate that the stable diffusion model consistently outperforms CycleGAN across all metrics. For instance, our method achieved a lower FID (12.56 vs 13.47), indicating higher fidelity to real NBI distributions. Similarly, LPIPS and ArtFID scores were significantly reduced (LPIPS: 0.32 vs 0.39; ArtFID: 17.88 vs 20.09), reflecting better perceptual similarity and fewer visual artifacts. Additionally, our method obtained a higher SSIM (0.26 vs 0.22), further confirming its ability to preserve structural details.
These findings support that the stable diffusion framework effectively alleviates common problems associated with GANs, such as mode collapse and artifact introduction, leading to the generation of more realistic and semantically consistent virtual NBI images.

2.5.
Outcome measurements
2.5.1.
Observation quality of surface structure, surface blood vessels, and lesion margins in WLE, NBI, and Vir–NBI images
Blind evaluation of WLE, NBI, and Vir–NBI images in the validation set was conducted by two experienced endoscopists with over 10000 cases of endoscopic examination experience. The evaluation focused on lesion surface structure, surface blood vessels, and lesion edges, with rating divided into three levels: poor, moderate, and good. For surface glandular ducts and surface blood vessels, ‘poor’ refers to difficult to identify morphology; ‘moderate’ refers to a form that is easy to observe but difficult to recognize, while ‘good’ refers to a form that is easy to recognize. For lesion margins, poor means the lesion is difficult to distinguish from the surrounding normal tissue, moderate means the lesion can be identified as an area and the edge can be inferred, and good means the edge between the lesion and the normal tissue is clear and distinguishable. In case of disagreement, the two endoscopists will discuss further until a consensus is reached.

2.5.2.
The accuracy of WLE, NBI, and Vir–NBI image diagnosis for EGC
Detailed observations of WLE, NBI, and virtual NBI images in the validation set were conducted by three junior endoscopists (with less than 5 years of endoscopic examination experience), three intermediate endoscopists (with 5–10 years of endoscopic examination experience), and three senior endoscopists (with more than 10 years of endoscopic examination experience), respectively. Endoscopists within the same seniority independently observed WLE images, WLI+NBI images, and WLI+Vir–NBI images, and finally determined whether the lesion was malignant (EGC) through consensus negotiation. The diagnostic accuracy for EGC of different observation methods was then compared across endoscopists with varying years of experience.

2.5.3.
The area concordance rate and successful whole-lesion diagnosis
Use Labelme software to randomly display all 111 paired images on a computer monitor, including 111 WLE images, 111 NBI images, and 111 Vir–NBI images. Three junior endoscopists, three intermediate endoscopists, and three senior endoscopists delineated the lesion boundaries on each WLE and Vir–NBI image in the validation set. Then, compare the lesion areas designated by endoscopists of different seniority with the pathological gold standard after ESD. The area concordance rate is defined as the ratio of the overlapping area between the lesion range defined by the endoscopist and the lesion range under the pathological gold standard to the collection area. The calculation formula is c/(a + b–c)×100%. The definition of successful whole-lesion diagnosis is defined as the lesion scope delineated by the endoscopists fully covering the lesion range under the pathological gold standard (Figure 3).

2.6.
Statistical analysis
The quality evaluation of WLE, NBI, and Vir–NBI images was compared using Friedman test and Steel–Dwass test. The McNemar test is used to compare the accuracy of diagnosing EGC by observing WLE images, WLE + NBI images, and WLE+Vir–NBI images. Mann–Whitney U test is used to analyze the area concordance rate. Pearson chi-square test is used to analyze the successful diagnosis rate of the entire lesion. p < 0.05 is considered statistically significant. All analyses were conducted using JMP version 17.0 (SAS Institute, Cary, NC, USA).

3.
Results
3.1.
Patient characteristics
In the training sets, the average age of patients was 63.00 ± 8.96 years. There were 148 males (67.89%) and 70 females (32.11%). In the validation sets, the average age of patients was 65.83 ± 9.41 years. There were 72 males (67.29%) and 35 females (32.71%). Detailed information on the location, pathology, invasion depth, differentiation type, etc. of the lesion is shown in Table 1.

3.2.
Comparison of WLE, NBI, Vir–NBI image quality evaluation
Two endoscopy experts evaluated the image quality of WLE, NBI, and Vir–NBI based on consultation (Table 2; Figure 4). In terms of surface structure evaluation, 8 (7.2%) WLE images, 44 (39.6%) NBI images, and 51 (45.9%) Vir–NBI images were rated as good. Compared with WLE images, both NBI images (Steel–Dwaas test, p < 0.001) and Vir–NBI images (Steel–Dwaas test, p < 0.001) showed better observation results. In terms of evaluating blood vessels, 11 (9.9%) WLE images, 39 (35.1%) NBI images, and 31 (27.9%) Vir–NBI images were rated as good. Compared with WLE images, both NBI images (Steel–Dwaas test, p < 0.001) and Vir–NBI images (Steel–Dwaas test, p < 0.001) showed better observation results. In terms of evaluating lesion margins, 11 (9.9%) WLE images, 39 (35.1%) NBI images, and 33 (29.7%) Vir–NBI images were rated as good. Compared with WLE images, both NBI images (Steel–Dwaas test, p < 0.001) and Vir–NBI images (Steel–Dwaas test, p < 0.001) showed better results.

3.3.
Comparison of diagnostic ability of endoscopists observing different modes of endoscopic images for EGC
The diagnostic results of endoscopists with varying years of experience can be seen in Table 3 and Figure 5. For junior endoscopists, the accuracy of diagnosing EGC by observing WLE images alone was 61.26%, while the accuracy of diagnosing EGC with WLE + NBI images was 79.28%, and the accuracy of diagnosing EGC with WLE + Vir–NBI images was 81.08%. The diagnostic accuracy of simultaneous observation of WLE + NBI and WLE + Vir – NBI images was significantly higher than that of observing WLE images alone (p < 0.001, p < 0.001), while there was no significant difference between the simultaneous observation of WLE + NBI images and WLE + Vir – NBI images. For intermediate endoscopists, the accuracy of diagnosing EGC by observing WLE images alone was 72.07%, while the accuracy of diagnosing EGC with WLE + NBI images was 86.49%, and the accuracy with WLE + Vir – NBI images was 84.68%. Similar to the junior endoscopists, the diagnostic accuracy of simultaneous observation of WLE + NBI images and WLE + Vir–NBI images was significantly higher than that of observing WLE images alone (p < 0.05, p < 0.05), with no significant difference between simultaneous observation of WLE+NBI images and WLE + Vir–NBI images. For senior endoscopists, the accuracy of diagnosing EGC by observing WLE images alone was 80.18%, while the accuracy of diagnosing EGC using WLE+NBI images is 95.50%, and the accuracy with WLE + Vir–NBI images is 92.79%. The diagnostic accuracy of simultaneous observation of WLE+NBI images and WLE + Vir – NBI images was significantly higher than that of observing WLE images alone (p < 0.001, p < 0.05). There was no significant difference in the accuracy of diagnosing EGC between observing WLE images alone and observing WLE + Vir–NBI images simultaneously.

3.4.
Comparison of the area concordance rate between WLE and Vir–NBI images
Overall, Vir–NBI images showed a significantly higher area concordance rate than WLE images (43.85% vs 39.32%, p < 0.001) (Table 4; Figure 6). In the subgroup analysis, the area concordance rate of Vir–NBI images with specimen size ≤3.5 cm and >3.5 cm was significantly higher than that of WLE images (42.46% vs. 39.25%, p = 0.013) (45.57% vs 39.40%, p < 0.001). The area concordance rate of elevated, flat and depressed type lesions in Vir–NBI images was significantly higher than that in WLE images (51.37% vs 43.58%, p < 0.001) (44.31% vs 39.57%, p = 0.030) (38.22% vs 35.25%, p = 0.035). The area concordance rate of lesions located in the middle 1/3 and lower 1/3 of Vir–NBI images was significantly higher than that in WLE images (43.38% vs 38.82%, p = 0.002) (45.42% vs 40.63%, p = 0.002). The area concordance rate of differentiated EGC in Vir–NBI images was significantly higher than that in WLE images (42.68% vs 38.41%, p < 0.001). The area concordance rate of epithelial neoplasia or mucosal layer and submucosal layer EGC in Vir–NBI images was significantly higher than that in WLE images (42.58% vs 38.57%, p < 0.001) (44.15% vs 37.19%, p = 0.041). The area concordance rate between junior and senior endoscopists in Vir–NBI images was significantly higher than that in WLE images (41.51% vs 37.39%, p < 0.001) (46.19% vs 41.24%, p < 0.001).

3.5.
Comparison of successful whole lesion diagnosis between WLE and Vir–NBI images
Overall, Vir–NBI images showed a significantly higher successful whole lesion diagnosis than WLE images (45.33% vs 32.87%, p < 0.001) (Table 5; Figure 7). In subgroup analysis, successful whole lesion diagnosis with specimen size≤ 3.5 cm and >3.5 cm in Vir–NBI images was significantly higher than that in WLE images (45.40% vs 36.49%, p = 0.011) (45.24% vs 28.57%, p < 0.001). Successful whole lesion diagnosis of elevated, depressed and mixed type lesions in Vir–NBI images was significantly higher than that in WLE images (38.17% vs 24.73%, p = 0.004) (53.24% vs 43.06%, p = 0.027) (45.14% vs 28.47%, p = 0.003). Successful whole lesion diagnosis of lesions located in the middle 1/3 and lower 1/3 of Vir–NBI images was significantly higher than that of WLE images (54.44% vs 32.59%, p < 0.001) (40.99% vs 31.08%, p = 0.030). Successful whole lesion diagnosis of differentiated and undifferentiated EGC in Vir–NBI images was significantly higher than that in WLE images (47.26% vs 32.65%, p < 0.001) (56.67% vs 30.00%, p = 0.003). Successful whole lesion diagnosis of epithelial neoplasia or mucosal layer and submucosal layer EGC in Vir–NBI images was significantly higher than that in WLE images (45.43% vs 31.51%, p < 0.001) (70.00% vs 38.33%, p < 0.001). Successful whole lesion diagnosis between junior and senior endoscopists in Vir–NBI images was significantly higher than that in WLE images (42.37% vs 29.28%, p < 0.001) (48.29% vs 36.45%, p = 0.004).

4.
Discussion
To our knowledge, this is the first study to generate Vir–NBI images based on stable diffusion for the diagnosis of EGC. The results demonstrate that Vir–NBI images significantly outperform WLE images in identifying surface structures and lesion edges, closely resembling real NBI images. Furthermore, compared to observing WLE images alone, junior and intermediate endoscopists showed a significant improvement in EGC diagnostic accuracy when simultaneously observing WLE + Vir–NBI images. Finally, Vir–NBI images not only offer superior delineation of lesion boundaries compared to WLE images but also assist endoscopists in identifying the entire lesion more clearly.
In recent years, stable diffusion has made significant strides in multiple fields of medical image processing. It has demonstrated notable progresses in tasks such as image translation, image reconstruction, image classification, and segmentation, showcasing strong potential in these areas [23]. Lyu et al. [25] applied stable to solve the translation problem from MRI to CT, and the results showed that stable diffusion outperform CycleGAN based methods in terms of image quality and structural similarity. Another study proposed a medical image classification method based on stable diffusion, which proved effective in assessing the grading of placental maturity, skin lesions and diabetes retinopathy [26]. These studies indicate that stable diffusion not only effectively handles complex features in medical images but also offers unique advantages in tasks such as cross modal data, unsupervised learning, and image generation. Its flexibility and powerful generation capability make stable diffusion widely applicable in medical image analysis, especially in situations with insufficient data or scarce labels. It can effectively improve image quality, enhance data representation, and provide strong support for precision medicine.
Although stable diffusion has demonstrated advantages in image generation and transformation tasks, it still has some technical limitations. For example, the large parameter count of stable diffusion requires a substantial memory to store weights and intermediate results. This high demand may limit the application of stable diffusion on devices with restricted resources. Despite advancements in hardware, the computational and storage overhead of stable diffusion remains a challenge for some embedded devices or low budget computing environments. In addition, due to its own characteristics, the speed of image conversion of stable diffusion is limited. In this study, the average time of converting WLE images into Vir–NBI images was 27.8s, and endoscopists could not get the converted images immediately. However, considering that the conventional endoscopy time often reached more than 5 min, stable diffusion could still ensure that endoscopists could complete the observation and diagnosis of Vir–NBI images during the examination.
In this study, we generated Vir–NBI images from WLE images based on stable diffusion, and the generated Vir–NBI images closely resemble real NBI images to the naked eye. According to the evaluation and assessment of endoscopists, Vir–NBI images provided better lesions visibility and more accurate identification of EGC compared to WLE images.
The area concordance rate of WLE and Vir–NBI images in the lesion area reached 39.32% and 43.85%, respectively. Based on actual clinical experience, both values seem slightly lower, which may be due to the relatively limited lesion information presented in static images, leading to insufficient ability of endoscopists in evaluation. A study from Japan [14]showed that the area concordance rate of WLE and NBI images in EGC were 37.2% and 43.1%, respectively. Additionally, a study by Suzhuki et al. [21]showed that the area concordance rate of WLE and VIC images in gastric tumors were 48.5% and 44.1%, respectively. The differences between this study and the above research results are small, suggesting that using WLE images as a control for Vir–NBI images in this study is reasonable.
This study has several advantages. First, through Stable Diffusion, WLE images can be easily and directly converted into Vir–NBI images that closely resemble real NBI images, enabling more endoscopists in medical institutions to observe lesions clearly on high-quality images. Second, Vir–NBI images effectively support the accurate diagnosis of EGC, particularly for less experienced endoscopists. Third, Vir–NBI images assist endoscopists in fully identifying the entire lesion, thereby achieving complete resection.
However, this study also has some limitations. First, we used static image pairs to verify the effect, but in clinical practice, endoscopists typically observe lesions in dynamic examination videos. Our goal is to collect more endoscopic video data in the future to further evaluate the effectiveness of stable diffusion conversion. Second, all endoscopic images collected in this study were obtained from Olympus 260 or 290 series gastroscopy systems, with no endoscopic images from other series brands such as Fuji. In the future, we aim to expand our dataset to include images from other manufacturers and develop an AI-powered virtual electronic staining endoscopic system that is not limited by suppliers. Third, although this study shows that the Vir–NBI images generated by stable diffusion perform well in diagnosing EGC, it is important to note that Vir–NBI images are not real NBI images. As a new imaging modality, endoscopists will need a certain learning period to become proficient in its application.
In summary, the Vir–NBI images generated by stable diffusion are helpful for the observation and diagnosis of EGC. In the future, we will continue to optimizing the model structure, reducing video memory usage, and enhancing computational efficiency, with the goal of enabling Vir–NBI image conversion in dynamic inspection videos.

Supplementary Material

Supplemental Material