Diagnostic accuracy of rapid on-site evaluation (ROSE) during robotic bronchoscopy.

Introduction
The incidence of lung nodules found on chest imaging is increasing. This is not only due to increased adoption of lung cancer screening programs but also due to an increase in incidentally discovered lung nodules (1,2). While many of these nodules can be serially followed for radiographic changes, a significant number will require biopsy for definitive tissue diagnosis (3). Lung nodules can be biopsied using a computed tomography (CT)-guided transthoracic approach or via a bronchoscopic transbronchial route. Transthoracic biopsies have a diagnostic yield of approximately 80–90%; however, they are also associated with a higher pneumothorax rate of roughly 30% (4,5). On the other hand, conventional bronchoscopic transbronchial biopsies (TBBx) were associated with a relatively lower diagnostic yield of 30–60%, but also a lower pneumothorax rate of 1–2% (6,7). Over the last few years, multiple robotic bronchoscopy (RB) platforms have been introduced, which appear to combine the best of both worlds, offering a high diagnostic yield with a low complication rate (8,9). RB has been associated with a diagnostic yield of 84.3% with a pneumothorax rate of 2.3% (10).
There is significant variation in the exact procedural protocol of RB between different institutions, and even between different proceduralists at the same institution: shape-sensing vs. electromagnetic navigational RB platforms, use of ancillary imaging modalities such as radial-endobronchial ultrasound (R-EBUS) and cone beam CT (CBCT), use of rapid on-site evaluation (ROSE), etc. ROSE involves microscopic assessment of bronchoscopic samples by a cytologist or a cytology technician, typically within the procedure suite. This qualitative and quantitative sample assessment is relayed to the proceduralist in real time. Qualitative assessment refers to whether the cytologist detects abnormalities that suggest a malignant or benign diagnosis. If the qualitative assessment is negative for malignancy or a specific benign diagnosis, the proceduralist may not be sampling the actual lesion and may need to reposition the catheter. On the other hand, quantitative assessment reflects the quantity of lesional material. This informs the proceduralist how many more passes they may have to obtain. For example, the presence of a few malignant cells may indicate that the proceduralist is sampling the correct site but needs to take additional passes to ensure adequate material for final pathology and molecular testing. ROSE can be performed on cytology specimens obtained by fine needle aspiration (FNA) or by brushing. Alternatively, touch imprint cytology (TIC) can be evaluated by ROSE. TIC involves scraping a tissue specimen obtained using forceps or cryobiopsies onto a glass slide, followed by ROSE.
There is debate on whether the incorporation of ROSE increases the diagnostic yield of bronchoscopic biopsies (11-13). However, it remains widely used because it may help the proceduralist determine which biopsy tools to use, how many samples to take, and which sites to sample. For example, if the ROSE demonstrates a cellular specimen with multiple groups of malignant cells, the proceduralist may decide not to deploy additional biopsy tools to save time and the complications associated with additional sampling. Despite these benefits, ROSE can be associated with a need for additional resources and investment, potentially increased intra-procedural times, and false-positive and false-negative results (14). To make an informed judgment whether to use ROSE in a particular case, the proceduralist needs to have a good understanding of the concordance between ROSE and final pathology results.
Prior studies have evaluated the concordance of ROSE and final pathology results during electromagnetic navigational bronchoscopy, R-EBUS-guided peripheral bronchoscopy, and linear EBUS-guided lymph node sampling; however, data on ROSE concordance for RB is very limited (15). We hypothesize that, in an extensive study, the concordance between ROSE and final pathology for RB might be lower than that observed for other modalities, such as linear EBUS. In linear EBUS, the node being sampled is typically adjacent to the airway, so the needle encounters very little connective tissue before entering the node. Similarly, with real-time ultrasound guidance, the blood vessels within the node can be avoided. Both factors result in a ‘cleaner’ specimen with minimal contamination by bronchial cells and red cells, making them easily amenable to ROSE. On the other hand, with RB, the lesion can be up to 3 cm from the RB catheter; therefore, the sampling tools must take a ‘cross-country’ path through the lung parenchyma to reach the lesion. This may result in a specimen contaminated with bronchial cells and blood, making ROSE interpretation more difficult. We present this article in accordance with the STARD reporting checklist (available at https://jtd.amegroups.com/article/view/10.21037/jtd-2025-1966/rc).

Methods
We performed a retrospective review of all consecutive RB cases performed at the Anschutz Medical Campus, University of Colorado, Denver, from May 2021 to December 2024. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the Institutional Review Board (IRB) of the University of Colorado [No. 23-0895(23-122)], and individual consent for this retrospective analysis was waived.

Objectives
Our primary objective was to assess the concordance between cytology-ROSE (via FNA) and TIC-ROSE (via forceps) and the final cytology and pathology results, respectively.
Our secondary objectives were: (I) assessing the concordance between ROSE and final cytopathology results for EBUS-guided mediastinal lymph node sampling; (II) assessing the diagnostic yield of RB for peripheral pulmonary lesions. Cases were considered diagnostic if: (I) a specific malignant or benign diagnosis (e.g., granulomatous inflammation, organizing pneumonia) was obtained; (II) non-specific benign diagnoses were considered diagnostic if the follow-up imaging showed a decrease in the size of the lesion or stability for at least 12 months, in an appropriate clinical context. Non-specific benign results where suspicion for cancer was high and further diagnostic tests were performed were considered non-diagnostic (ND). All cases with a specific malignant diagnosis were considered diagnostic, irrespective of whether enough material was available for further molecular testing.

Statistical analysis
All ROSE and final cyto-pathology results were divided into three categories: benign, malignant, and ND; 3×3 tables were constructed to assess the concordance between ROSE and final cytopathology results. Concordance was calculated as the number of cases with concordant results divided by the total number of cases. The Pearson Chi-squared (χ2) test statistic was calculated to determine the significance of the association between the ROSE and final cytopathology results. We decided to use the Pearson Chi-squared (χ2) test statistic, since it’s a robust, nonparametric test most appropriate for assessing the association between nominal variables. Furthermore, Cohen’s kappa (κ) was calculated as a measure of inter-rater reliability between the ROSE and final cytopathology results.

Procedures
Procedures were performed in the endoscopy suite by two proceduralists. RB was performed using the Ion System (Intuitive Surgical, Sunnyvale, CA, USA). A pre-procedural CT scan was used for planning purposes. The Ion PlanPoint software was used. The desired target was marked, and a virtual pathway to it was created. Registration was performed by driving the robotic catheter into the bilateral upper and lower lobes. After registration, we navigated to the target. Adequate navigation was confirmed using R-EBUS (including both concentric and eccentric signals) and fluoroscopy. A 21-gauge needle was used to perform FNA or transbronchial needle aspiration (TBNA), and flexible forceps were used for TBBx. TBNA passes were smeared on a slide, and ROSE was performed. Similarly, the tissue sample obtained with forceps was also smeared onto slides for TIC-ROSE. Further tissue samples were collected for histopathology examination in formalin.
ROSE workflow:
❖ ROSE involved a team approach: cytotechnologist + cytopathologist + trainees.

❖ Cytotechnologists prepared and stained all slides.

❖ A cytopathologist performed an on-site evaluation by reviewing the slides under the microscope in the procedure room.

❖ Generally, 4–6 TBNA passes and forceps biopsies were obtained for each lesion. At least 2–3 of these passes/biopsies were assessed by ROSE.

❖ If the cytopathologist found sufficient evidence to make a call for a benign or a malignant disorder, on-site evaluation was stopped, and further passes were collected in formalin. If, despite multiple smears being evaluated by ROSE, the cytopathologist could not make a benign or malignant call, ROSE was considered ND.

Slide preparation for each FNA/TIC pass:
❖ Two direct smears were made:
One air-dried smear;

One alcohol-fixed smear.

❖ The remaining needle material was squirted into formalin for the cell block.

Staining and evaluation:
❖ The air-dried slide was stained with Diff-Quik stain;

❖ Alcohol-fixed slides were sent to the cytology laboratory for Papanicolaou staining;

❖ The formalin-fixed cell block was processed in the cytology laboratory.

The final cytology and histopathology results served as the reference standards against which we evaluated the accuracy of our ROSE and TIC-ROSE. Samples were obtained under fluoroscopic guidance. RB cases for which on-site evaluation had not been performed (most due to lack of availability) were excluded from the analysis.

Results
One hundred and fifty-six patients were included in the study. The mean age of the patients was 68.1 [standard deviation (SD) 10.9] years; 59% of patients were female, while 41% were male; 137 patients undergoing RB also had ROSE performed and were therefore assessed to determine the concordance between ROSE and final cytopathology results. While ROSE was requested for all cases, it was not available in 19 cases due to scheduling conflicts. The mean lesion size was 18.8 mm; 28.2%, 35.9%, and 35.5% of lesions were located in the medial 1/3rd, middle 1/3rd, and lateral 1/3rd of the lung, respectively. Concentric, eccentric, and no signals were obtained with the R-EBUS in 50%, 45.4% and 4.6% of the cases.
Table 1 shows the relationship between ROSE and final cytopathology results for these 137 patients. Overall concordance was 88%. Concordance for benign, malignant, and ND ROSE results was 100%, 94% and 70%, respectively. Pearson χ2-value was 174 (P<0.001) (Table 2). Cohen’s kappa was 0.709 [95% confidence interval (CI): 0.62–0.80], indicating substantial strength of agreement. The sensitivity, specificity, positive predictive value and negative predictive value of ROSE for cancer were: 91.7%, 93.9%, 94.4% and 88.3%, respectively.
We also assessed the concordance between the FNA-ROSE and final cytology (Table 3) and TIC-ROSE and final pathology (Table 4). The concordance between FNA-ROSE and final cytology was 83%. Cohen’s kappa was 0.721 (95% CI: 0.62–0.83), indicating substantial strength of agreement. The concordance between TIC-ROSE and final pathology was 81%. Cohen’s kappa was 0.70 (95% CI: 0.59–0.85), again indicating substantial strength of agreement.
Sixty-eight patients also underwent linear EBUS-guided lymph node sampling. The decision to perform linear EBUS-guided lymph node examination was based on the pretest probability of lymph node involvement per the established guidelines (16,17). Table 5 shows the relationship between EBUS-ROSE and final cytology results. The median number of lymph node stations sampled during EBUS was 3. The concordance between EBUS-ROSE and final cytology was 80.1%. All patients underwent RB first, followed by EBUS-guided sampling of mediastinal lymph nodes.
The overall diagnostic yield for RB in our study was 75% (117/156). Of the 117 diagnostic cases, 18 cases had nonspecific benign results and required further follow-up for confirmation. The outcome of these cases was as follows:
❖ Resolved (n=8);

❖ Decreased in size (n=7);

❖ Stable in size for 2 years (n=2);

❖ Stable in size for 1 year (n=1).

The overall complication rate was low. Pneumothoraces were observed in 3% of the cases. One case of significant bleeding was noted, requiring endobronchial placement.

Discussion
This is one of the first studies assessing the concordance between ROSE and final cytopathology results for RB. Our study demonstrates a high level of concordance between FNA-ROSE and final cytology, as well as TIC-ROSE and final pathology, at 83% and 81%, respectively. The concordance was even higher at 88% when the two ROSE modalities were combined. These results are comparable to those previously reported by Vu et al. (15). The results demonstrate a higher degree of concordance than previously demonstrated for navigational bronchoscopy at 77.3% (12). The difference could be due to variations in the number of passes performed for ROSE in the two studies or to differences in local expertise. Muto et al. previously reported the concordance between the TIC-ROSE and final pathology for cryobiopsies obtained via R-EBUS and guide sheath at 76.2% (18). This is comparable to our results for TIC-ROSE. Caupena et al. and Nakajima et al. have previously reported the concordance between EBUS-ROSE and final cytology for mediastinal lymph nodes at 96.1% and 98.2%, respectively (11,19). These rates are higher than our rate of 80.1%.
Although our study was not designed to determine the impact of the ROSE on the diagnostic yield of RB, our results have significant implications. The high level of concordance between the ROSE and final cytopathology results provides the proceduralist performing RB with the confidence to make important intraoperative decisions guided by ROSE. Firstly, in cases with multiple similar appearing pulmonary lesions, if a diagnosis is obtained on ROSE for one of the lesions, it may obviate the need to sample additional lesions. Secondly, one of the techniques advocated for sampling pulmonary lesions via RB is “cloud biopsy” (20). This entails sampling at multiple adjacent spots in and around the virtual target, in an effort to bridge the gap between “tool-in-lesion” and “lesion-in-tool” scenarios. This is especially important in cases of heterogeneous lesions, where certain parts may be necrotic and therefore sampling multiple spots in the lesion would be expected to increase the diagnostic yield. If ROSE is diagnostic, it may allow the bronchoscopist to skip sampling at adjacent spots and potentially save time and risk of additional complications. Thirdly, a bronchoscopist can use the ROSE results to determine the need for additional biopsy modalities. Studies have shown that the use of multiple biopsy tools, such as needles, brushes, forceps, and cryoprobes, has an additive effect on the diagnostic yield. Oberg et al. showed that the use of a 1.1 mm cryoprobe for transbronchial cryobiopsies during RB resulted in an additional 18% cases being diagnosed that would have otherwise been missed (21). The use of these additional biopsy modalities may be associated with increased procedural time, costs, and complications. Therefore, ROSE can guide the bronchoscopist on whether these additional biopsy modalities are necessary. Fourthly, in cases of ND ROSE results, the bronchoscopist can adjust the RB catheter to explore the vicinity of the “virtual target”, which may, in turn, increase the chances of obtaining a diagnosis. Nishiyama et al. demonstrated that when ROSE was sequentially repeated on brushings, initially negative ROSE results became positive in 79.5% of cases (22). Additionally, the proportion of specimens with high tumor cell counts increased from 42.1% to 69.0%. Fifthly, multiple centers are now using intra-operative CBCTs for real-time three-dimensional (3D) imaging, to improve the success of navigation and diagnostic yield for RB (23,24). However, concerns exist regarding the additional time required for CBCT spin, the risk of breath-hold maneuvers during CBCT spin, and the potential radiation exposure to both the patient and the bronchoscopy team. The results of ROSE may allow the bronchoscopist to judiciously use CBCT only when initial ROSE results are negative.
Recently published reports indicate growing interest in single anesthesia diagnostic and treatment procedures, whereby patients undergo bronchoscopic biopsy, lesion marking, and resection in the same setting (25,26). During these cases, if ROSE confirms the presence of malignancy on bronchoscopic biopsies, surgeons proceed with resection of the lesion at the same time. The success of this strategy greatly depends on the accuracy and concordance of ROSE with the final pathology. Our results, indicating a high degree of ROSE concordance, may have implications for the adoption of single anesthesia diagnostic and resection protocols.
When performing navigational bronchoscopy or RB and linear EBUS in the same setting, the order of the two procedures has been a matter of debate (27). In fact, a prospective clinical trial is currently underway to answer this question (28). We prefer to perform RB first because our goal is to navigate to the peripheral lesion as soon as possible after intubation. The longer the patient has been on mechanical ventilation, the greater the atelectasis, which in turn increases the CT-body divergence (the discrepancy between the pre-procedural CT scan on which the RB planning is based and the real-time patient anatomy intra-procedurally). CT-body divergence can make it harder to navigate to the intended target and thereby may reduce the diagnostic yield of RB (29-31). On the other hand, the potential benefit of performing EBUS first is that, in the event metastatic disease is established in the nodes, it may obviate the need for RB-assisted sampling of the peripheral nodule, thereby saving time, costs, and potential complications. In our study, 11 patients were noted to have malignant nodal disease on ROSE during EBUS. However, on the final cytology, 4 of these cases were determined to be false positives. This is a significant false-positive rate. Had we performed the EBUS first, we might have stopped the case when malignancy was detected on the EBUS ROSE. When the final cytology results were released, we would have had to perform a separate procedure to sample the peripheral lesion.
Our diagnostic yield for RB at 75% is comparable to multiple prior studies (32-34). While there is debate on how the diagnostic yield should be defined, our definition of diagnostic yield is pragmatic. Gonzalez et al. have advocated for a strict diagnostic criterion, whereby only the lesions with a specific benign diagnosis should be considered diagnostic, while non-specific benign results should be considered ND even when the lesion remains stable or decreases in size on follow-up (35). The issue with this strategy is that for a lot of benign pathologies, such as partially treated infections and post-infectious inflammatory changes, it is incredibly difficult to obtain a specific diagnosis based on histopathology and microbiology. Most of these lesions will resolve on follow-up and pose no threat to the patients; however, labelling them as ND would unnecessarily lead to repeat biopsy procedures, anxiety, and costs. At the same time, most of the studies assessing the diagnostic yield of CT-guided transthoracic biopsies didn’t use the proposed strict definition either. Therefore, using a strict definition for RB can mislead the readers when comparing the two modalities. Consequently, we believe that in the appropriate clinical context, a “non-specific” benign result should be considered diagnostic if appropriate radiographic follow-up reveals stability or resolution. Furthermore, the majority of lesions that initially showed “non-specific” benign results and were considered diagnostic only after follow-up imaging either resolved or decreased in size on subsequent imaging. Therefore, the chances of false negative biopsies in these cases are very low. If we applied the strict diagnostic yield criteria as proposed by Gonzalez et al., our diagnostic yield drops to 63.5%.
Our study has several limitations. Firstly, being retrospective, it is not possible to control for various biases. Secondly, although the standard operating procedure for our on-site cytology team, comprising technicians and cytopathologists, is the same for all technicians and cytopathologists, the number of FNA and biopsy passes assessed for ROSE varied and was generally dependent on the pathologist performing ROSE. Thirdly, we do not perform transbronchial brushings or transbronchial cryo-biopsies during RB. At several centers, these modalities are routinely used for obtaining cytology and histopathology specimens and are also routinely assessed by ROSE. We are unable to say if our ROSE concordance results from FNA and forceps biopsies can be extrapolated to brushings and cryobiopsies. Fourthly, there could have been variation among different pathologists regarding whether the degree of atypia seen on ROSE met the threshold to be considered malignant or whether it was deemed ND. We did not employ a standardized lung cytopathology reporting system for ROSE, such as the World Health Organization (WHO) reporting system (36). Fifthly, although we have demonstrated concordance between ROSE and final cytopathology results, due to the single-arm study design, we cannot conclusively state whether ROSE increases the diagnostic yield. Sixthly, not all RB cases at our institution were done with ROSE. While ROSE was requested for all RB cases, its availability depended on the availability of pathology staff. This could have introduced selection bias. In 19 cases (12%), no ROSE was available. Typically, these cases were performed at the end of the day, after 5 PM, when ROSE staffing was limited. Seventhly, for cases with non-specific benign diagnoses, we used a 12-month follow-up to ensure stability. A more rigorous diagnostic yield definition might entail a follow-up for 24–36 months. However, considering that the vast majority of the non-specific benign lesions in our study either resolved or shrank, it’s unlikely that our diagnostic yield would have been significantly lower even if we had used a more extended follow-up period. Lastly, we did not perform a cost-effectiveness analysis. Other authors have reported on scenarios where ROSE would be cost-effective (14).

Conclusions
In conclusion, our study demonstrates a high degree of concordance between the ROSE and final cytopathology results for RB. Based on the results of our study, the proceduralist can rely on the ROSE results to make critical intra-procedural adjustments that may, in turn, have implications on the diagnostic yield, safety, duration, and costs associated with RB. Furthermore, well-designed prospective trials are needed to establish standardized protocols for performing ROSE during RB and to assess its impact on diagnostic yield in the era of rapidly advancing technologies, such as thin cryoprobes and advanced intraoperative imaging modalities.

Supplementary
The article’s supplementary files as