Guiding treatment decisions in early breast cancer: A model-based comparison of the OncotypeDX and MammaPrint tests.

1
Introduction
Clinical risk algorithms, such as Adjuvant! and PREDICT, use patient and tumor characteristics to help identify those most likely to benefit from treatment in early-stage breast cancer [[1], [2], [3], [4]]. When it is unclear whether treatment benefit outweighs the side-effects of chemotherapy, genomic testing can provide additional prognostic value beyond traditional clinical factors, enabling more personalized treatment decisions [5,6].
The MammaPrint and OncotypeDX are examples of genomic tests that have accumulated substantial evidence supporting their clinical validity [7,8]. The landmark MINDACT and TAILORx trials demonstrated that the MammaPrint and OncotypeDX, respectively, help identify patients with favorable prognoses beyond traditional clinical risk factors, for whom chemotherapy does not sufficiently improve survival outcomes[[9], [10], [11]]. Based on this evidence, recent guidelines (ASCO, ESMO, St. Gallen) approved both tests to guide the choice for adjuvant chemotherapy for patients whose clinical profiles (e.g. ER+, HER2-) do not provide a clear indication to start or withhold chemotherapy[[12], [13], [14]]. Although both tests have been approved, these guidelines advise using only one test per patient, without a clear preference for either.
Studies comparing MammaPrint and OncotypeDX (one using commercial versions [15], others in silico reconstructions [16,17]) illustrated mostly similar performance [7,18], leaving the questions if one test should be preferred over the other, insufficiently answered. While ideally a randomized trial would assess their comparative impact on treatment decision making [19], such trials are cost heavy and time consuming [20,21]. Alternatively, decision analytic modelling allows for a model-based assessment of how these genomic tests might influence treatment decisions and patient outcomes [22,23]. The reported results from the MINDACT and TAILORx trials provided sufficiently detailed information to build such a model-based comparison.
We aimed to compare the clinical usefulness of MammaPrint and OncotypeDX tests in guiding adjuvant chemotherapy decisions for early-stage breast cancer patients populations similar to the MINDACT and TAILORx trials. We evaluated treatment decision making with Net Benefit – a weighted difference of preventing events and the number of treatments given. We hereto extend a previously developed decision-analytic model that evaluated the MammaPrint [24], by incorporating OncotypeDX and comparing both tests within the contexts of both the MINDACT and TAILORx cohorts.

2
Methods
2.1
Decision analysis
We evaluated both the MammaPrint and OncotypeDX tests in both trial cohorts to avoid a potential (‘home-field’) advantage for one test. Two synthetic cohorts were generated: one resembling the cohort of patients included in the MINDACT trial, and another resembling the TAILORx cohort. Each synthetic cohort included 1 million unique patients. To compare potential patient outcomes with and without treatment, we cloned each patient, creating digital twins: one with treatment and one without. This resulted in a total sample size of N = 2,000,000 for each synthetic cohort.
To directly compare both genomic tests in both trial cohorts, we simulated three test scores for every twin: a clinical risk score (continuous), a MammaPrint score (high/low), and an OncotypeDX score (0–100, and dichotomized into low: 0–25 vs high: 26–100 [11]). The distribution of these test scores closely matched those observed in the trials. A continuous MammaPrint version was not included due to absence of information on its continuous score distribution. In the trials, clinical risk was estimated using Adjuvant!Online and subsequently classified as high or low clinical risk. We used a continuous clinical risk score for a more detailed clinical risk assessment as would be available in current tools such as PREDICT version 3.0 [4]. The trial cohorts differed in inclusion criteria. We therefore adjusted the genomic test scores when applying them to the other trial's cohort, as the TAILORx trial included more clinical low-risk patients than the MINDACT trial (70 % [11] vs. 50 % [10]). Second, we assumed similar prognostic performance of MammaPrint or OncotypeDX when applied to the other synthetic cohort. This assumption is supported by evidence from the RxPonder trial, which demonstrated prognostic value of OncotypeDX in node-positive disease [25], a group that was excluded in TAILORx but included in MINDACT.

2.2
Simulating the survival outcome
The outcome was defined as 10-year distant metastases or breast cancer-related death, excluding death from unknown or known other cause. To simulate these events, we used a survival model with clinical risk, MammaPrint, OncotypeDX, and chemotherapy as predictors.
The prognostic strength for both markers was taken from their respective trials. Chemotherapy effectiveness was estimated in the MINDACT trial, whereas the TAILORx trial could not estimate treatment effectiveness for the high risk group (score>26), because all patients received chemotherapy by design. The treatment estimate from MINDACT closely aligned with the value used in the PREDICT algorithm and was therefore considered appropriate for use in the synthetic TAILORx cohort. While some studies suggest an interaction effect between the OncotypeDX (high risk scores) and chemotherapy effectiveness [11,26,27], conclusive evidence and precise estimates for this interaction are lacking [7,11,25,28]. Because the TAILORx trial did not provide an estimate for this interaction [11,29], we could not include an interaction term and accordingly assumed no predictive effect of OncotypeDX on treatment benefit. However, the effect of using different estimates for chemotherapy effectiveness are explored in the sensitivity analysis. More information on the trials, assumptions and data generating process can be found in the supplementary materials.

2.3
Treatment decision-making strategies
Chemotherapy decisions were based on estimated individual treatment benefit, defined as the absolute 10-year risk (of distant metastases or breast cancer-related death) reduction by giving chemotherapy. To allocate treatment, we set a treatment threshold, which reflects the minimum treatment benefit required for chemotherapy to be recommended. We realize that recent guidelines refrain from setting a specific threshold and emphasize the importance of discussing estimated treatment benefit with individual patients [13]. However, to evaluate treatment decision-making strategies, we need a specific threshold [30]. We used 5 % risk reduction as a default treatment threshold as suggested clinically relevant in the MINDACT trial [10], with alternatively 3 % in the appendix. When visualizing the clinical usefulness of the genomic tests, we presented a wide range of thresholds (0–20 %) to reflect varying patient preferences [31]. For the default 5 % treatment threshold, we assumed that all patients whose individual treatment benefit exceeded 5 % received chemotherapy. Given the treatment indication, we selected the appropriate twin from each pair (with or without chemotherapy) to count its outcome.
We focused on four treatment decision-making strategies. The clinical reference strategy was informed solely by clinical characteristics, reflecting decision-making without any genomic testing. Three strategies integrated clinical plus genomic information from either the MammaPrint, dichotomized OncotypeDX, or the continuous OncotypeDX. We explored how integrating genomic tests changed treatment decisions through better risk prediction. For some patients, treatment benefit based on their clinical risk scores was so low or high that performing a genomic test could not change the treatment decision. Therefore, genomic testing was only considered if the treatment decision based on clinical risk assessment could be reclassified by the genomic test result (Fig. 1). The four different decision-making strategies resulted in four indications to treat or not treat an individual patient.

2.4
Performance evaluation
To evaluate the treatment decision making strategies, we used Net Benefit as a summary measure [32,33]. As genomic tests are used to guide treatment decisions, we evaluated them on their intended use; how they affect treatment allocation and patient outcomes on the population level. Treatment benefit is defined as the number of prevented events by giving treatment, while treatment harm refers to the general burden of treatment (costs, risks of side effects) and is quantified by the number of treatments given [32]. However, to subtract the treatment harms from the benefit, a weight is needed [32].
In clinical practice, the harm of an event is considered greater than the harm of treatment. To reflect this in net benefit, a weight w is introduced that represents the importance of preventing one event relative to giving one treatment [32]. This weight w refers to the question: 'how many treatments am I willing to give to prevent one event?’, which coincides with the treatment threshold [33]. For example, if chemotherapy should reduce 10-year risk by at least 5 %, that implies accepting treating 100 patients to prevent 5 events, or treating 20 people to prevent one event. Therefore, by multiplying the number of treatments by 5 % (0.05), the weighted harms can be subtracted from the benefit: Net Benefit = Number of prevented events – number of treatment given ∗w, presented per 1000 patients. A higher Net Benefit indicates a more favorable balance between preventing events and the number of chemotherapy given. The clinical usefulness of a genomic test is the improvement in net benefit relative to decision making using only clinical characteristics.

2.5
Sensitivity analyses
To assess the robustness of our results, we varied the inputs of our decision analysis. We varied the quality of the clinical reference model and chemotherapy effectiveness over plausible ranges. To assess if using one test is significantly better than using another, we assessed the uncertainty for the difference in Net Benefit between using the MammaPrint and continuous OncotypeDX by drawing subsamples with identical sample size as the MINDACT (n = 6693) and TAILORx trial (n = 10,253). All analyses were conducted using R version 4.2.1 [34], the R-code is available in the Supplementary Material.

3
Results
3.1
Decision analysis for MINDACT context
In the synthetic MINDACT cohort, 110 out of 1000 patients were expected to experience distant metastases or breast cancer related death within 10 years if no treatment was given. In contrast, treating all 1000 patients with chemotherapy (HR = 0.64) would reduce the number of expected events to 74, preventing 36 events (Table 1). Following the MINDACT strategy, we would treat 270 patients at high clinical and high risk based on the MammaPrint (performing 501 MammaPrint tests) expecting to prevent 20 events.

3.2
Impact of genomic testing on treatment allocation and prognosis
Assuming the 5 % treatment threshold, integrating the MammaPrint test can reclassify 39 % of the patients whose treatment benefit based on clinical characteristics lies between the 3.2 and 8.8 % (1.6–4.8 % assuming a 3 % threshold). For the binary OncotypeDX test potential reclassification can affect 65 %, namely patients in the 1.9 and 7.7 % range (1.4–5.3 % assuming a 3 % threshold). The continuous OncotypeDX test implies testing for almost all patients (99 %).
Treatment benefit estimates, and thus treatment decisions, change after integrating the genomic tests (Fig. 2). Reclassification occurred in 16 % of the total patient population with the MammaPrint: 6 % were reclassified to no treatment and 10 % to receive treatment (Fig. 2). The continuous OncotypeDX also led to reclassification in 16 % of all patients, with 9 % reclassified to no treatment and 7 % to receive treatment.

3.3
Clinical usefulness of genomic testing
To assess the clinical usefulness of genomic testing, we counted the number of treatments given and the number of events prevented following the different treatment strategies. The Net Benefit of treating no one was 0, since there were neither treatment benefits nor harms, resulting in expecting 110 events per 1000 patients. Treating all patients resulted in a negative weighted balance between preventing events and treatments given (NB = −13.8, per 1000 patients; Table 1), as treating all patients prevented 36 events at the burden of treating 1000. Assuming at most 20 treatments per event prevented (i.e. consistent with the 5 % threshold), we would have wanted at least 50 events prevented. Following the MINDACT strategy, 20 events would be prevented with treating 270 patients, resulting in a positive Net Benefit = 20–270∗0.05 ≈ 6.3.
Treatment decision-making informed by the clinical risk alone implied treating 220 patients with a treatment threshold of 5 %, expecting to prevent 16 events, resulting in a positive Net benefit of 4.8 per 1000. Including the MammaPrint, dichotomized OncotypeDX or continuous OncotypeDX, improved the balance between treatment benefit and harms (+1.6, +2.6, +3.4 Net Benefit per 1000 patients respectively). Compared to the MammaPrint, the OncotypeDX variants tended to allocate less chemotherapy, but more genomic tests would need to be performed.
Using a 3 % treatment threshold, the improvements in Net Benefit compared to clinical risk assessment alone decreased: +1.2, +1.5, +2.4, for integrating the MammaPrint, binary OncotypeDX and continuous OncotypeDX respectively (Appendix Table 1). Across all treatment thresholds considered, the OncotypeDX tests consistently showed a slightly higher Net Benefit compared to MammaPrint, with the continuous OncotypeDX test being the highest (Fig. 3). However, the difference in Net Benefit between the continuous OncotypeDX and MammaPrint showed substantial uncertainty, precluding strong conclusions (Fig. 4).

3.4
Sensitivity analyses
Extensive sensitivity analyses showed that changes in inputs did not change the ranking of the MammaPrint and OncotypeDX. However, the improvements in Net Benefit by adding the genomic tests did became smaller as the clinical risk model improved, or if a less effective chemotherapy was assumed (Appendix Fig. 1).

3.5
Decision analysis for TAILORx context
Compared to the MINDACT context, the 10-year risk of a distant metastasis or breast cancer related death was lower in the synthetic TAILORx cohort (83 compared to 110, per 1000 patients). Treating all patients reduced the number of expected events from 83 to 55 (Appendix Table 2). Due to a lower event rate, the Net Benefit for all strategies were lower compared to the MINDACT context. Differences in Net Benefit using different genomic tests were smaller, but followed similar ranking (Appendix Tables 2 and 3). In contrast, MammaPrint had a higher Net Benefit compared to the binary OncotypeDX on certain thresholds, but only by a small margin. Again, the Net Benefit of the continuous OncotypeDX was consistently highest, but with substantial uncertainty (Appendix Figure 2 and 3). Sensitivity analysis showed similar patterns as in the MINDACT cohort (Appendix Fig. 4).

4
Discussion
This study provided a model based comparison of using the OncotypeDX and MammaPrint in guiding adjuvant chemotherapy decisions in populations similar to the MINDACT and TAILORx trials. We focused on how these tests affect the balance between number of chemotherapy given and preventing 10-year distant metastases or breast cancer related deaths. Both MammaPrint and OncotypeDX improved treatment allocation and patient outcomes compared to using clinical characteristics alone, supporting the claims from the MINDACT and TAILORx trials that both tests are clinically useful. Comparing their binary versions, the OncotypeDX showed a slightly more favorable balance between the number of treatments given and the number of events prevented, at the cost of needing to perform more OncotypeDX tests. Using the continuous OncotypeDX score further improved this balance, but needing to test nearly all patients (99 %) rather than testing 39 % with the MammaPrint. Since these differences in clinical impact were quite modest, our evaluations support the broadly equivalent value of these tests at the population level [7,15,18].
Despite concluding similar performance, our results provide practical information to support informed discussions about what to expect in terms of the number of treatments given, preventing events, and the number of genomic tests needed to perform. Clinical consensus should agree on what the current position of these new tests are or whether additional research is warranted. While our results illustrate potential clinical usefulness of genomic markers, their clinical usefulness can decrease when using different treatment thresholds (for example 3 % rather than 5 %), the PREDICT algorithm improves or chemotherapy is less efficient than we assumed. Notably, the way test scores were presented also influenced clinical usefulness: using the underlying continuous OncotypeDX score, rather than its standard dichotomized version, improved its clinical usefulness, though it required testing nearly all patients due to a wider reclassification range. A similar pattern is likely for MammaPrint, but could not be evaluated due to unavailable information on its continuous score.
4.1
Implications
To fully leverage their clinical potential, the MammaPrint and OncotypeDX tests should be integrated into established risk algorithms such as PREDICT to align with contemporary real-word clinical decision-making [16]. Since PREDICT is commonly used for clinical risk assessment, and treatment decisions are adjusted based on results of the MammaPrint or OncotypeDX test, the risk estimates from PREDICT should be updated accordingly [35]. When clinical characteristics are known and a patient specific treatment threshold is established, it becomes possible to determine whether genomic test results could result in reclassification. Women with a clear treatment benefit above or below the threshold may decide upon chemotherapy without the need of genomic testing. Further research to evaluate the performance of genomic tests in combination with PREDICT is warranted, with potential further updating of the PREDICT risk calculator [4].
Another benefit of integrating genomic tests into PREDICT is that identifying candidates for genomic testing is based on individual treatment benefit rather than group-level clinical risk profiles. Identification of clinical risk profiles that might benefit from genomic testing often relies on clinical consensus [13]. For example, ASCO guidelines recommend both the MammaPrint and OncotypeDX test for HER2 negative, ER positive, postmenopausal women with node negative or up to 3 positive nodes [12], and suggest MammaPrint primarily for those classified as clinical high risk per MINDACT. However, this dichotomization of a continuous clinical risk may overlook patients at intermediate clinical risk. We showed that patients can be reclassified in both directions by genomic test results. Once genomic tests are integrated into PREDICT, a clinician can identify whether a test might reclassify the treatment decision. For example, with a 5 % treatment threshold, MammaPrint was only informative for patients with an estimated benefit between 3.2 % and 8.8 % based on clinical characteristics. This approach reduced testing to 39 % of patients in our analysis compared to 50 % in the original MINDACT trial.
Dichotomizing continuous test scores, as is done with MammaPrint and OncotypeDX [29,36], should be approached with caution, as our findings show that how a test score is presented can influence its clinical usefulness. In our analysis, the continuous OncotypeDX had the highest effectiveness in guiding treatment decisions, but at the cost of performing more genomic tests. This raises the question about MammaPrint's potential clinical usefulness if its continuous score were utilized. Improvements of moving beyond dichotomized MammaPrint scores are already evident, such as the identification of an ultra-low-risk group [37]. Continuous scores offer more precise and individualized predictions [38], but require testing more patients. In our example, the continuous OncotypeDX would imply testing 99 % rather than 39 % of the patients. This is a considerable increase in testing costs, which would need to be evaluated in a cost-effectiveness analysis against the reduction in chemotherapy use and the number of events prevented by better targeting of chemotherapy. To mitigate extra testing, a more refined approach can be used by estimating the likelihood of a specific genomic test score based on a clinical risk profile, as not every test result is equally likely for every clinical profile.

4.2
Limitations
Preferably we would have access to the individual patient data to perform the analysis. With access to this data, we could have explored two potential interaction effects that might affect the effectiveness of chemotherapy. However, we did not incorporate them due to a lack of conclusive evidence.
First, some studies have suggested that high OncotypeDX scores may also predict chemotherapy benefit [26,27]. Although the TAILORx trial also hinted towards a potential interaction effect, it was not designed to formally test for an interaction between recurrence score and treatment [11]. Moreover, the RxPonder found that OncotypeDX was prognostic, but not predictive for women with ER+, HER2-, N+ and 0–25 OncotypeDX scores [25]. To date, no genomic test has demonstrated predictive capacities in a prospective design [7]. Accordingly, clinical guidelines continue to emphasize the test's prognostic role rather than its predictive ability [13,14].
Second, subgroup analyses in both MINDACT and TAILORx suggested possible interaction with menopausal status and chemotherapy effectiveness [10,11]. However, large meta-analyses (e.g., EBCTCG), have shown consistent chemotherapy effectiveness across clinical subgroups, with no indication of reduced chemotherapy benefit in women aged older than 55 years [28]. Accordingly, the PREDICT algorithm assumes that chemotherapy effectiveness is not different for pre or postmenopausal women [2,4]. Once effect estimates are available, these estimates can be used to extend further model-based comparisons by adding these potential interaction effects.
The treatment threshold is essential in identifying whom to test and whom to treat with chemotherapy. Lowering the treatment threshold from 5 % to, for example, 3 % absolute risk reduction, would result in treating more women as we are willing to treat more to prevent one event. While predicting survival outcomes is one piece of the puzzle, international projects such as 4D PICTURE (https://4DPicture.eu) emphasize the importance of patient-reported outcomes and individual preferences in establishing personalized decision-making [39]. Recent guidelines also stress the importance of discussing patient's preferences to guide treatment decisions [13]. Updates to PREDICT now include information on treatment side effects, helping patients weigh benefits and risks to establish personal treatment thresholds [40]. For our analysis, we simplified this process by using the same treatment threshold for every patient to evaluate treatment decision strategies. However, we recognize that in clinical practice individual patient preferences can lead to different treatment thresholds, with some patients willing to undergo treatment for minimal benefit and others opting for more conservative choices [31].

5
Conclusions
We conclude that both the MammaPrint and OncotypeDX tests improve identifying candidates for adjuvant chemotherapy among women with intermediate clinical risk. Using their continuous versions might further improve their ability to guide treatment decisions and to improve patient outcomes, however, using continuous scores comes at the cost of more genomic testing to reach their full potential. Future studies should integrate genomic tests in existing clinical decision-making models such as PREDICT to improve individualized decision-making and refinement of chemotherapy allocations.

CRediT authorship contribution statement
Frank Doornkamp: Writing – review & editing, Writing – original draft, Visualization, Validation, Methodology, Investigation, Formal analysis, Conceptualization. Liesbeth C. de Wreede: Writing – review & editing, Writing – original draft, Visualization, Methodology, Formal analysis. Elfi Verheul: Writing – review & editing, Visualization. Agnes Jager: Writing – review & editing, Visualization. Ewout W. Steyerberg: Writing – review & editing, Writing – original draft, Visualization, Supervision, Methodology, Formal analysis, Conceptualization.

Data availability statement
The code generating the data underlying this article are available in its online supplementary material.

Funding
Funded by the European Union under Horizon Europe Work Programme 101057332. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Health and Digital Executive Agency (HaDEA). Neither the European Union nor the granting authority can be held responsible for them. The UK team are funded under the Innovate UK Horizon Europe Guarantee Programme, UKRI Reference Number: 10041120.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.