The utility of large language models in oncological multidisciplinary team meetings: A systematic review.

Large language models (LLMs) have emerged in recent years as innovative artificial intelligence systems with early potential in clinical decision-making. This is the first systematic review to evaluate LLMs' oncological decision-making and compare their treatment recommendations to "gold standard" oncological multi-disciplinary team (MDT) decision-making. PubMed, EMBASE and Medline databases were last searched on 20th January in line with PRISMA guidelines. All relevant peer-reviewed publications comparing LLM and MDT treatment recommendations in patients with cancer were included. Studies using fictional cases, case reports, and conference proceedings were excluded. Modified QUADAS-2 tool was used for bias assessment. The primary outcome was the concordance between LLM and MDT treatment recommendations. 34 publications met the inclusion criteria with a total of 3513 patient cases included in this review. Studies were highly heterogenous with regards to study design, sample size, cancers studied, and LLM models evaluated, among others. Concordance rates ranged from 16 to 100% across all studies. Highest concordance rates were noted in prostate cancer cases, where the LLM was directed to incorporate established international guidelines in decision-making. One third of studies exhibited a high level of bias. Limitations to LLM decision-making include overtreatment of frail patients, lack of reproducibility, insufficient niche knowledge, occasional life-threatening recommendations, and medico-legal issues including privacy and confidentiality. LLMs may be capable of generating appropriate oncological treatment recommendations, but early outcomes are inconsistent, and conflicting across the various studies with regards to safety. Robust prospective comparative studies are yet needed to better determine their utility in this setting.