A multiagent large language model-based system to simulate the liver transplant selection committee: a retrospective cohort study.

[BACKGROUND] Transplantation is one of the few areas in medicine in which the definitive treatment is rationed. Subjective decision making poses challenges for the transplantation selection process. Large language models (LLMs), through autonomous artificial intelligence (AI) agents, have implications in objective decision making for complex problems; however, there are gaps in the literature regarding their clinical applications. We aimed to examine the performance of a multidisciplinary committee of AI agents in the liver transplantation selection process as a proof of concept towards improving the objectivity of decision making.

[METHODS] This retrospective, hybrid cohort study incorporated both real-world observational data and artificially generated data derived from the Scientific Registry of Transplant Recipients (SRTR) database in the USA. We retrospectively analysed adult patients (aged ≥18 years) who received liver transplants from Jan 1, 2004 to June 1, 2024, and generated a hypothetical cohort of patients with standard absolute contraindications to liver transplantation from a random subset of the cohort. An AI selection committee was configured, consisting of four LLM-based AI agents using GPT-4o-2024-08-06 prompted with specialised roles: transplant hepatologist, transplant surgeon, cardiologist, and social worker. This committee was instructed to assess clinical vignettes generated from the SRTR tabular data (eg, demographics, social determinants of health, end-stage liver disease, comorbidities, and laboratory measurements), placing candidates on the transplantation waitlist if liver transplantation would likely offer a 6-month or 1-year survival benefit or rejecting candidates for transplantation if contraindications were present or if such a survival benefit was unlikely. The primary objective was to assess the accuracy, sensitivity, and specificity of the AI selection committee in predicting 1-year post-transplantation survival. The secondary objectives were the accuracy, sensitivity, and specificity of identifying contraindications to liver transplantation and predicting 6-month post-transplantation survival, and the numbers of false-negative cases (ie, patients who were rejected by the AI selection committee but did receive a transplant) and false-positive cases (ie, patients who were placed on the waitlist by the AI selection committee but had assigned contraindications to transplantation).

[FINDINGS] Of 8412 patients, 7033 (83·6%) were placed on the transplantation waitlist and 1379 (16·4%) were assigned to the hypothetical cohort and had contraindications to liver transplantation. For the prediction of 1-year post-transplantation survival, the AI selection committee had an accuracy of 92·00% (95% CI 91·43-92·58), a sensitivity of 1·00 (0·99-1·00), and a specificity of 0·66 (0·64-0·68). For identifying absolute contraindications to liver transplantation and predicting 6-month survival, accuracies were 98·19% (97·90-98·44) and 94·88% (94·37-95·29), respectively; sensitivities were 1·00 (0·99-1·00) in both cases; and specificities were 0·91 (0·89-0·92) and 0·75 (0·73-0·77), respectively. The most common reason for the AI selection committee to reject patients in false-negative cases was hepatocellular carcinoma that did not meet the Milan criteria (16 [62%] of 26 as a contraindication to transplantation, 16 [62%] of 26 rejected on the basis of 6-month survival prediction, and 14 [61%] of 23 rejected on the basis of 1-year survival prediction); the most common cause of death in false-positive cases was malignancy at 1 year (137 [28%] of 481).

[INTERPRETATION] LLMs can be leveraged through a multiagent AI system to simulate discussions of a liver transplantation selection committee and provide accurate, objective insights on patients who might or might not benefit from a liver transplant. Lessons learned from this study are a step towards making the liver transplantation selection process more equitable and objective.

[FUNDING] Transplant AI Initiative, Ajmera Transplant Centre, University Health Network.