AI oncology treatment decision support uses software to help cancer teams choose treatments, prepare tumour boards and match patients to clinical trials. The most famous early product, IBM’s Watson for Oncology, agreed with expert tumour boards about 82% of the time in a 2021 meta-analysis, yet it faded. The lesson: support the work around the decision, not repeat it.
Every week, in hospitals around the world, a tumour board meets. Surgeons, medical and radiation oncologists, radiologists, pathologists and nurses gather around a screen and work through a list of patients. For each one, someone presents the story: the scans, the biopsy, the genetic results, prior treatment, the patient’s wishes and fitness. The group agrees a plan. Then the next case. There are rarely enough minutes, and the preparation beforehand, pulling results from half a dozen systems, often happens late at night.
A decade ago, this looked like the perfect place for artificial intelligence. Cancer treatment is guided by complex, fast-changing evidence. Genomics was multiplying the options. IBM’s Watson for Oncology promised to read the literature, apply expert guidelines and recommend treatment for each patient. It was sold to hospitals in many countries. By 2022, IBM had sold off its Watson Health business.
The usual telling of that story is a cautionary tale about hype. The more useful telling is about what “decision support” should actually support. The hard part of oncology decisions is rarely knowing the guideline. It is assembling the facts, knowing which options are feasible for this patient in this health system, and finding the trial nobody had time to look for. Tools that help with those tasks are useful now. Tools that simply produce a recommendation for experts to agree with are not.
What happened to IBM Watson for Oncology?
Watson for Oncology was trained with oncologists at Memorial Sloan Kettering Cancer Center and deployed in hospitals across Asia, the Middle East and elsewhere. It ingested a patient’s details and returned treatment options rated as recommended, for consideration or not recommended.
Its difficulties became public from 2017. A University of Texas System audit that year found that MD Anderson Cancer Center had spent more than $62 million on a Watson-based project that was later shelved. Reporting in the following years raised questions about the quality of some recommendations. In January 2022, IBM announced the sale of Watson Health assets to the private equity firm Francisco Partners.
What makes the case instructive is that, by the metric it was usually judged on, Watson did reasonably well. A meta-analysis published in Scientific Reports in March 2021 pooled 9 studies covering 2,463 patients and found an overall concordance of 81.52% between Watson’s recommendations and multidisciplinary teams, counting both “recommended” and “for consideration” options. Concordance was highest in breast cancer and lowest in gastric cancer.
So why did a system that agreed with experts four times out of five fail to earn a lasting place?
Why is agreement with the tumour board the wrong measure?
Concordance asks whether the machine recommends what the experts would have recommended anyway. That is a reasonable safety check. It is a poor measure of value.
When the tool agrees with the tumour board, it has told the experts something they already knew. When it disagrees, the experts must decide whether the machine has spotted something or made an error, and in a system trained largely on one institution’s practice, disagreement often reflects different local circumstances rather than insight. Either way, it is hard to see what the tumour board gained.
The variation by cancer type in the meta-analysis hints at this. Where practice is highly standardised, agreement is high and the tool adds little. Where practice varies between regions and health systems, agreement falls, and that is exactly where a recommendation built on another country’s practice is least likely to fit.
A 2023 study from South Korea in Prostate International made the feasibility problem concrete. In 55 patients with urological cancers, Watson’s concordance with the tumour board was 92.7%. But in some cases it could not propose suitable treatment, and some recommended options were not feasible under the Korean national insurance system. A recommendation that cannot be delivered is not support.
What should AI decision support in oncology actually do?
Look at where time and errors actually accumulate in cancer care, and the useful jobs for AI come into focus.
Assembling the case. Before a patient reaches the tumour board, someone must collate pathology, imaging reports, genomic results, prior treatments and performance status, often from different systems. Errors and omissions here shape the whole discussion. Language models are well suited to drafting structured case summaries from records, provided a clinician checks them.
Finding trials. Matching patients to clinical trials is slow, manual work, and eligibility criteria are long and complex. It is also where patients most often miss options. TrialGPT, a framework from the US National Institutes of Health published in Nature Communications in November 2024, used large language models to retrieve and rank trials. In testing on 183 synthetic patients, it recalled more than 90% of relevant trials while retrieving less than 6% of the collection, judged eligibility criteria with 87.3% accuracy, and in a user study reduced screening time by 42.6%. The patients were synthetic and the study was not a clinical deployment, but it targets a real bottleneck. We have argued that trial diversity is largely a staffing problem; screening tools that cut staff time address part of it.
Checking feasibility. Useful support knows which drugs are on the local formulary, what insurers or national systems will pay for, and which services are available nearby. Watson’s Korean experience shows what happens without this.
Flagging what is missing. A tool that notices a missing biomarker test, an overdue scan or an eligibility criterion no one has checked is less glamorous than a treatment recommendation, and more likely to change what happens.
There is also a regulatory reason to prefer transparent support. The FDA’s final guidance on clinical decision support software, issued in September 2022, turns partly on whether clinicians can independently review the basis for a recommendation. Tools that show their sources and reasoning are easier to trust and to govern.
A worked example: the Thursday tumour board
Hypothetical, with illustrative numbers. A regional cancer centre’s thoracic tumour board reviews 20 patients in 90 minutes each Thursday. Preparation takes a coordinator and two registrars most of Wednesday. Trial screening happens informally, when someone remembers.
In version one, the centre buys a treatment recommendation engine. For each patient, the screen shows a suggested regimen. Most of the time it matches what the board would have chosen; occasionally it suggests something the board rejects for local reasons. After three months, attendees stop looking at it. Preparation time has not changed.
In version two, the centre uses AI differently. The night before, a model drafts structured summaries for each patient from the record, flagging missing items such as an outstanding molecular result. A trial-matching tool lists the two or three open trials each patient might be eligible for, with the criteria that still need checking. Registrars verify the summaries rather than writing them from scratch. At the meeting, the board spends its time on the decisions that need judgement.
Suppose preparation time halves and two extra patients a month are referred for trial screening who would otherwise have been missed. Neither number would appear in a concordance study. Both are what the board actually needed.
Aren’t modern large language models far better than Watson?
The strongest counterargument is that Watson belongs to a previous generation of AI. Today’s language models have absorbed far more of the medical literature, can reason over complex cases and can explain themselves in plain language. Where oncologists are scarce, as in many low- and middle-income countries, a good recommendation engine might bring expert-level treatment planning to places that have none. Dismissing treatment recommendations because of Watson, on this view, is fighting the last war.
Part of this is right. Current models are far more capable, and for under-resourced settings a well-validated, locally adapted recommendation tool could be valuable. But the lessons from Watson were not mainly about model capability. They were about measuring value by agreement, recommending options patients cannot access, and building on one institution’s practice. A more capable model that repeats those errors will fail for the same reasons, only more fluently. The evidence standard should also rise: as we have noted, AI evidence is strongest where the question is narrowest, and “what should this patient’s cancer treatment be” is about as broad as a clinical question gets.
What this means for cancer centres, oncologists and developers
For cancer centre leaders, start with the workflow bottlenecks: tumour board preparation, trial screening, missing-test detection and feasibility checks. Measure preparation time, trial referral rates and delays to treatment, not concordance with your own board.
For oncologists, treat AI outputs as a well-prepared draft. Use summaries and trial lists to save time, and verify them, especially for details that change treatment such as biomarker status and prior therapy lines.
For developers, the market lesson from Watson is that a recommendation experts already agree with is not a product. Local formularies, payer rules, available trials and transparent sourcing should be built in. Prospective evidence from real tumour boards will matter more than retrospective agreement rates.
For health systems and payers, supporting shared trial-matching and case-summary infrastructure across regional networks could spread benefits to smaller hospitals, which have the least time for manual screening.
For patients, asking whether a clinical trial might be suitable, and whether the care team has checked, is always reasonable. AI can widen the search. The decision remains a conversation with your oncology team.
Frequently asked questions
What was IBM Watson for Oncology?
Watson for Oncology was an AI system developed by IBM with Memorial Sloan Kettering Cancer Center to recommend cancer treatment options from patient data. It was deployed in multiple countries, particularly in Asia. After criticism of its value and recommendations, IBM sold its Watson Health assets to Francisco Partners, announced in January 2022.
How accurate is AI at recommending cancer treatment?
Studies usually measure concordance with multidisciplinary tumour boards. A 2021 meta-analysis of Watson for Oncology found 81.52% overall concordance across 2,463 patients, varying by cancer type. Concordance shows safety and alignment with expert practice, but not whether the tool improved decisions or outcomes, which few studies have measured.
Can AI help match cancer patients to clinical trials?
Early evidence is promising. The NIH’s TrialGPT, published in 2024, recalled over 90% of relevant trials in testing on synthetic patients and reduced screening time by 42.6% in a user study. Real-world deployments still need evaluation, and clinical staff must confirm eligibility before referral.
What is a tumour board?
A tumour board, or multidisciplinary team meeting, is a regular meeting where specialists including surgeons, oncologists, radiologists and pathologists review individual cancer cases and agree a treatment plan together. It aims to bring several kinds of expertise to each decision and is standard practice in many cancer centres.
Will AI replace oncologists in treatment decisions?
Not in the foreseeable future. Treatment decisions depend on the patient’s goals, fitness, preferences and local options, and on judgement across specialties. The more realistic role for AI is preparing information, finding trials and flagging gaps, so that oncologists and patients spend more time on the decision itself.
Oncology does not need a machine that agrees with its experts. It needs one that does the preparation they never have time for, and finds the trial nobody remembered to look for.
Related on the GHC blog: Why Universal Health Coverage Stalls After the Pilot Phase and Paying for Outcomes Rents Behaviour. It Doesn’t Buy It..
If AI could take one job off your tumour board’s plate tomorrow, which would you choose?
