AI model found to accurately diagnose thyroid cancer

A research team led by the LKS Faculty of Medicine at The University of Hong Kong (HKUMed) has developed an artificial intelligence (AI) model designed to classify both the cancer stage and risk category of thyroid cancer, achieving accuracy in excess of 90%. Described in the journal npj Digital Medicine, the AI model is expected to cut frontline clinicians’ pre-consultation preparation time by approximately 50%.

Thyroid cancer is among the most prevalent cancers in Hong Kong and globally. Precision management of the disease often relies on two systems: the eighth edition of the American Joint Committee on Cancer (AJCC) or Tumour-Node-Metastasis (TNM) cancer staging system to determine the cancer stage; and the American Thyroid Association (ATA) risk classification system to categorise cancer risk. These systems are crucial for predicting patient survival and guiding treatment decisions; however, the manual integration of complex clinical information into these systems can be time-consuming and lack efficiency.

The research team developed an AI assistant that leverages large language models (LLMs), like ChatGPT and DeepSeek, which are designed to understand and process human language, to analyse clinical documents and enhance the accuracy and efficiency of thyroid cancer staging and risk classification.

The model leverages four offline open-source LLMs — Mistral (Mistral AI), Llama (Meta), Gemma (Google) and Qwen (Alibaba) — to analyse free-text clinical documents. The AI model was trained with US-based open-access data with pathology reports of 50 thyroid cancer patients from The Cancer Genome Atlas Programme (TCGA), with subsequent validation against pathology reports from 289 TCGA patients and 35 pseudo cases created by endocrine surgeons.

By combining the output of all four LLMs, the team improved the overall performance of the AI model, achieving overall accuracy of 88.5% to 100% in ATA risk classification and 92.9% to 98.1% in AJCC cancer staging. Compared to traditional manual document reviews, this advancement is expected to halve the time clinicians spend on pre-consultation preparation.

“Our model achieves more than 90% accuracy in classifying AJCC cancer stages and ATA risk category,” said Professor Joseph T Wu, Managing Director of the InnoHK Laboratory of Data Discovery for Health at HKUMed. “A significant advantage of this model is its offline capability, which would allow local deployment without the need to share or upload sensitive patient information, thereby providing maximum patient privacy.

“In view of the recent debut of DeepSeek, we conducted further comparative tests with a ‘zero-shot approach’ against the latest versions of DeepSeek — R1 and V3 — as well as GPT-4o. We were pleased to find that our model performed on par with these powerful online LLMs.”

HKUMed’s Dr Matrix Fung Man-him added, “In addition to providing high accuracy in extracting and analysing information from complex pathology reports, operation records and clinical notes, our AI model also dramatically reduces doctors’ preparation time by almost half compared to human interpretation. It could simultaneously provide cancer staging and clinical risk stratification based on two internationally recognised clinical systems.

“The AI model is versatile and could be readily integrated into various settings in the public and private sectors, and both local and international healthcare and research institutes. We are optimistic that the real-world implementation of this AI model could enhance the efficiency of frontline clinicians and improve the quality of care. In addition, doctors will have more time to counsel with their patients.”

Dr Carlos Wong, also from HKUMed, said the next step is to evaluate the performance of the AI assistant with a large amount of real-world patient data. Once validated, the AI model can be readily deployed in real clinical settings and hospitals.

Image credit: iStock.com/FatCamera