Second Opinions

“We were all shocked by the result,” Ni Xin, the hospital director and surgical specialist at the ENT department who chaired the joint human-AI consultation, told NewsChina.　

Driven by the release of new AI DeepSeek in January, the use of AI medical applications has rapidly spread to over 100 hospitals across the country, transforming the medical landscape, despite debates over technological advances and ethical concerns. In November 2024, the National Health Commission (NHC) issued guidelines on AI application scenarios, covering all medical services from patient appointments to doctors’ training.

Replacing search engine Baidu.com, DeepSeek is an upgraded platform that can provide comprehensive and systematic answers to patients before they see a doctor, Yu Jintai, chief physician of the neurology department, Huashan Hospital Affiliated to Fudan University in Shanghai, told NewsChina.　

In August 2024, BCH cooperated with Baichuan AI, established in 2023, to roll out five AI Large Models, including the AI pediatrician to aid human doctors.　

Ni’s ultimate goal is to have a million AI pediatricians to bridge the healthcare gaps in underserved communities and remote regions.　

The country had only 205,800 pediatricians in May 2024, according to the NHC, whereas the population aged between 0-15 was almost 248 million in 2023, China’s National Bureau of Statistics revealed.　

“Since we did our first AI pediatrician consultations, at least six hospitals contacted us hoping to introduce similar products,” Ni said.　

At Huashan Hospital Affiliated to Fudan University in Shanghai, internal tests of AI applications have covered scenarios of multiple scientific research projects and clinical practices.　

Cheng Sijie, deputy director of the big data center at Huashan Hospital, told NewsChina that it only took a few days for the hospital to launch a platform run by DeepSeek, but the critical challenge is how to standardize AI products in the medical field.

However, Cheng stressed the records must be absolutely accurate.　

“Only if they pass extremely stringent tests can AI large models particularly trained to make clinical records be used at our hospital,” Cheng told NewsChina.　

According to Cheng, Huashan Hospital has tested different large models run by DeepSeek. One essential criteria is their abilities in filing and updating clinical records.　

“Doctors must double-check all digital records taken by AI models to ensure the quality and safety of healthcare services,” Cheng said.　

His caution is warranted due to a phenomenon called AI hallucinations, which means AI models misinterpret patterns and generate incorrect or misleading outputs.　

Because of these glitches, some clinicians, like those at Tongji Hospital in Shanghai and Huazhong University of Science and Technology in Wuhan, Hubei Province, are more cautious. 

“AI models can make blunders in clinical practices, say by taking signal interference in medical imaging as signs of lesions,” Guo Wei, deputy chief of the infectious diseases department at Tongji Hospital, told NewsChina.　

According to Yu, the credibility of AI platforms depends on accurate and professional data input, as mistakes lead to erroneous results.　

At BCH, more than 300 top doctors have contributed their knowledge and decades of records of cases to the AI pediatrician application.　

“To reduce AI hallucinations, it is essential to guarantee data uniqueness and accuracy from the very beginning of the modeling,” Ni said, adding that compared to DeepSeek, AI pediatricians designed for the medical field perform in more precise ways.　

The possibility of hallucinations requires doctors to double-check AI responses to avoid medical negligence, otherwise they will be responsible for the consequences. Besides, the hospitals and developers are accountable for negligence caused by medical staff operating the AI without training or flawed algorithm design, Deng Yong, professor of medical and health law at the Beijing University of Chinese Medicine, told NewsChina.

According to Guo, medical imaging diagnosticians are probably the first to be substituted. “AI can discover nodules that can’t be discerned by the human eye from X-rays or computerized scans,” Guo said. Human doctors may only need to assist or supervise AI operations.　

But a majority of doctors disagree. “AI and doctors are interdependent. The former will assist the latter without replacing them,” Ni said.　

In interviews with NewsChina, a number of doctors from various fields including infectious disease, neurology and surgery said AI is a supplementary tool that cannot replace doctors’ decision-making roles.　

“No matter how advanced the technologies get, the deep thinking and quick reactions of doctors are irreplaceable,” Sun Yong’an, chief neurologist at Peking University First Hospital in Beijing, told NewsChina.　

“In addition to their professional background, doctors judge diseases according to their clinical experience, which is critical but hard for AI to emulate,” Yu said.

BCH and Baichuan AI plan to expand the program to 150 pilot primary hospitals in Hebei Province before introducing AI pediatricians nationwide.　

Huang Hong, director of the big data center of Huashan Hospital, advised hospitals to establish different AI platforms instead of relying solely on DeepSeek. “So you can support all kinds of needs from different medical fields,” she said, predicting that the future of AI healthcare could be an integration of thousands of models.　

In addition to large models, AI hospitals, running in a closed loop to help AI doctors improve their skills from diagnosis to rehabilitation, are being trialed. In November, Agent Hospital, an AI hospital developed by the Institute for AI Industry Research at Tsinghua University in Beijing, tested 42 AI doctors to diagnose and treat over 300 diseases.　

“The hospital marks big progress from AI large models, as it enables AI doctors to deal with diseases in a complex environment by communication, coordination and equipment utilization,” Liu Yang, executive director of the institute, told NewsChina, adding that their next goal is to connect the AI hospital with the real world through advanced virtual reality (VR) technology.　

However, Zhao is not so upbeat about AI healthcare. “It’s still too early to know whether AI technologies can be truly applied or will only be used as a kind of convention,” he said.　

Guo is cautious too. “Many hospitals introduce AI platforms because they’re afraid of lagging behind, but few use the technology to meet the clinical demands for deepened research.”　

Deng pointed out that medical AI technologies developed without industry standards and the discrepancies caused by different platforms will affect cooperation between hospitals and increase the difficulty of supervision.　

“AI technology shouldn’t be developed in a rush. Its true value can only be realized by stable and well-designed progress,” Jin Chunlin, director of the Shanghai Health Development Research Center, told NewsChina.