Last August, a case study published in the “Annals of Internal Medicine” generated national interest after a patient wound up in the emergency room after following ChatGPT’s “salt substitute” advice.

The 60-year-old man, who wanted to cut down on table salt, asked ChatGPT for a sodium chloride substitute, according to the case study.

ChatGPT suggested sodium bromide, which landed the man in the ER three months later with neuropsychiatric symptoms and some abnormal laboratory tests, according to the study.

“When we asked ChatGPT 3.5 what chloride can be replaced with, we also produced a response that included bromide,” researchers wrote. “Though the reply stated that context matters, it did not provide a specific health warning, nor did it inquire about why we wanted to know, as we presume a medical professional would do.”

Dr. Michael Endris, a pediatrician with OSF Medical Group in Bloomington, said patients who once turned to Dr. Google are now consulting ChatGPT.

“Everybody has the app on their phone,” Endris said. “And they’re using it to ask questions: ‘Hey, what’s the best steakhouse near me?’ Where do I go for this?’” Endris said.

So naturally, parents will ask AI, “Should I worry about this fever?” when their kids get sick, Endris said.

“And it’s going to return some, probably broad, response without a lot of clinical context,” Endris said. “So it can definitely induce some worry about some potentially bad diagnosis that it’s going to spit out at you.”

According to a recent Kaiser Family Foundation tracking poll, 32% of adults nationally have turned to an AI chatbot in the past year with questions about their health.

According to the poll, “adults under age 30 are about three times as likely as adults ages 50 and older to use AI for mental health information (28% vs. 8%). Uninsured adults also are more likely than those with insurance to do so (30% vs. 14%), as are Black and Hispanic adults compared to White adults.”

Dr. Kanan Modhwadia, a psychiatrist and medical director of behavioral health and psychiatric emergency room services at Northwestern Medicine Central DuPage Hospital, said that as more people use AI, she cautions people that “AI is not your friend and it’s not a therapist.”

Asking very specific questions when interacting with AI raises the chances of accurate answers, Endris said.

For instance, a doctor “digging into research” for precise articles on a topic won’t ask if metformin is good for diabetes, Endris said.

A doctor would ask if metformin is a good medication for an adolescent with type 2 diabetes, he said.

Or a doctor might compare medications for preventing chronic kidney disease, he said.

However, patients should not use AI for medical advice, “especially around any diagnosis or treatment,” Endris said.

“And you don’t necessarily want to put a lot of protected health information out there,” Endris said.

For solid online information, the American Academy of Pediatrics has a website called healthychildren.org, with topics such as nutrition segmented by the child’s age, Endris said.

“The information they put on their website is either evidence-backed or has expert opinion in those areas where there’s not enough research and evidence to guide it,” Endris said.

What if the child gets sick in the middle of the night?

“I would strongly encourage them to call their doctor, even after hours,” Endris said.

For instance, OSF has a 24-hour nurse triage service that can “run through the child’s history” with the parents and ask targeted questions to determine whether the child needs emergency care, be seen in the morning or should be seen within three days, Endris said.

The National Institutes of Health’s MedlinePlus.gov website has more than 1,000 health-related topics, along with a tutorial for evaluating online health information.

“It’s very easy to engage with AI. AI is always there,” Modhwadia said. “It’s not going to speak against you or make fun of you, and it seems like it’s listening.”

However, future AI “may provide ways for patients to engage more deeply with their health system about their healthcare to understand more about what’s going on,“ said Dr. Nirav Shah, associate chief medical informatics Officer, AI and innovation, and director of investigational innovation at Endeavor Health.

Patients may be able to “import their own data and learn what that means” and then help patients “understand what questions to ask your physician,” Shah said.

Modhwadia said that even now, AI can help patients generate precise questions to ask their doctor, such as “What do these results mean for me?” and “What test do I need next?”

“Patients can come to the appointment feeling empowered and organized,” Modhwadia said.

However, AI often causes “health anxiety” in patients, Modhwadia said.

With electronic health records, patients often receive test results before the doctor reads and interprets them – and then asks AI to explain them, resulting in many possible diagnoses heedless of the patient’s history, Modhwadia said.

“AI won’t categorize them: ‘These are the rare ones; there are the more common ones,’” Modhwadia said. “You’re going to get everything, even the scariest stuff. If AI is not prompted in a specific way – ‘Give me the common diseases first’ – you’re going to think you have cancer right off the bat."

Dr. Harvey Castro, known professionally as “Dr. GPT,” said during an OSF Healthcare Innovation Showcase at Jump Trading Simulation & Education Center, stressed that patients should never rely on the technology alone for medical advice, that “we need that doctor in the loop.”

“The problem with ChatGPT is it sounds so real and it sounds so correct and it personalizes to you and it knows what words to use,” Castro said during the showcase. “So that if it’s making it up, it sounds so real, so the average person won’t know if it’s hallucinating or is it saying something wrong or is it right?”

Shah likened AI to nuclear energy, which has pros and cons, depending on its use.

Although AI’s hallucination rate is decreasing, AI will never be 100% accurate, Shah said. The need for verification, safety protections and critical thinking will never go away even as AI “becomes embedded in everything,” he said.

“It’s going to be a very interesting time for healthcare,” Shah said.