Rethinking the Physician Response to Patient AI Use, With Robert Shpiner, MD

A patient who arrives at a visit having already consulted a chatbot about chest discomfort or an unsettling home blood pressure reading is no longer an outlier. General-purpose large language models have become a common first stop for clinical questions, and the safety data are concerning.

A JAMA Network Open evaluation of 21 off-the-shelf large language models found differential diagnosis to be the weakest reasoning task, with failure rates exceeding 80% for every model.¹ A BMJ Open audit of 5 widely used chatbots found approximately half of responses problematic and 1 in 5 highly problematic.² Other findings are more mixed: a 2025 systematic review reported diagnostic accuracy ranging from 25% to 97.8%, with large language models outperforming clinicians in roughly a third of studies, and an emergency department study found broad triage agreement with physicians despite a tendency toward overtriage.^3,4

Patients use these tools anyway. A 2026 KFF tracking poll found that 32% of US adults had used AI for health information or advice in the previous year. About 1 in 5 cited an inability to get an appointment or the lack of a usual provider, and a similar share cited difficulty affording care, though larger shares said they wanted quick answers, previsit information, or privacy.⁵

Robert Shpiner, MD, of the David Geffen School of Medicine at UCLA, argues that warnings about chatbot inaccuracy, though correct, are incomplete because they name the hazard without explaining the demand behind it.

"The attraction is not that the tool is wise but that it is available," Shpiner wrote.⁶

In the following Q&A, Shpiner discusses how internal medicine physicians should respond when patients bring AI-generated research into the exam room, and why correction alone is an insufficient response to why patients turned to a chatbot first.

Q&A: Rethinking the Physician Response to Patient AI Use, With Robert Shpiner, MD

Patient Care Online: When a patient comes in having already asked an AI chatbot about their symptoms, how should physicians respond?

Shpiner: Yeah, there's legal action against AI now for both not doing enough and for doing too much. On one hand, they didn't do enough for a potential suicide patient, and on the other, they intervened too much with someone who died of a PE. So they're caught at both ends of the spectrum.

What we need to do is open up our door and say, "Oh, I see this is interesting information. Let me tell you what I think, and it gives you insight — it's another part of your history." So if your patient is looking this up and is worried about cancer, and you're open to what's going on, you suddenly get to "Oh, they're worried about this because my dad died at an early age, and I'm deathly afraid that every time I get a headache, it's going to be brain cancer." Well, let's talk about that. We talk about the genetics of brain cancer.

Now, what's a system-wide impediment to this? We have 15 minutes with patients. We're looking at computer screens. We're timed so precisely, and we've limited their contact with us. Back in the golden years, I could sit with a patient, and if someone needed 10 minutes and another needed 30 or 40, you could do that. So if you had someone who needed an extra 10 minutes discussing their family history of cancer and why they're worried, you could prevent a lot of trips and a lot of expense. They don't have to go to a neurologist. They don't need a CAT scan. They don't need a bunch of tests. There's a lot you can do by just saying, "I understand why you're worried, and let me explain why it's just a tension headache. Here's my plan: we're going to do A, B, and C, and if I'm wrong in a month, we'll get that CAT scan."

Whatever it is, we have a plan. You're using the information they bring you as part of the history. The web searches, the internet, the AI searches — they should be incorporated and used as a tool instead of something to fight against and be upset about.

I remember when this first started with Google, and my first reaction was defensive — "I'm the doctor, not you." As a profession, we have to get away from that and recognize how we're going to incorporate this. Part of the way we incorporate it is by acknowledging it with patients: "Yes, I know you're using it, and I'm very interested in what questions you're asking and what the responses were, so I can help you understand what it means. Yes, I know you're probably on Instagram and seeing a lot of stuff." People are seeing things like sunblock claims, methylene blue, whatever the health topic of the moment is, cyclospora — someone saying "there's a lot more of this going around, maybe there's something else going on" — and suddenly they're into a conspiracy theory.

I have to understand that people are worried about that, so I can explain what the problem actually is and why. And yes, it's okay for you to look into this, and here's what I'd avoid. Being aware of what's out there is a crucial part of how we're going to deal with patients, and a lot of us have a lot of catching up to do. We have to learn how to use the technology, learn how to understand it, and learn to not be as rigid in the "I'm right, I have 30 years of experience, I know what I'm doing, they're wrong" sort of attitude.

So what would I tell people in practice? Be open rather than closed. Bring people in. Understand they're using it, understand they're going to be asking questions, and even expand on it by saying, "What else have you learned from Instagram or social media?" That way they get a subtle sense that you aren't judging them — that you're open. Then they can say, "I saw this thing on Instagram," or whatever it may be. You're opening a trust channel there, and you're not judging it, because it's going to be there regardless.

Second, address it openly: "Claude was right about this" or "Claude has some wrong information here, let me tell you what I know" or "Geez, I didn't even think of that, let me get back to you, or we'll think about it together" — because it may be a good idea. There's nothing wrong with saying, "Huh, maybe, let me check it out." That way, if nothing else, you gain trust.

I was hoping at some point there would be a system where AI access came through the practice, so that overnight use of AI could be monitored — you'd find out what patients were asking, and the next day someone could screen it, like we do for an ER. In the ER, a nurse reviews all the discharges the next day and follows up to make sure nothing was missed.

In a similar way, if we incorporated an AI platform into practices — "here's this platform, we'd like you to use ours because we know the information going through it, and we have a trust model, you know we're not spiking the punch" — go ahead and ask your question, and within a day or a day and a half, we've reviewed it. So the patient who flagged "there's bright red blood running out of my ear" might get a call back quickly so we can address that issue.

My idea is the general concept of embracing rather than rejecting this, and incorporating it, because it's a tool that can be useful. I think we have to have people up front more. We all know the experience of calling somewhere and hearing "your call is important to us," pressing 1 for this, 2 for that, and then getting put on hold, or getting triaged to another place and told you can see a doctor in a week.

That distance is the concept. It makes sense to me why people are turning to other avenues. The introspective part is that we need to address that. We don't have the manpower to have everyone seen every time, but there's got to be ways to triage it. If you're in a practice where you know that if you have an emergency you'll be seen, and if it's urgent they'll help parse that, there'll be more trust.

So it's a runny nose, I have a fever, I'm not worried about it, I'll see in 3 days if it turns into something. Or my leg just really swelled up — there's a way to triage that. I'd rather have people incorporated in the system that we're dealing with rather than not. So my answer is: incorporate, and minimize as much as you can where possible.

So, sequentially: one, embrace the technology, because it isn't going away. Two, learn about it as best you can. In my own ICU rounds, I've already done this — once a week I have "current event rounds," where I gather everyone around and say, "This is what's going on out there." It may be nothing, but everyone is at least aware that this is part of what we're incorporating.

Embrace it, add it to your practice, elevate the level of interaction around it, and be non-judgmental. Remain in partnership with your patients. They're looking for a reason — we may not understand it, we may not like it, but they're looking for a reason, and the more we understand what's driving that reason, the better we'll be at helping them.

Editors’ note: Shpiner reports no relevant disclosures.

References:

1. Rao AS, Esmail KP, Lee RS, et al. Large language model performance and clinical reasoning tasks. JAMA Netw Open. 2026;9:e264003. doi:10.1001/jamanetworkopen.2026.4003

2. Tiller NB, Marcon AR, Zenone M, et al. Generative artificial intelligence-driven chatbots and medical misinformation: an accuracy, referencing and readability audit. BMJ Open. 2026;16:e112695. doi:10.1136/bmjopen-2025-112695

3. Shan G, Chen X, Wang C, et al. Comparing diagnostic accuracy of clinical professionals and large language models: systematic review and meta-analysis. JMIR Med Inform. 2025;13:e64963. doi:10.2196/64963

4. Alomari LM, Alshammari MM, Arbaeen AO, et al. Safety and accuracy of AI in triaging patients in the emergency department. Int J Emerg Med. 2025;18:243. doi:10.1186/s12245-025-01069-x

5. Montero A, Montalvo J 3rd, Kearney A, et al. KFF tracking poll on health information and trust: use of AI for health information and advice. KFF. Published March 25, 2026. Accessed April 22, 2026. www.kff.org/health-misinformation-and-trust/poll-finding/kff-tracking-poll-on-health-information-and-trust-use-of-ai-for-health-information-and-advice

6. Shpiner RB. Why patients ask the chatbot first. Ann Intern Med. Published online July 28, 2026. doi:10.7326/ANNALS-26-01819