How to Ask AI About Your Health the Right Way – Insights from a Stanford Doctor

Overview

In this episode of Silicon Valley Girl, Dr. Jonathan Chen, a Stanford physician and computer‑science researcher, explains why AI can diagnose better than many doctors yet still needs careful handling.

He walks listeners through the surprising finding that GPT‑4 alone outperformed doctors who were given the same model, and then unpacks the practical steps patients should take when they turn to chatbots for medical advice. The conversation is a guide to asking health‑related questions without bias, using personal records wisely, and knowing when to double‑check AI output.

For anyone curious about the intersection of AI and everyday healthcare, the episode offers a roadmap to harness AI’s power while protecting against its pitfalls.

Key Takeaways

• GPT‑4 can generate more accurate diagnoses than clinicians who rely on it, highlighting a gap in human‑AI collaboration.
• Leading prompts steer AI toward the answer you expect, often reducing accuracy.
• Organizing your own medical records into a structured format can make AI assistance far more useful.
• High‑stakes advice (e.g., emergency triage) still requires verification by a qualified professional.
• Choosing the right AI model and using multiple models for a second opinion improves reliability.
• Doctors remain essential for accountability, contextual judgment, and patient communication.
• Over‑testing and unnecessary scans are not automatically beneficial, even when AI suggests them.

AI Can Outperform Human Clinicians—But the Human Factor Still Matters

Dr. Chen’s study showed that “GPT‑4 alone made better diagnoses than doctors using it,” a result that challenged the long‑standing belief that a human plus computer always beats either alone.

He explained that many participants had never used a chatbot before, so “the human is getting in the way. ” When clinicians received proper training, the combined performance improved, yet it still fell short of the AI‑only baseline.

This paradox underscores that raw computational power can surpass untrained users, but it also reveals a need for better education on AI tools within medical practice.

The Art of Prompting: Avoiding Leading Questions

The episode warned that “when we’re asking questions, we’re kind of hinting the answer that we want to hear,” which can cause the model to amplify bias. Dr.

Chen illustrated this with a personal anecdote: a rash photo led the chatbot to suggest “scabies, go to the emergency room,” while the doctor diagnosed an allergy. He advises phrasing queries with objective facts only—e.

g. , “My daughter has a rash, it’s been scratching for two days, here’s a description”—and avoiding speculative language like “Is this a serious infection?

” to prevent the model from chasing a preconceived narrative.

Leveraging Your Own Records to Make AI a Better Ally

One listener described building a “Perplexity chat with all my labs for the past seven years” and feeding it into the model, which then identified a genetic marker and generated a customized PDF for doctors. Dr.

Chen praised this approach, noting that “patients can understand their own information, make sense of that, and use the time with clinicians more efficiently. ” However, he cautioned that the tool is “like a chainsaw”—powerful but dangerous if misused, so users should curate relevant data rather than dump every detail.

When to Trust and When to Verify AI Advice

For routine questions, such as “Tylenol versus ibuprofen,” the AI’s answer is usually reliable. But for high‑risk decisions—like “Should I go to the ER?

”—the episode emphasized “double‑checking with someone who has accountability. ” Dr.

Chen referenced a lawsuit where a patient followed a chatbot’s reassurance and suffered a fatal clot, highlighting that “accountability, trust, responsibility” are still human responsibilities that AI cannot replace.

The Evolving Role of Doctors in an AI‑Heavy Landscape

Even as AI becomes “smarter than most doctors,” Dr. Chen reminded listeners that “knowing everything doesn’t make you somebody who’s worthy of trust.

” He argued that doctors provide empathy, nuanced judgment, and the ability to interpret ambiguous data—qualities that a model cannot replicate. The future, he suggested, will involve doctors guiding AI, verifying its outputs, and focusing on the relational aspects of care that machines lack.

Practical Applications

1. Before asking an AI, write down only the objective facts of your symptom or condition; avoid inserting your own hypothesis.
2. Gather your recent lab results, imaging reports, and medication list into a single, well‑structured document to feed the model.
3. Use a reputable, health‑focused AI (e.g., GPT‑4, Perplexity) and cross‑check answers with at least one other model for a second opinion.
4. For any recommendation that could lead to emergency care or major treatment decisions, contact a licensed clinician to confirm.
5. Stay informed about AI‑related updates in your health system and consider a brief training session on effective prompting.

Final Thoughts

This conversation shows that AI is a powerful diagnostic aid, but its value hinges on how we ask, what we feed it, and the safeguards we keep in place. The real breakthrough will be a partnership where clinicians harness AI’s speed while preserving human judgment and accountability.

Why This Matters

As AI tools become ubiquitous in everyday health queries, patients who learn to ask unbiased questions and verify critical advice will experience safer, more effective care, while doctors will shift toward roles that emphasize empathy and oversight.

---

Source

Podcast: Silicon Valley Girl

Guest: Dr. Jonathan Chen

Channel: Silicon Valley Girl

Published: September 25, 2026