Today's AI models score almost perfectly on medical exams. On MedQA, the US licensing-style question set, nearly every new model clears 95%, and the leaders sit around 96%. Other familiar medical multiple-choice sets like MedMCQA and PubMedQA are approaching the same ceiling.
But a high exam score says little about real-world quality. The team behind OpenAI's HealthBench made the same point, that exam-style benchmarks have saturated, so they built a new evaluation grounded in real medical conversations scored by physicians. On that benchmark, even the top models reach only about 60%. Work on clinical safety goes further, showing that the advice a model gives can still carry a risk of harm. The gap between these numbers says something simple: acing a multiple-choice exam is not the same as conducting a real consultation.
That gap isn't about knowledge. It's about how the models are built. Most medical AI today learns from academic question-and-answer data built on clinical vignettes, where a self-contained case description is paired with a single conclusion. That kind of training produces a model that scores well on exams, but real care isn't about handing back an answer. It is a consultation, where the doctor builds a full picture of the patient before reaching any conclusion.
A doctor who hears "chest pain" doesn't jump to "heart attack." They take a history. They ask. They find out when the pain started, what it feels like, whether it radiates, and what the patient is afraid of. The conclusion is the last step of a consultation, not the first.
To build the ambient scribes and triage assistants that can work beside a doctor, we need to teach a model how to take a history, not how to hand back a conclusion. Meddies Consultant is a first step toward that: a synthetic dataset of Vietnamese and English clinical consultations, built so a model learns to consult rather than to conclude.
Building AI as a second brain for doctors.
To us, an AI that knows how to ask, that poses the questions which surface a blind spot, is worth more than an AI that rushes to a conclusion. The conclusion stays with the doctor. The AI acts as a second brain, helping the doctor see possibilities they hadn't considered: the uncommon diagnoses that are still possible, and often missed.
Two problems: privacy and language
The first problem is privacy. The most direct way to teach a model to take a history is to train it on real doctor-patient transcripts. But those transcripts are tied to patient identifiers, and in healthcare that kind of data is almost impossible to share publicly. The whole field runs into the same bottleneck: the algorithms keep improving and the models keep getting better at reasoning, yet there is little real conversational data to train and strengthen them in a clinical setting.
The second problem is language. Most high-quality medical data is in English, so the usual way to get data for another language is to machine-translate a Western dataset. In medicine, that shortcut is risky. How a patient describes their pain is clinical evidence, not linguistic packaging.
Vietnamese has many ways to capture the texture of a single symptom. Đau nhức is a dull, deep ache. Đau buốt is a sharp, stabbing pain that comes and goes. Đau thắt is a constricting, squeezing pain, the word that makes a doctor think of the chest and stop to pay attention. Flatten all three into "aching, sharp, tight" and you lose more than the nuance; you lose the line that separates one diagnosis from the next.
This is the main reason we build our own generation pipeline. It lets us control the entire process, from the patient's context to the way the consultation unfolds. And it lets us start from a different reality: Vietnamese patients come from many backgrounds. Some are farmers, some are office workers, and each explains a symptom in their own way, in the everyday words of their region.
How we built the dataset
The hard part of generating consultation data isn't volume. A simple prompt handed to a language model can generate plenty of dialogue, but it tends to fall into a few familiar molds: the same opening, the same length, the same agreeable, easy-to-interview patient. Karpathy has pointed out this risk in LLM-generated synthetic data: each sample looks reasonable on its own, but the dataset as a whole clusters around a handful of phrasings and situations. It looks large, while its real diversity barely grows, and the model ends up learning a few repeated patterns instead of the breadth of clinical practice.
So instead of only writing a prompt, we constrain both the input and the generation itself, so that each consultation is forced to differ and forced to resemble a real visit. There are three main constraints.
Personas. Using personas is a proven way to build diversity into generated data: PersonaHub showed that a large enough set of personas can produce diverse synthetic data at scale. But the best public persona sets, like Nemotron-Personas-USA, are built around Americans, in English, and describe everyday life in general terms. They bring diversity of people, but no clinical context. A consultation dataset needs more than demographic diversity; it needs diversity of clinical situations. So we pair each persona with a disease drawn from a 1,236-condition taxonomy and an inner state defined by FIFE, so that every patient is as distinct as a real person and carries a specific clinical context. Following the same idea, we also developed our own Vietnamese persona set, Meddies Persona.
Three standard clinical frameworks. Generation is constrained by three frameworks from medical communication:
- Calgary-Cambridge sets the arc of the visit: opening the session, gathering information, building the relationship, and closing.
- OPQRST makes the AI doctor gather symptoms systematically: onset, provoking factors, quality, region and radiation, severity, and time course.
- FIFE describes the patient's inner state: feelings, ideas, function, and expectations.
OPQRST keeps the AI doctor disciplined, while FIFE keeps the patient human and a little messy: worried, prone to rambling, and sometimes needing to be guided back on track.
One feature of the dataset is that every AI-doctor turn carries a short reasoning trace. It names the Calgary-Cambridge phase in progress, follows OPQRST, and updates FIFE after each patient reply. This trace shows that the consultation is driven by a clinical structure, rather than generated freely and labeled afterward to look the part. The example further down makes this visible.
Meddies ConsultantHow we score a consultation
After generation, every sample passes through two layers of control that drop the consultations which don't make the bar.
The first layer is automatic. It checks the structure and format of each sample, catches parsing anomalies, and removes duplicates at both the per-file and whole-dataset level, so that the same consultation is never counted twice.
The second layer is a review against clinical criteria. A consultation that passes is more than a fluent block of text, so we score each sample on several dimensions:
| Criterion | Why it matters |
|---|---|
| Completeness | Does the consultation gather the clinical information it needs, or stop too early? |
| Appropriateness | Is the advice medically sound, without saying anything wrong or dangerous? |
| Naturalness | Does the patient sound like a real person, or like a repeated template? |
| Empathy | Does the AI doctor acknowledge the patient's worry, not just process symptoms? |
| OPQRST quality | Are symptoms explored in a clear, deliberate order? |
| FIFE adherence | Is the patient's inner state tracked through the whole conversation? |
| Structural coherence | Does the consultation move through the phases of a real visit? |
| Safety | Does the AI doctor know its limits and avoid concluding in a real doctor's place? |
Answering versus consulting
The two samples below come from two different datasets. On the left is a real sample from ChatDoctor-HealthCareMagic-100k, a set of doctor-patient exchanges collected from an online consultation platform: the patient describes a problem, the doctor answers, all in a single turn. On the right is a sample from Meddies Consultant, drawn from the Vietnamese split: a 12-year-old with swollen neck nodes, a low fever in the afternoons, and slight weight loss, interviewed over several turns under clinical frameworks.
Real data, but each sample is a single exchange: one question, one answer.
Phase 1 · Opening
Phase 2 · Gathering · FIFE: worried
Phase 2 · Gathering
Phase 3 · Structuring · Safety
Red flags
Synthetic data, but each sample is a multi-turn consultation under clinical frameworks. Target disease here: lao hạch (TB lymphadenitis).
The difference isn't that one is better than the other, but what each one is for. The sample on the left teaches a model to give a reasonable answer to a complaint. The sample on the right teaches a model to conduct a consultation: to update the patient's anxiety through FIFE, to gather symptoms systematically, and above all to know when to stop. Because the nodes have been swollen for more than two weeks and the patient appears to be losing weight, the model doesn't try to diagnose through a screen. It points the patient toward an in-person visit and lists the signs that warrant going to a hospital right away.
The target diagnosis in the right-hand sample is tuberculous lymphadenitis (lao hạch), a serious condition that has to be ruled out. A model that rushes to "a sore throat" might miss it; a model that knows how to ask will notice the signs that call for a closer look.
What's in the dataset
Meddies Consultant is divided into four configs, each serving a different training purpose.
| Config | Rows | Patient turns | Doctor turns |
|---|---|---|---|
| English | 109,005 | 826,308 | 930,683 |
| Vietnamese | 58,064 | 329,728 | 386,082 |
| RandomQA | 67,372 | 67,372 | 67,372 |
| RandomQuestion | 61,162 | 61,162 | 0 |
The English and Vietnamese subsets are the core: multi-turn consultations. The ratio of turns to rows shows that each sample is an exchange over many turns, not a single question-and-answer pair.
RandomQA provides ordinary question-and-answer knowledge supervision. RandomQuestion is a special case: it contains no doctor replies at all. With the answers removed, it trains active listening, forcing the model to work out what to ask next instead of producing a reply.
The scope of this release
Meddies Consultant is synthetic data, and it should be used as synthetic data. It is good for teaching a model how to listen, how to gather information, and how to hold a conversation. It is not a standard that replaces a doctor's judgment.
Because it is generated at scale, the data still contains errors. We are releasing the whole dataset so the community can help find and fix them.
This release also does not include the full generation and review pipeline. Treat it as documented training data, not a finished benchmark. Before trusting a model trained on this dataset in a specific clinical setting, validate it again on data from that setting.
What to use it for
The most direct use is as supervised fine-tuning (SFT) data, to teach a base model how to take a history. Beyond that, the data supports two other uses. The first is red-teaming: putting an existing medical chatbot in front of FIFE-driven synthetic patients who are anxious and prone to rambling, to see how it handles a difficult case. The second is training ambient scribes; because the output is already structured, it is well suited to teaching a model to write a reliable clinical brief.
Meddies Consultant is the first step in a clinical-data infrastructure that we will keep building. If a model can reliably take on the charting and the information-gathering, the doctor can spend less time looking at a screen and more time looking at the patient in front of them.
A clinical dataset only matures when real users put it to the test. We hope the community will download Meddies Consultant, bring it into problems of their own, and show us where it still falls short. That is how, together, we build a clinical-data foundation that Vietnamese healthcare can trust.