Question answering is not the wrong task for medical AI. It is the wrong proxy for a consultation.
PubMedQA asks a model to answer biomedical research questions with yes, no, or maybe from an abstract. MultiMedQA combines several medical question-answering benchmarks, including USMLE-style multiple-choice questions. These tasks measure useful abilities. Neither requires a model to notice that the person has left out the one fact that would change the next question.
One release can contain both tasks
Meddies Consultant makes the distinction visible inside one dataset. Its English and Vietnamese consultation configs contain 109,005 and 58,064 multi-turn rows. Each row includes a target disease, a patient persona, and a sequence of messages.
The same release also contains 67,372 two-turn RandomQA examples and 61,162 RandomQuestion prompts with no answer. Those configs are not inferior. They support different experiments: answer generation, retrieval, prompt construction, and focused evaluation.
The consultation configs add state over time. A later turn can depend on what the patient said earlier, what remains unknown, and which concern has not been explored. That is the behavior the data format makes possible. It is not proof that a trained model will use the history well.
Frameworks and phase labels give the conversation a specification
The public card says the release uses patient-centered interview frames such as OPQRST and FIFE. Its review criteria include OPQRST quality, FIFE alignment, completeness, appropriateness, naturalness, empathy, structural coherence, and safety. Previewed generation traces also use visible functions for initiating or opening the consultation, gathering information, providing structure, and closing. The release does not assign those labels to a named framework.
The two documented frameworks and the visible phase labels constrain different parts of the conversation:
- Interview phases mark initiating or opening the consultation, gathering information, providing structure, and closing.
- OPQRST structures symptom details such as onset, triggers, quality, location or radiation, severity, and timing.
- FIFE keeps the patient's feelings, ideas, function, and expectations in view.
That is a design rationale, not a row-level audit. The public repository does not publish the full generator, reviewer setup, or per-criterion results. We therefore cannot say that every dialogue satisfies both frameworks and every phase, or that a label makes a row clinically correct.
No model-level result is published yet
No training result or clinical evaluation on the public card shows that framework-shaped data teaches a model to keep asking or improves patient safety.
A useful experiment would compare models trained on the consultation and QA configs while holding other factors steady. The evaluation would need to score whether follow-up questions are relevant, whether the model preserves details across turns, whether it concludes before key information is available, and whether it handles urgent findings appropriately. Clinical reviewers would also need to assess the consequences of the errors, not only their frequency.
Until that work is published, Meddies Consultant supports a task-design claim: consultation data represents a different supervision shape from question-answer data. It gives researchers material for testing whether multi-turn structure changes model behavior, but it does not answer that question on its own. Train on the consultation when the consultation is the behavior under study. Keep QA when QA is the task. Then evaluate the model rather than letting the dataset name stand in for the result.