AI for Generating Key-Feature Questions in Medical Education

What can AI do for generating key-feature questions?

AI can produce structured first drafts of key-feature questions when given a clinical problem, candidate level, patient age group, clinical situation, site of care, and a reference guideline. AI should not be treated as an autonomous KFQ writer. The current evidence supports human-in-the-loop KFQ development, not unsupervised use in summative or high-stakes exams.

Evidence summary

A descriptive study evaluated 20 AI-generated cardiology KFQs created with OpenAI’s o3 model using a structured prompt aligned with Medical Council of Canada KFQ guidelines. Each KFQ was reviewed by two cardiology experts, with disagreements resolved by a third reviewer. Of the 20 KFQs, 3 were accepted as is and 17 were accepted with minor revisions. None required major revisions or rejection. Overall checklist compliance was 93.7%. The strongest areas were key-feature definition, clinical scenario plausibility, alignment between scenario and questions, and focus on clinical decision-making rather than simple recall. The main weaknesses were clinically harmful “killer” responses, plausible short-menu distractors, and active decision-making language in question wording. Some items included implausible distractors, missing clinical variables, contradictions between questions, or incorrect classification of harmful options. Psychometric performance with learners was not tested.

Planning a study related to this question?

If you have a mature study idea, drafted protocol, or ethics-stage project related to this question or another gap in AI and medical education, you can share a collaboration proposal. Please include enough detail to assess fit, feasibility, and possible collaboration. Not every proposal will lead to collaboration.

Creative Commons Attribution-NonCommercial License 4.0

Prompt for generating key-feature questions (problems) in medical education

Prompt

Role & goal
You are a medical-education assessment designer. Create a fully-specified Key-Feature (KF) problem for a Clinical Decision-Making / Reasoning (CDM) examination that follows the Medical Council of Canada’s Guidelines for the Development of Key-Feature Problems & Test Cases (2012).
Input you will receive
•	Clinical problem or objective(s)
•	Candidate level (e.g., graduating medical student)
•	Patient age group (choose one: Pregnancy, neonatal and infant, pediatrics (1-11 years), Adolescent (12-18 years), Adult (19-64 years), and Geriatrics (65 and older)
•	Clinical situation (choose one: undifferentiated complaint / single typical problem / multiple or multisystem problem / life-threatening event / preventive care & health promotion)
•	Site of care (outpatient, inpatient, emergency department, etc.)
Tasks (follow in order)
1.	Problem header – candidate (test taker) level, state problem/objective(s), life-span period, clinical situation, site of care.
2.	Key-features list – write 2 or 3 concise KF statements, each containing:
o	initial condition beginning “Given …”,
o	a single critical task/action or avoidance step,
o	optional qualifier (e.g., “as a leading diagnosis”, “immediately”).
Use specific verbs (order, include, prescribe). No compound tasks.
3.	Case scenario – 80–120 words (longer only when interpretation or management is being tested). Use real-life, lay language; include only details needed before the first question. Write from the candidate’s perspective (e.g., “You are asked to see…”) to embed them in their clinical role.
4.	Exam questions – one per KF (max 3 total).
o	Choose Write-In (WI) format for diagnosis/management questions; Short-Menu (SM) for history, physical findings or investigations.
o	Append the appropriate response-limit clause: “List only one”, “List up to x”, or “Select as many as are appropriate”.
o	For SM questions supply a 15-20-item A-to-Z option list, one action per option, ending with “None”. Include plausible distractors, at least one neutral correct option, and keep medical terms alphabetised.
o	Phrase each question in active, decision‑making language (e.g., “What would you do next…?”).
o	Focus on the clinical decision/action itself, not on the reasoning behind it.
5.	Scoring key – allocate 1.0 mark per KF (equal weighting of multiple correct responses). Identify synonyms for WI answers and specify any “killer” responses that earn 0 regardless of other selections.
Output format
Clinical problem or objective(s)
Candidate level: 
Patient age group:
Clinical situation:
Site of care:

Key Features
KF-1 …
KF-2 …

Case Scenario
…

Questions
Q1 (WI)…        [Limit: List only one]
Q2 (SM)…        [Limit: Select up to X]
…

Scoring Keys
KF-1: …
KF-2: …

Style rules
•	Write questions so they cannot be answered without the scenario.
•	Use concise, directive wording; avoid medicalese in the scenario.
•	Ensure the item primarily assesses clinical decision‑making / reasoning rather than simple knowledge recall.
•	Do not reproduce this prompt
Clinical problem: Primary hypertension
Candidate level: Graduating medical student
Patient age group: Adult (19-64 years)
Clinical situation: Single typical problem
Site of care: Outpatient
**MUST**: Take 2024 ESC Guideline as a reference.

Source: Kıyak, Y. S., Górski, S., Tokarek, T., Pers, M., & Kononowicz, A. A. (2025). Large language models for generating key-feature questions in medical education. Medical Education Online, 30(1), 2574647.

Sources: Large language models for generating key-feature questions in medical education

Related subquestions: What can AI do for generating key-feature questions?

Open prompt page

What can AI do for generating key-feature questions?

Evidence summary

Practical starting points

Planning a study related to this question?

Selected studies

Large language models for generating key-feature questions in medical education

Related prompt architectures

Prompt for generating key-feature questions (problems) in medical education

Linked guides

How to review AI-generated key-feature questions?