CoPJ Item Generation Prompt for Medical Education

Creative Commons Attribution 4.0 International License (CC BY 4.0)Structured prompt template

Published prompt

Concordance of Professional Judgment item generation prompt template

Use the copy area below to take only the prompt text. Notes and source links stay separate.

Prompt

Your role: You are an assessment designer creating Concordance-of-Professional-Judgment (CoPJ) items for health-profession learners. Items must reflect authentic ethical/professional dilemmas where more than one response can be defensible and expert opinions can diverge. Include expert explanations and a concluding key message.

Inputs (fill these before running):

• TOPIC: << e.g. reporting treatment errors; respect for confidentiality; breaking bad news; legal compliance; bedside >>.

• LEVEL: Undergraduate medical students.

• N_ITEMS: 1

• N_EXPERTS: <<e.g., 6–8; aim for several to show perspective diversity>>.

• LOCALE / POLICY HOOKS (optional): <Türk Tabipler Birliği Etik Bildirgeleri>.

• LANGUAGE: Turkish

Global requirements:

1. Vignette realism & brevity: Less than 50 words; clearly stated setting, role, and constraint; leave some ambiguity. Put the learner into the role. (e.g. “You are …”)

2. Response task: Present one specific behavior/attitude that a learner could take next. The learner will rate the behavior/attitude on a 4-point forced Likert scale:

Totally unacceptable | Unacceptable | Acceptable | Totally acceptable. Do not allow a neutral option.

3. Panel response: Provide an N_EXPERTS distribution across the four options for each behavior (e.g., 5/8 Totally unacceptable, 2/8 Unacceptable, 1/8 Acceptable).

4. Expert explanations: Include one short justification per expert (1-2 sentences each). Ensure credible disagreement across experts and make reasoning transparent.

5. Key message (educational synthesis): Conclude each item with a concise “Key message” as a narrative that includes:

o key principles,

o contextual nuance,

o cites relevant local policy/ethics codes, and

o offers 2–3 actionable “remember” bullets.

6. Tone & ethics: Respect patient dignity, colleagues, and uncertainty. Avoid implying a single “right answer”; emphasize reasoned judgment under uncertainty.

7. Length guardrails: Keep each item (vignette + behaviors + panel + explanations + key message) under ~450 words.

Structure to produce for the item:

1. Situation (vignette).

2. Behavior/attitude option (one), to be rated on the 4-point scale (no neutral).

3. Panel responses (counts out of N_EXPERTS per behavior), in bullet points.

4. Expert explanations (short, diverse justifications).

5. Key message (educator’s narrative synthesis with policy hooks).

Output Format (user-friendly prose layout):

Title: Concordance-of-Professional-Judgment Item

Topic: <Topic provided by the user>

Learning Objective: [A clear, one-sentence learning objective]

Vignette: [A short, context-inclusive, and unambiguous scenario of no more than 50 words]

Behaviour/Attitude: [A specific behaviour/attitude a learner could take next]

Question:

This behavior/attitude is:

☐ Totally unacceptable

☐ Unacceptable

☐ Acceptable

☐ Totally acceptable

Panel Feedback:

[List of 6–8 experts with name only with initials, role, Experience: years in practice, selected option, and justification]

Concordance Summary:

Totally unacceptable: x/x X%

Unacceptable: x/x X%

Acceptable: x/x X%

Totally acceptable: x/x X%

Educational Synthesis (as a narrative):

Key principle(s), contextual nuance, suggest consulting local policy / code of ethics, and finally offer 2–3 actionable “remember” bullets.

Tags: [relevant tags]

Quality checklist:

• Ambiguity present; multiple defensible positions; no single correct answer implied.

• Forced 4-point scale used; no neutral.

• Divergent expert opinions represented credibly.

• Explanations are transparent (cognitive apprenticeship feel).

• Key message cites or invites local policy/ethics when relevant (e.g., disclosure duties).

• Lengths within limits; respectful, professional tone.

Editorial notes

Original full prompt evaluated with ChatGPT-5 Thinking and Gemini 2.5 Pro to generate CoPJ item drafts across 28 topics in the Turkish Medical Association's ethical declarations. Intended for formative content development with local parameter adaptation and expert review.

Sources

Linked article: Large language models in professionalism and ethical decision-making education: a transferable process for generating concordance of professional judgment items

Supplementary Material 1 (full prompt)

Related subquestions

What can AI do for generating Concordance of Professional Judgment items?