Today was the first (and introductory) session of the new VIC FACEM Study Group. (This is the re-branded “James Hayes sessions”). Today Don Liew and Victor Lee ran through the new exam format, with loads of pearls for those preparing to sit in the next 12 months. Victor spoke about the MCQ, EMQ and SAQ, and Don spoke about the OSCE.
Here’s a summary of today’s session. Some of this information is widely known, and available from ACEM and the multitude of online resources, but there are some insider pearls and tips that anyone sitting the exam needs to know.
The main thing I got out of today is that new exam is definitely more geared to what we actually do at work, (compared to the old format), notwithstanding that, it means your preparation must include:
- Facts: the required factual knowledge is the same, and the same textbooks are required. You need to be selective about which blogs and other resources to use to avoid information overload.
- Prioritisation practice: This is a much bigger component of the exam, particularly in the SAQ and OSCE. Ask your bosses how and why they prioritise their decisions, grill each other during your practice sessions, watch other people run resus cases and think about how you’d prioritise your decisions and actions.
- Written question and OSCE practice: The new format is not completely intuitive, and getting used to providing coherent, prioritised and correct answers, under pressure, will take some practice. Do lots of questions, do lots of OSCEs and do all of your WBA’s thinking about how they could translate into an OSCE station.
Written Exam: MCQ, EMQ, SAQ
Written Exam format:
MCQ/EMQ – 3 hours on computer
SAQ – 3 hours answers written in booklets
Questions are written around a theme/Dx/Rx/Ix.
Selection is from a list of up to 20 options
You may select the right option more than once (i.e. for different questions, you may select some of the same answers).
Tip: Read the vignette/stem & try to come up with the answer/Dx first, then look at the list of options. This will stop you getting distracted by the options.
EMQ’s may have props, e.g. blood gas, biochem results, ECG
It’s not clear if they’ll have photos
The SAQ’s are designed to demonstrate integration of knowledge.
How are the SAQ questions written and marked?
Item writers prepare a question and a marking guide.
Examiners review this and can make reasonable changes based on candidate responses.
Examiners will also exercise their judgement as clinical experts when marking for any response that is clinically reasonable but hasn’t been included previously.
This results in a revised marking sheet that is used for all candidates.
So don’t freak out about your answers not exactly matching the pre-defined answers. If it’s clinically reasonable, you’ll most likely get marks for it.
Are all questions worth the same amount of marks?
The previous exam was very focused on fixed timing per question. The new exam is not like this. The new paper is balanced so that you can complete it over the allocated time, and in the last exam, it was very uncommon for candidates to not finish the SAQ in the allocated time.
ACEM is reviewing this process and may make refinements to ensure that candidates can complete their entire exam in the allocated time.
The number of marks per question will continue to be provided as a guide.
Writing your answers:
- Answers should be able to be written in:
- Phrases/short sentences
- i.e. Compared to the old VAQ/SAQ – relatively few words/short phrases are required
- The space provided is the clue to the amount to write – so DON”T WRITE OUTSIDE OF THE LINES
- Try and be succinct – less may be more, don’t say in a sentence what you can say in 1-2 words
- If you accidentally write 4 things in a question that asks for 3, or you want to re-prioritise your answers, you can go back & write:
- Next to each line if you want to re-prioritise.
Prioritising answers in the SAQ is a big issue, and probably the hardest part of the exam.
Some suggested methods to practice your prioritisation are:
- Simply to practice, practice, practice – do lots of questions.
- Grill your study group partners about why they chose to prioritise answers certain ways
- Talk to your bosses at work – ask them why they prioritised decisions the way they did, and how their practical/real life decisions may differ from “exam” answers
- Watch others running resus cases or seeing non-resus patients, and think about how you’d prioritise the management and investigations
Cut the Crap
Another big tip for the SAQ answers was to cut to the chase, and cut the crap
i.e. The generic: “Triage to resus, cat 2, IV/O2/Monitor”
Won’t be necessary as the questions will be way more specific
List 3 drugs
List 4 indications
Tables are also becoming more common:
e.g. list 3 options for ___ & your rationale
Only use them if they are common parlance e.g. Dx/Rx
Don’t use medical terms/colloquialisms e.g. FOOSH
Timing is everything
Timekeeping is hard in the SAQ! Apparently each question should be able to be answered in 6 minutes, even the “longer” questions. Reports from those who sat the last exam would suggest otherwise…
There may be up to 5 parts per question. If it’s a longer question – there will be shorter sections (e.g. 1 word answers/short lists) and longer segments. Technically – all questions should be answerable in 6 mins. So be sure that you don’t get bogged down writing a paragraph for a section that asks for a list, and only has 2-3 lines, when you can see further down the page that there’s a large table, or a section with 5-6 lines for an answer.
Is there reading time?
No. There is no allocated reading time, so:
Tip: Flick through the whole question & check the next page before you start writing to avoid the surprise of extra sections on the next page
Will critical errors kill me?
A big area of concern was: Do you fail for critical errors?
A critical error in one section will score you zero for that SECTION
It depends what else you write for the rest of that whole question whether you fail the whole question, and failing a whole question for a critical error with other sections answered poorly won’t cause you to fail the exam.
Is there negative marking?
There is NO negative marking in the MCQ/EMQ
In the SAQ, answers may still be awarded marks, but may be weighted less if they are reasonable, but not on the official answer list.
Is there a set “pass rate” for the exam?
What’s the standard?
The apparent “standard” is: imagine you are walking into an ED on day 1 as a new FACEM.
Remember, this may involve calling for help!
IMPORTANT – Read the curriculum & know the domains!
You must download and read the ACEM Curriculum Framework.
The number of people in the group today who had looked at the curriculum was alarmingly low. You will find it very hard to pass the OSCE if you do not understand the concept of the Curriculum Framework.
The “medical expertise” domain is the main one for factual medical knowledge, but you need to know all of the domains, and how to approach and answer questions with regard to the domain/s being tested. The OSCEs will simulate vignettes that test all of the domains in the framework.
Each station stem will have the domains being tested on the question sheet/stem, that you can read outside the room, and use to start preparing your approach.
How are OSCE’s made and marked?
16 FACEMs create a list of topics and decide on competencies required, and what sort of clinical scenario it can be tested in. They base it on curriculum framework. Again – it’s the “answer looking for a question” system.
Station then gets workshopped over months, it gets road tested, and it meeds to be fairly done in 7 mins.
Tip: Communication will be in nearly all of them.
When approaching an OSCE station, think:
“What competencies do I need to demonstrate
to show that I’m at an expert level”?
There will be “up to 18 OSCE’s”, each one is standardised to a certain score, you then get a composite score for the whole OSCE. There’s a mathematical method used to determine the overall pass mark, and this is based on “what is the minimum standard” – roughly the level of the “borderline” (read “bordeline pass”) candidates.
There is no set number of stations you need to pass, and no single station will determine your fate for the whole exam.
You need to get out of narrow-focus Registrar mode
The OSCE will test your ability to switch from just managing just your small set of patients in ED, to managing the whole department, and all of the associated complexities this brings.
Most of you get to do this on nightshift, but you need to start getting your head around doing it all the time, at an expert level.
How does Miller’s Pyramid relate to the exam?
The basic test of factual knowledge – tested in the MCQ & SAQ
How to apply the knowledge, may be tested in the SAQ
This was tested in the old SCE, because most of it was just talking not doing!
The new OSCE is designed to get to you demonstrate you know how to DO things.
e.g. teach a med student how to read an X-ray, discussion with an under performing registrar or recalcitrant colleague, refer to an inpatient team, run a resuscitation.
It’s hard as you have to do it with a complete stranger in a foreign environment
This is what you actually do at work. This can be tested in the WBA’s. So practising WBA’s followed by proper feedback would be a great way to prepare for the OSCE.
The OSCEs are very short and you have no feedback! So a lot of OSCE practice is crucial.
Can we throw out Talley & O’Connor?
The OSCE may assess your ability to do a clinical exam, BUT not in the choreographed Talley & O’Connor format. You may still need to:
- Take a focused Hx – e.g. syncope vs seizure
- Do a focused examination related to that history
- Potentially interpret a bedside investigation (e.g. ECG, CxR)
All as part of the same station. Often Hx & Ex will happen together.
Practice and technique are important
- It is an artificial situation, but be mindful of the elements of the question
- Pretend the examiners are NOT THERE unless they are part of the scenario!
- Rememeber 50% of communication is LISTENING!
- Gaming is inevitable
Classic example is a snakebite question: What you say to the patient in the OSCE to demonstrate that you know how to manage a snakebite may not be what you’d actually say to the patient say in real life
i.e. striking a balance between what is appropriate for the patient to hear, vs getting “facts” out for the examiners is tricky
If it’s a difficult patient scenario e.g. breaking bad news
Be sure to LISTEN to what the patient/actor says
Be mindful of your language & choice of words
Learn the OSCE marking sheet!
It is very important to be familiar with domains listed on the OSCE marking sheet, and the global/overall rating at bottom. The domain ratings are converted to a numerical score.
9 + 9 + 9 + 9 for all domains, score = 36
At bottom – this will be a “clear pass”
What are “double stations”
Double stations involve 3 mins reading time, and 17 mins testing time.
Often a resus, or critical situation scenario.
Tip: It will be rare for it to be a procedure/anything messy, due to the reset time required!
May be the “thank god your here” type of situation (i.e. the patient has just arrested)
It won’t be a test equipment that you’re unfamiliar with.
They’re aware that you won’t have scene familiarisation time, but it may go on for a while!
There may be an extra component
e.g. talk to a relative, talk to an RMO about how they performed
There is a good chance you’ll finish early and you can go to the next station, and often there will be a rest station after a double.
Unlike real simulation – you get no feedback!
Timing tip for OSCE Exam
You may elect to use a wristwatch with a stopwatch to help keep track of time during each station. This may be helpful in the 7 minute stations, if you think it will help you keep track of time.
However the actors in the scenarios will prompt you to move along, or will move you to the next topic/section of the station if you are taking too long.
A huge thanks must go to James Hayes for organising these sessions, and hopefully the current group of future exam candidates find it as useful as we all did “in the old days”! I hope you found that run-down useful, and I’ll endeavour to bring you more info from these sessions as they unfold.