So the 2015.1 written exam happened a few of weeks ago, and perhaps not unexpectedly, there are a lot of disgruntled candidates out there. You can read more about the specific issues raised by people who sat the exam over at Emergencypaedia.
I’d encourage anyone who sat the 2015.1 exam, and anyone who’s planning on sitting the 2015.2 exam to read these posts, and all of the comments. If you’ve just sat, it may help to know that a lot of people are feeling shafted, and if you are aiming for 2015.2, you may consider delaying once you read how the first of the new exams went.
Some serious issues have been raised, mainly about the SAQ component, but apart from some technical issues like the quality of the written exam booklet, and being able to see each others answers on the MCQ computers, a lot of the deficiencies and potential risks of the exam should have been apparent to those who sat well beforehand. Whilst perhaps you should feel entitled to put your trust in the college, (especially given the large sums of money involved – see the maths by this commenter) the exam was brand new, untested, with minimal practice questions available, essentially no preparation resources (apart from the AFEM course, which, if you were lucky enough to attend, sounds like it was a lot better than the actual exam), conflicting information regarding critical errors, question timing, and timing devices, and (from speaking to several trainees, DEMT’s and FACEMs) a widespread general lack of knowledge about how trainees should be advised to prepare.
Given the glaring deficiencies, and the high stakes involved, I would have had major hesitations about sitting this exam. So kudos to those who gave it a crack, and it’s an utter shame that it failed so miserably to meet what are clearly reasonable expectations of the candidates.
So what is ACEM doing about it?
ACEM has formally expressed regret at the distress being experienced by those who sat the 2015.1 written, in an email sent to candidates a fortnight after the exam. In that email they state the college is ensuring that the exam is being “marked in a fair and efficient way”, and that “candidates should focus on the OSCE in May”. Is there any solace in this after the fact apology? Probably not much.
ACEM has also released a whole host of new fellowship exam preparation resources, which continues to grow, which can be found here.
One of these is a brand new online module, (mind-bogglingly released 3 weeks after the 2015.1 writtens) which contains a huge amount of information about, and rationale for, the new exam. It is actually really good, but it’s 6-12 months too late!
You can see the online module here.
In it the process of “standard setting” is mentioned (although not really explained). This is the new way in which the exam will be marked, and I think it will be take at least a year or two for everyone to get their heads around. I’ll post a followup article soon explaining what standard setting is how the new exam pass marks are determined. There’s a lot of other useful information in the module, and I’d encourage all 2015.1 and 2015.2 (and future) candidates to give it a look.
How did it go so wrong?
If it’s any consolation, 2015.1 candidates should know that whilst they feel they’ve been shafted by “the college”, the people responsible for setting the new exam were a combination of non-medical ACEM staff with very strong backgrounds in specialist medical education, and a large group of your FACEM colleagues. The latter are your bosses and your DEMT’s. They are people you work with on the floor every day. They are people you have drinks with at work Christmas parties and end-of-rotation drinks nights. They just happen to be on exam committees at ACEM. And they are the ones who wrote the questions, tested them, and signed them off as suitable for the new exam.
Yes ACEM is the representative body, and is ultimately responsible for the quality of the exam, but the building blocks were supplied by your workmates. So in a sense, either partly or wholly, collectively we only have ourselves to blame for the poor quality of the new exam.
Does this mean you shouldn’t complain? Hell no. If I’d sat it, I reckon I’d be furious, and as an angry letter writer from way back, I’m sure would have complained my ass off. But I also wouldn’t be as surprised as many people appear to have been about the way it panned out, given the alarm bells that were ringing loud and clear in the lead-up to the exam.
So the new exam is here, and it’s here to stay. We can only hope that the exam committees take the feedback on board and the OSCE and the next written exam are drastically improved.
In the meantime I’ll endeavour to keep providing whatever resources I can to help future rounds of trainees get through the new, improved-but-far-from-perfect, exam.