Fallout from the 2015.1 written exam and more (better late than never) ACEM fellowship exam resources

So the 2015.1 written exam happened a few of weeks ago, and perhaps not unexpectedly, there are a lot of disgruntled candidates out there.  You can read more about the specific issues raised by people who sat the exam over at Emergencypaedia.

An Open Letter to ACEM (March 2015)

Dr Edward Reje’s New Exam Verdict

I’d encourage anyone who sat the 2015.1 exam, and anyone who’s planning on sitting the 2015.2 exam to read these posts, and all of the comments.  If you’ve just sat, it may help to know that a lot of people are feeling shafted, and if you are aiming for 2015.2, you may consider delaying once you read how the first of the new exams went.

Annoyed

Some serious issues have been raised, mainly about the SAQ component, but apart from some technical issues like the quality of the written exam booklet, and being able to see each others answers on the MCQ computers, a lot of the deficiencies and potential risks of the exam should have been apparent to those who sat well beforehand. Whilst perhaps you should feel entitled to put your trust in the college, (especially given the large sums of money involved – see the maths by this commenter) the exam was brand new, untested, with minimal practice questions available, essentially no preparation resources (apart from the AFEM course, which, if you were lucky enough to attend, sounds like it was a lot better than the actual exam), conflicting information regarding critical errors, question timing, and timing devices, and (from speaking to several trainees, DEMT’s and FACEMs) a widespread general lack of knowledge about how trainees should be advised to prepare.

Given the glaring deficiencies, and the high stakes involved, I would have had major hesitations about sitting this exam. So kudos to those who gave it a crack, and it’s an utter shame that it failed so miserably to meet what are clearly reasonable expectations of the candidates.

So what is ACEM doing about it?

ACEM has formally expressed regret at the distress being experienced by those who sat the 2015.1 written, in an email sent to candidates a fortnight after the exam. In that email they state the college is ensuring that the exam is being “marked in a fair and efficient way”, and that “candidates should focus on the OSCE in May”.  Is there any solace in this after the fact apology? Probably not much.

ACEM has also released a whole host of new fellowship exam preparation resources, which continues to grow, which can be found here.

ACEM Fellowship Exam Resources

One of these is a brand new online module, (mind-bogglingly released 3 weeks after the 2015.1 writtens) which contains a huge amount of information about, and rationale for, the new exam. It is actually really good, but it’s 6-12 months too late!

You can see the online module here.

In it the process of “standard setting” is mentioned (although not really explained). This is the new way in which the exam will be marked, and I think it will be take at least a year or two for everyone to get their heads around. I’ll post a followup article soon explaining what standard setting is how the new exam pass marks are determined. There’s a lot of other useful information in the module, and I’d encourage all 2015.1 and 2015.2 (and future) candidates to give it a look.

How did it go so wrong?

If it’s any consolation, 2015.1 candidates should know that whilst they feel they’ve been shafted by “the college”, the people responsible for setting the new exam were a combination of non-medical ACEM staff with very strong backgrounds in specialist medical education, and a large group of your FACEM colleagues.  The latter are your bosses and your DEMT’s. They are people you work with on the floor every day. They are people you have drinks with at work Christmas parties and end-of-rotation drinks nights. They just happen to be on exam committees at ACEM. And they are the ones who wrote the questions, tested them, and signed them off as suitable for the new exam.

Yes ACEM is the representative body, and is ultimately responsible for the quality of the exam, but the building blocks were supplied by your workmates.  So in a sense, either partly or wholly, collectively we only have ourselves to blame for the poor quality of the new exam.

Does this mean you shouldn’t complain? Hell no. If I’d sat it, I reckon I’d be furious, and as an angry letter writer from way back, I’m sure would have complained my ass off. But I also wouldn’t be as surprised as many people appear to have been about the way it panned out, given the alarm bells that were ringing loud and clear in the lead-up to the exam.

So the new exam is here, and it’s here to stay. We can only hope that the exam committees take the feedback on board and the OSCE and the next written exam are drastically improved.

In the meantime I’ll endeavour to keep providing whatever resources I can to help future rounds of trainees get through the new, improved-but-far-from-perfect, exam.

 

 

 

 

2 Responses to Fallout from the 2015.1 written exam and more (better late than never) ACEM fellowship exam resources

  1. kannan ramanathan March 23, 2015 at 8:05 pm #

    Dear Andy,

    I am one of the unfortunate guinea pig.

    Every one up to the exams said that no one knew , how the exams were going to be, and even the examiners were in the dark. It had been tested (? )but only with certain number of questions and not the whole paper. So everyone started preparing for the exam through resources available to them from Adelaide, NSW, PAH and AFEM , APEM etc.

    While everyone did their best( all DEMT’s and other FACEM) but what was lacking was clarity of the exam that one would be taking . The secrecy upto the exam was astounding. was the secrecy really necessary? The new video released by ACEM should have been released long time back.

    Looking at the exam pattern and the WBA and rest of changes, it is not new. There is another college in this world, which has been doing the same thing http://www.rcem.ac.uk/Training-Exams/Exams/FCEM
    and one would not have to be a genius to say ACEM did not follow the same structure . So , only if we had known that the exam format is similar to UK exams, then exam materials are available in plenty. our preparations would have been much better and exam stress would have been minimised.

    what was more distressing was some misinformation.

    while examiners and others did not know how the exams would be till few days(?) weeks before the exam, some of them had been advising about writing the SAQ as a consultant , where as the real exam wanted only few words. I would be flabbergasted if one can examine a candidate whether he has consultant perspective by three words?

    My sleeping pattern is in disarray ever since the exam and having night mares about the results that is going to be posted in few days, I am not sure what i would do if the result turns out to be otherwise. should I quit? after all the years of reading, leading upto this final exam? should I take a break and come back when every thing settles? or should I continue with the same tempo and do try again in August? If I clear, then I would not think about this again , ever, in my life. It has given me sleepless nights not only to myself but my family also.

    My two cent worth advice . UK college had been in turmoil. Initially the emergency exam was conducted by a surgical college, Edinburgh. Everyone had FRCS (ED) then. Then there was AFRCS and MRCS and then came MCEM and FCEM. College of emergency medicine UK was formed.These changes all happened in the span of 10-13 years. So they have had tested and learnt few things, but they still have problems. So , while one follows similar pattern , let the ACEM college keep in mind the hurdles UK College is facing and solve that on paper atleast , before testing it on the candidates.

    There are many many Advanced trainees whose native language is not english. I am worried about them as the OSCE is mainly about communication – communicating yourselves for the exam, to juniors, to colleagues and others. you don’t need a crystal ball to see that many non native speakers of english will score badly in this exam. There should be more practice for communication , if one has to pass this exam. Will the college take this into consideration and be mindful of the shortcomings of the large number of candidates who are non native english speakers?

    There are lots of positive things about this exam.
    1. it removed the “techniques”, needed to pass the exam and tests the knowledge.
    2. it is structured. but they have to tweak the time and questions in SAQ, and need to bring a computer version
    3. unsure about OSCE, but if is standardised then they don’t need many examiners in the room,(s) the candidate can be videotaped and a select group of examiners can assess the candidates performance by looking at the video. It can also serve to tell the candidate what he had done wrong and can help him to improve his skill for next exam if he does not succeed.
    4. it removes the bias. bias of handwriting( if computer version for SAQ ) and removes fatigue of examiners which may inadvertently affect candidates scores.

    While I am appreciative of the huge task the college comittee members had taken, I ask for the college to be sympathetic towards the candidates who had boldly taken the first exam.

  2. miss_marple March 24, 2015 at 11:19 am #

    Looks like a disastrous exam, thinking of taking it in August, might be better off to take it Feb next year, wait until the dust settles and ACEM fix all the teething problems

Leave a Reply