ACEM2011 Day 2

EM Training in USA – Susan Promes
Susan gave a nice succinct picture of the history, current state and future of EM training in the USA:
1971 U.Cinicinatti 1st training program, 1979 Recognised as 23rd specialty

At UCSF this year there were 12 Residency (Registrar) spots, 750 applications!

NO MANDATORY RESEARCH PROJECT in the US programs!

There’s Allopathic (traditional) vs Osteopathic Programs

Programs need 1 board certified Emergency physician for every 3 trainees, only 28hrs clinical per week for specialists, so there’s free time for teaching & research. BUT: you need to be publishing as a faculty to maintain accreditation.

5 hours protected didactic teaching per week for trainees – same as here

Number of clinical hours per week = max 60hrs per week
In external rotations = 80 hrs per week max
1 day in 7 must be rostered off

Regulation of hours comes from pressure from public: Hours are tightly regulated, and closely monitored by accreditation body (ACGME)  that accredits programs, and they can cancel your programs accreditation if your residents do too many hours. They have other good things like taxi vouchers to get home

Move towards Competency based assessment – these apply to all specialties
Patient care, medical knowledge, Communication, Professionalism. System based practice (eg when to get a consult how to follow up patinets), Practice based learning & improvement (eg self reflection)

Future:
Milestones Project: Expansion of general competencies/outcomes
EPA: Entrustable Professional Activities: What should your specialists be able to do?

Program accreditation by ACGME:
Requires:
Program Information form: – 150-200 pages about your program/teaching
Surveys: Residents & Faculty
Site Visit
Board passage rate

Can be accredited up to 5 years, may be moving up to 10 years, possibly more.

Board Exam: 2 types
Allopathic: ABEM (min 3yrs training before you sit)
AOBEM (Osteopathic) (min 4 yrs training before you sit)

Then: Continuous Certification
Professional standing
Mini exam every year
Big Exam every 10 years
Assessment of practice
25hrs per year of CME

Terry Brown – Training in UK
Terry gave an entertaining talk on the UK training program. He highlighted the fact that it is hugely over-assessed, insanely inflexible, and the complexity leads to a huge amount of paperwork.
Average attendance of UK ED’s = 60,000.  Average no of specialists per ED = 4.9 huge undersupply.
Projected need is much greater, but it’s going to take another 10-15 years to reach this.

As of 1995, you could meander into ED via being in college of surgeons/physicians, then have some “experiential learning”…
Now: Primary exam, Exit exam and some bloody complicated things to tick off along the way.

Next year: 1st cohort of ED physicians coming out who’ve only known the 4hr rule! (from internship  to specialist)!

Different paths into EM training, similar to here, PGY1 &2, then 6 years of training.  BUT by European law you can apply to be recognized as a specialist without doing any training based on experience, and any CMO can sit the exam without having done the training program.

Competitive: 10 applicants per position. Once you’re in, you’re in. Hard to move around, hard to work part time – almost impossible

College Curriculum: 300+ pages
Formal teaching in non-clinical skills, Mandated Courses: ALS, APLS, etc, Web-based modules

Assessment:
Exams & workplace
MCEM-A = first part 2hr MCQ
MCEM-B = clinical scenario & data interpretation – written
OSCE: clinical, communication, conflict resolution

Highly variable pass rates: initially 20%, now around 70%

2nd part/Exit exam
FCEM
Academic, Management & Clinical parts:
Must pass each section, <50% pass rate, 35% for first go. Academic Section:
Critical Appraisal SAQ
Clinical topic review (3000 words, summarise/analyse, then actually implement change in your ED, and re-assess how that went!)  = Most commonly failed section, makes the 4:10 look easy…

Management:
2 x 15min vivas & a long case
3 recommended texts JUST FOR THAT SECTION
Moves are afoot to bring this forward and get this out of exit exam. BUT this will make it a 7-module with projects for each, covering a huge range of non-clinical management-waffle that looks like  a nightmare.

Clinical
SAQ’s: 20Q’s 2 hrs
16 OSCE’s

PLUS:
Workplace based assessments, Summative & Formative
12 different types,: Eg: Mini-CEX, DOPS, Portfolio,… lots
Insanely detailed, wordy & complicated.
Different ones for
Anaesthetocs
Crit Care
Paeds
Ultrasound

EM: Total = 27 assessments in 6 months, less in other rotations.  This generates a HUGE workload for the ED consultants, and Terry quoted a current British EM Physician who basically said “we glance at them, if they look like their mum filled them out or they’re written in crayon, we have a closer look, otherwise we just pass them”!

PLUS:
30 E-learning modules

Surprisingly, despite all of this: 79% satisfaction rate from surveyed trainees (with 47,000 out of 54,000 responding to survey!)

Future:
Debulk FCEM – bring components forward, eg management portfolio

In summary, EM training in the UK is  a complex mix of absolute rigidity & constant change. ACEM can learn a lot about what has & hasn’t worked when designing their program.

Question from audience: If satisfaction is really 80%, why are there so many who move to Australia?! Answer: 20% of 2800 is a reasonable number (1 in 5). But many don’t know anything different, and given the inflexibility and over-assessment they may need some enlightenment to see what a training program could be, and satisfaction rates may drop! The excellent ability of UK trainees to fill out forms was acknowledged!

The panel likened the detailed curriculums in the USA, UK and Australia to measuring every leaf on the tree, and losing site of what the tree looks like.

The pressure to get into training programs overseas leads to embellishment of CV’s, and training program directors are checking up on people on facebook, and other mediums to make sure they’re legit!

The continuum of learning & training, even after passing final exams was acknowledged, and that the US guys get 6 years of experience in a 3 year training program, and they do nights and 80hrs a week as Consultants, and in the UK new Consultants may be more competent than newly graduated Aussies, but within a few years you won’t find much difference between the 3 nationalities.

The ED Exam opinion: There’s no such thing as a perfect training program, and given the flexibility of post-fellowship work, the lack of nightshifts for consultants here, and the low cost (both financial, administrative, and lifestyle), the ability to essentially choose where you work as a Registrar and the lack of competition for ED training places, ED training in Australia is not so bad! We can only hope the College learns from the “mistakes” of the US & UK, and doesn’t make things more complicated, over-regulated, less flexible as may be the temptation.

Apologies – I didn’t get to summarise the excellent talk on EM training in Nepal by Chris Curry, as I was editing the talks about the UK/USA which I felt may be more useful to Aussie ED trainees.

 

NOW THE SESSION YOU’VE ALL BEEN WAITING FOR:
The discussion of the Australasian training program. For the first time I was actually fairly impressed from what I saw.  There are still flaws and inconsistencies, but for the first time since I signed up as a baby registrar nearly 10 years ago, I was actually impressed by the Colleges committment to improving the training program with real responses to real feedback from its members.  They should be applauded for putting their money where their mouth is.  Will they pull it all off? Remains to be seen, but you can bet not many other specialty Colleges are putting this amount of effort in. Cheers ACEM!

The Australasian Training Program
Yuresh Naidoo (Censor in Chief) & Mary Lawson (Director of Education) ACEM

TARWG: 18month review process:
Analysis of what makes a good FACEM, how many are needed, ensure training remains contemporary, and remains accountable to registration bodies and public.

What was identified in the review?
Clinical relevance of primary exam – yes it’s needed, needs to be more clinically relevant
Quality & quantity of clinical training
Timing of clinical assessments
Progressive nature of training: teach & assess some things earlier on, rather than assess it all in a final exam.
Many skills are needed earlier on (eg VAQ/SAQ interpretation skills)
Provision of educational resources: modules, teacher training, online resources. Looking to fill gaps in current online resources rather than provide everything trainees may need.
Emphasis on non-clinical domains of practice: eg cultural safety, many others.

Move toward outcome based learning, with a structured set of learning outcomes

Curriculum framework – adapted from CanMEDS framework

CanMEDS

Recommendations:
Which bits of training should be retained, changed (& how), what new elements need to be developed/introduced?
General aims –  see ACEM website

General Aims (from review) Many were generic, “best practice” blah blah, but a couple were exciting, in particular:
Move of College to increase role as an educational provider – this is a new philosophy for the College, long awaited & welcomed (but questioned by some in  the discussion – see below)
Move to online education where possible
    Online learning resources
    Online learning portfolio
    Eg: Paeds log book
Increase clinical component of primary exam
Formal WBA’s (workplace based assessments) in advanced training, & revision of the Fellowship exam to take these WBA’s into consideration.
Increase provision of clinical teachers training: has started for the non-specialist course, looking to expand this
Smooth transition between current & revised program.

Curriculum Revision Project: CRP – will be happening based on recommendations of review
Lots of work still to be done. Phased implementation over at least 5 years (2012-2016) to spread cost & workload

Andrew Perry
Trainee Committee Chair, and has experience with AMC & AMA, so has a lot of knowledge of ACEM & other Colleges training programs.  Andrew gave a well informed and entertaining insight into the ACEM training program highlighting some of the ways ACEM is leading the pack as a specialist College in Australia/NZ.

MTRP (medical training review panel – government body) stats:
Total ACEM college trainees (2010): 1684 ED trainees = 3rd biggest after college of physicians & GP’s
2/3 of ACEM members are trainees = 2076 (Nov 2011), 43% in provisional training.  Number of trainees is going up faster than number of fellows.

So what are we doing right? (as people are voting with their feet)
Accessible entry – essentially no barriers!
   Desire & interest required, & not sociopathic/grossly incompetent
Flexibility: GP’s are close, but we are the most flexible college/program
   Eg: Part time vs full-time
Timing, frequency & quantity of assessment (especially primary exam – although this is going to change – see discussion below)
Content fairly flexible: non-ED time is very flexible
Clear direction on what to learn: matrix/curriculum/texts – compared to other colleges (eg physicians) ACEM is not so bad, feedback is quite good as well (in both directions)
Generous recognition of prior learning compared to other colleges
Relatively flat hierarchy: when you’re at work, you see the boss, all shift, and culture of accessibility of bosses (compared to a physician/ICU consultant who the Registrars only see for ward rounds twice a day!)
Reasonably priced! Only one cheaper is College of rural & remote medicine!

Not many colleges accept/actively seek feedback from trainees, compared to ACEM OTA’s. ie you MUST fill out the Trainee Term Feedback form = 100% response rate (compared to 80-odd% in UK!)
Skewed in that trainees tend to over-rate their ED in these feedback forms. So don’t be shy, if your ED isn’t performing, say it in your feedback, the College takes notice.

Improvements wanted by trainees:
Assessment: more clinically relevant, when/how/what is assessed
Increased role of College as education provider for trainee, including non-clinical aspects.
Move to online education

2013.2
Will have to pass all 4 MCQ’s to get through to viva
Tradeoff:
College will make primary more clinically relevant, especially in vivas
Will be more focused on things you’d actually see at work

Overall a great reflection that despite the perceived difficulties, on a local & worldwide scale the ACEM program is pretty good, and is interested in taking onboard members opinions, having high standards, and striving for what is best for all of its members.

College needs to be careful about WBA’s (workplace based assessments), over-assessing trainees and over-working fellows by burdening them with more and more forms to fill out for trainees. Tradeoff is that it may lessen the burden of the fellowship exam.

Increased role of ACEM role as education provider:
Content, Resources, Assessments.  Regardless of location trainees will be able to access the same content.
EDExam applauds this philosophy, as the discrepancy in access to training material around the country is one of our driving motivators

E-learning:
Click here, log in and look up Emergency Medicine Certificate Learning Materials – some very good topics & resources available, and we’d strongly suggest that ED trainees check these out.

ACEM now has facebook page  & Twitter

Andrew gave a shoutout to LITFL! & @sandnsurf acknowledging that the College couldn’t (& really shouldn’t) try & replicate what they do.  He also flashed a screenshot of the #acem2011 twitter feed, (that we’re proud to say we at EDExam started!)

Bob Dunn:
Trainee research requirement:
For one of only 3 mandatory requirements, the 4.10 component had a lot of inadequacies, and holes in assessment (Bob described it as “a fairly rubbery marking system”) until recentl
y. A considerable amount of work has been done to plug some holes in what was a very leaky boat…

4.10 Evaluation/Review – based on survey results
56% response rate. 80%  believed 4.10 had some benefit! (Can you believe that number? Personally I’m struggling with it…)
Limitations of 4.10
Poor quality of research – teaching bad habits
Lack of ED support (time & supervision)
Time consuming
Marking inconsistent
No specific criteria
Not perceived to improve research understanding.
Crap feedback for failures
As well as many other problems…

Feedback:
46% wanted uni courses introduced
27% wanted workshops (run by college)
20% wanted online resources

College has taken this onboard, and now has the “alternative pathways” (38 different ones in fact!)
Criteria based assessment
Crieria are defined
Same process for all trainees
Specific written feedback for all failures (to supervisors, trainee & adjudicators)
Online content – loads of guidance on how to successfully get through your 4.10

NOW: 2/3 of people are doing alternative pathway! Likely to go up in future…
Reduction in pass rate for TRP’s: 93% – big drop initially – now up to 85%
Mainly drop in pass rate was for published papers, because they didn’t meet all criteria (could’ve passed with supplemental information that satisfied criteria, although ACEM admits they probably didn’t make it clear enough). More explicit instructions have been provided.

Future issues:
Are the standards too high? Which components should change?
Purely qualitative research isn’t supported by the current model (because it’s not as applicable to our type of work)
Should the scientific method be mandatory?
Is the system too rigid?
Is it too time consuming? Research vs clinical & other administrative demands (including looming WBA’s…)
Does ACEM want quality or quantity?
Is alternative pathway too easy (or too hard…?) compared to TRP.
ACEM believes the work for both is about equivalent.

Future goals
Improved understanding of research
Increased quality of research
Increased capacity to supervise research
Critical Lit reviews are part of curriculum review & are likely to be introduced.

ED Exam says: Best advice – read ALL of the online material before you embark on a pathway to get your 4.10 so you are well informed about what the criteria are, and so you don’t waste your time doing it wrong.

Discussion:
A lively discussion followed, including questions from our hero @cliffreid of Resus.me fame…

Indiginous health will form part of the new curriculum framework (initiated by College but also there’s an AMC requirement to include this)

Are we risking alienating people or dampening individuality by homogenising training program and making curriculum too prescriptive? Bob’s response was that there are certain key skills eg Communication, that you must possess to function in our job, and we shouldn’t feel bad about excluding people who don’t “meet the grade”, and there are NO personality trait criteria in the assessment/selection guidelines.

Question about passing both parts of fellowship exam together vs individually.
Response:
One thing ACEM is doing is re-adjusting content of fellowship exam, bringing some parts forward into training time
Training committee has considered it several times over recent years
College says: it’s an exam in totality. You need all competencies at same time. 
Trainee committee would prefer to see it split up, and holds hope for this happening in future.

A trainee suggested video-taping valuable presentations such as this one and distributing them online, response form Andrew was “follow the twitter feed”! (#acem2011)

Peter Cameron congratulated College on bringing in alternative pathway for 4.10
Concern about provision of consistent, standardized & regularly updated educational content online produced with voluntary labour – he thinks they should focus on assessment, & just tell people where to access information
Repsonse: ACEM will focus on filling the gaps in online resources
Trainees will give cred to ACEM-endorsed pathway
Commonwelath funding for Non-specialist pathway resources, which are available to all, and Andrew re-iterated that trainees should make use of these: – look for Emergency Medicine Certificate Learning Materials

Cliff Reid:
What is the purpose of 4.10? What is end-point? Most fellows aren’t active in research, but all need to critically appraise research.
Critical appraisal is far more useful than going through 4.10
Will they rename the research project the “alternative pathway”?
Bob’s response:
Doing research gives you insight into practical difficulties as opposed to just reading a book about it
But: the alternative is there now…
Critical Lit review – is being looked at as being a separate component of new curriculum.
Andrew’s comment:
70% of trainees are doing alternative pathway, expected that will increase to 80-90%.  Why? It’s a known quantity, with clear outcomes, provides certainty, compared to TRP’s which can drag on forever and you’re not guaranteed of passing.

Another question from floor:
Should some of the 18 months of non-ED time be allowed to be a research rotation? It will allow those really interested to go ahead and do it?
Response: it’s already available and several people have done it.

Ruth Hew: Summary
Keep in view the tree while examining the leaves
Need for accountability: both educational and beaurecratic is increasing
Hot news: 2013.2 Primary:
If you haven’t done all 4 subjects by then, you’ll need to re-sit all 4 VIVA’s at once regardless of how many you’ve already done (you don’t have to re-sit the passed MCQ’s).

Phew, what a densely packed morning at ACEM2011, all that before lunch!!
We’ll endeavor to keep the live stream flowing over the next couple of days.

Any feedback? Quickest way to get onto us is via Twitter @edexam