Staff Competency Assessments

[EpiEMS]

My agency is currently reviewing how we do a recurring clinical & physical competency assessment (think skills test & CPAT). How does your agency reassess clinical competency and physical ability to do the job? Do you do an annual skills reassessment?

 

[Aprz]

We do quarterly airway assessments. We'll do an adult, pediatric, and infant intubation. It'll usually be a scenario where you'll eventually intubate the patient. They'll usually have a couple of pointers or questions that aren't related to airway so that's good. I don't feel like it's really a test, but just a way to talk.

Once a year, we'll do an adult, pediatric, infant, and high risk OB scenario. It sounds like it used to be a test and you could lose your job over it, but since I've been here, it sounds like they are a lot more relaxed with it. I think it's going the same route as the quarterly airway assessment, just trying to talk and give pointers.

I think the thing I hate most with a lot of these things is that I feel like people are just going through the motion, trying to get this stuff over with, and not really assessing or teaching. That's how it feels to me. They are just pushing us through and checking off the box that we did it.

 

[DesertMedic66]

We do quarterly airway assessments. We'll do an adult, pediatric, and infant intubation. It'll usually be a scenario where you'll eventually intubate the patient. They'll usually have a couple of pointers or questions that aren't related to airway so that's good. I don't feel like it's really a test, but just a way to talk.

Once a year, we'll do an adult, pediatric, infant, and high risk OB scenario. It sounds like it used to be a test and you could lose your job over it, but since I've been here, it sounds like they are a lot more relaxed with it. I think it's going the same route as the quarterly airway assessment, just trying to talk and give pointers.

I think the thing I hate most with a lot of these things is that I feel like people are just going through the motion, trying to get this stuff over with, and not really assessing or teaching. That's how it feels to me. They are just pushing us through and checking off the box that we did it.

The old yearly testing would absolutely take clinicians off the flight line and place them either back in indoc, back on a training status, or on a PIP (Personal Improvement Plan)/PEP(Personal Education Plan).

You were able to fail one scenario and still pass. If you failed 2 you would go on on the the options listed above and then have to retest. If you failed again you would typically be let go.

 

[Tigger]

Our paramedics all "test" with the medical director annually, and new paramedics test as soon as they get their card. This is done via some scenarios (usually a megacode and a trauma) and some skills are thrown in. BLS providers test on waived skills yearly with the hospital EMS coordinator. The medic testing used to be pretty stressful but like the others, it's more of a sit and talk situation now. We have a pretty small group with extensive hospital QI so this isn't where issues are really noted.

Our only yearly fitness test is the Pack Test, which you're supposed to have to pass but that seems to have faded away. We are trying to switch a met test equivalent as it seems less physically abusive for some of our staff. Make sure you are using a validated physical test if you're going to implement one, departments have gotten themselves into trouble with "homemade" tests.

 

[Aprz]

The old yearly testing would absolutely take clinicians off the flight line and place them either back in indoc, back on a training status, or on a PIP (Personal Improvement Plan)/PEP(Personal Education Plan).

You were able to fail one scenario and still pass. If you failed 2 you would go on on the the options listed above and then have to retest. If you failed again you would typically be let go.

Mhmm, but are they still doing that? It doesn't seem like it.

 

[DesertMedic66]

Mhmm, but are they still doing that? It doesn't seem like it.

No. This last year, 2021, was the first year they changed the entire process. It just used to be a 4 hour day of you come in test and then go home. Now it is the 8 hour day of everything they did, Multiple scenario based patients that were not treated as a pass or fail but rather a learning experience and some difference activities.

It sounds like the feedback was very favorable so they are going to continue with this style for now but things may change as there is a lot of tension right now between the educational team.

 

[NomadicMedic]

As part of the PA Rules and regulations, we have annual required competencies that each provider must perform. (https://www.health.pa.gov/topics/Documents/EMS/EMSAct_37_Chapter 81.pdf)

The specifics of the competencies are left open to the Medical Director, but usually include basic and advanced airway management, a set of cardiac skills including a megacode, some pedi med math, trauma management and an OB station. I’ve always run my annual review sessions as a day long set of evolutions mixed with small lectures and providers perform the competencies through a set of rotating skill stations.

 

[DrParasite]

To be honest, it's a joke. We are required to do annual competencies, but no one ever fails. Do we have horrible providers? yep.... do they complete annual competencies? yep. I convinced they are just paperwork to CYA in case a negative outcome happens.

Do I want anyone to lose their job? not really, but I am also in favor of raising standards, or at least making sure people know enough to do that meet the actual standard of care.

Is it easier to coast by when your agency is BLS only? I guess. ALS providers are held to a higher standard (and they should), but we all know paramedics who are only still paramedics because they have known the evaluator for 20 years.

To be honest, I wish all of our staff competency assessments were outsourced to a third party, similar to NREMT practical examinations. The evaluators don't know you, but their job is to evaluate everyone objectively, to ensure they are doing the job as expected, individually, without the help of a crew or partner. Everything is rigorously documented (both the good and the bad), and if you don't pass, you get remediated. if you don't pass a second time, you are suspended for 30 days. if you don't pass a third time, then this job isn't for you. But that would be a decision made above my paygrade

 

[MMiz]

We didn't have any sort of annual assessments when I worked in private EMS.

I know Wake County EMS requires an annual physical ability test.

 

[NomadicMedic]

We have remediated providers based on annual review performance and at least one was terminated. It’s a legit review of competency.

 

[mgr22]

I preferred a combination of daily, randomly sampled case reviews plus periodic (monthly or quarterly) quantitative evaluations based on reproduceable field data.

Most places I worked offered only qualitative annual reviews based on contrived numerical scales that were almost impossible to back up with data. Two ridiculous shortcuts were (1) having employees rate themselves, and (2) expediting past-due evaluations by automatically grading employees 3 on all 1-5 scales.

 

[NomadicMedic]

I preferred a combination of daily, randomly sampled case reviews plus periodic (monthly or quarterly) quantitative evaluations based on reproduceable field data.

Most places I worked offered only qualitative annual reviews based on contrived numerical scales that were almost impossible to back up with data. Two ridiculous shortcuts were (1) having employees rate themselves, and (2) expediting past-due evaluations by automatically grading employees 3 on all 1-5 scales.

Only takes one of those to come to light in the deposition to show that there is no verifiable method to determine employee competency.

 

[mgr22]

Only takes one of those to come to light in the deposition to show that there is no verifiable method to determine employee competency.

Yep. And from a management perspective, quantitative ratings made it easier to justify higher or lower merit pay raises and promotional opportunities.

Like many of us, I learned what to do and what not to do by observing the people I worked for.

 

[NomadicMedic]

Yep. And from a management perspective, quantitative ratings made it easier to justify higher or lower merit pay raises and promotional opportunities.

Like many of us, I learned what to do and what not to do by observing the people I worked for.

That sounds like an awful place.

 

[EpiEMS]

Thanks all very much for these suggestions! I have recommended:

• An annual written (protocol) test
• Integrated out of hospital scenario-style assessment, covering both clinical & operational skills (specific clinical skills TBD based on task/practice analysis), including physical ability (e.g., lifting/moving)

I think that is probably sufficient, anything missing?

 

[EpiEMS]

So it sounds like we may be leaning away from pushing an assessment and focusing on more training. I personally don’t love that model - hard to know what people are doing right/wrong without a physical, cognitive, and clinical assessment standardized across everybody.

 

[ffemt8978]

So it sounds like we may be leaning away from pushing an assessment and focusing on more training. I personally don’t love that model - hard to know what people are doing right/wrong without a physical, cognitive, and clinical assessment standardized across everybody.

Unfortunately this seems like the natural progression of using computers to teach and even test in some ways.

 

[EpiEMS]

Unfortunately this seems like the natural progression of using computers to teach and even test in some ways.

I'd be ok with a mix of live and online training, but no in person clinical test just concerns me.