Standardized STEMI Identification Assessment

[Christopher]

I've worked on putting online a standardized STEMI Identification Assessment using 36 ECG's from a survey-based study on STEMI identification by physicians. It originally utilized a Google Form and would email you a huge 9MB PDF with your results. I've now converted it over to a Google site as a proof of concept.

I would be much obliged if folks could take this assessment and see what bugs, if any, are present. I know it works in IE9, IE10, Firefox, and Chrome; but that's all I've tested it in.

After you take the test it will tell you your accuracy, sensitivity, specificity, and how you compared versus computerized interpretation, emergency physicians, and cardiologists. It'll also tell you what sort of ECG's you miss ("subtle", "LVH", etc) and whether they cause you to make false positives or false negatives. It also lets you know whether your answer for each ECG was consistent with the physician's answers.

Your email address is optional, but it can be helpful if you encounter a problem for me to look up your results.

Thank you for your time!

Take the test here, see if you find any bugs, and see how you stack up against man and machine!

 

[TomB]

Good Lord, Christopher! There's not a straight forward ECG in that entire test. Each was was difficult, IMHO. I would not feel comfortable calling any of them a STEMI without clinical correlation. My accuracy: 83% with 2 false positives and 4 false negatives (I answered conservatively). You could see many of these STEMI-like ECGs with other diagnoses (e.g., sepsis, etc.). I think paramedics and physicians alike would have extreme difficulty with this test.

Tom

 

[Christopher]

Good Lord, Christopher! There's not a straight forward ECG in that entire test. Each was was difficult, IMHO. I would not feel comfortable calling any of them a STEMI without clinical correlation. My accuracy: 83% with 2 false positives and 4 false negatives (I answered conservatively). You could see many of these STEMI-like ECGs with other diagnoses (e.g., sepsis, etc.). I think paramedics and physicians alike would have extreme difficulty with this test.

Tom

I enjoyed the test when I learned all 36 ECG's were actual STEMI activations (which I take to mean somebody thought adequate clinical correlation existed).

 

[Handsome Robb]

I did better than I thought after seeing those ECGs....shoulda known when Tom said something that I was in for an adventure.

78% accuracy, sensitivity 71% and specificity of 92%
1 false positive and .... 7 false negatives.

I agree about it being difficult without any sort of clinical correlation.

Thanks for this Christopher!

 

[Ewok Jerky]

Very cool, thank you Christopher. In retrospect I wish I hadn't done it on my phone and pulled out my calipers...did better than the algorithm but not quite as good as a physician.

I think this is really difficult without clinical context, but a good exercise regardless.

 

[teedubbyaw]

"This ECG shows an anteroseptal myocardial infarction with reciprocal changes in the inferior leads."

That's not something I remember learning.

or this
"Marked elevation is visible in V3 and V4, with subtle ST-depression in lead III, cinching the diagnosis of STEMI."

 

[MonkeyArrow]

For the record, if you see someone who absolutely bombed it and looked like they almost guessed their way through, it's cause they did . You said to check for bugs, and no, I didn't find anything. So, yeah, you may wanna exclude that one outlier who got the easy ones wrong and the hard ones right. The wonderful powers of guessing on a phone combined with 50/50 odds.

 

[medicsb]

No bugs. Good test; not easy at all!

My results:
Your accuracy was: 83 %
Your sensitivity was: 88 %
Your specificity was: 75 %
You had 3 false positive(s) and 3 false negative(s).

 

[TransportJockey]

I'll have to take it on my computer. For some reason chrome on my phone didn't like the site


EDIT: Dear god I thought I was good at interpreting ECGs... But maybe not as much as I thought
Your accuracy was: 75 %
Your sensitivity was: 79 %
Your specificity was: 67 %
You had 4 false positive(s) and 5 false negative(s).

 

[Christopher]

For the record, if you see someone who absolutely bombed it and looked like they almost guessed their way through, it's cause they did . You said to check for bugs, and no, I didn't find anything. So, yeah, you may wanna exclude that one outlier who got the easy ones wrong and the hard ones right. The wonderful powers of guessing on a phone combined with 50/50 odds.

Do you remember roughly when you took it or what your accuracy was so I can exclude it?

 

[Christopher]

"This ECG shows an anteroseptal myocardial infarction with reciprocal changes in the inferior leads."

That's not something I remember learning.

or this
"Marked elevation is visible in V3 and V4, with subtle ST-depression in lead III, cinching the diagnosis of STEMI."

Part of the problem with current STEMI education is the misunderstanding as to what ST-depression actually means. Most textbooks will tell you localized depression is either (A) reciprocal change, or (B) focal ischemia.

The truth is only A is correct.

Ischemia instead produces diffuse ST-depression found in multiple leads. Localized ST-depression is a reciprocal change.

Many of these ECG's have a correct answer found by a careful analysis of ST-depression.

 

[Christopher]

78% accuracy, sensitivity 71% and specificity of 92%
1 false positive and .... 7 false negatives.

Your accuracy was: 83 %
Your sensitivity was: 88 %
Your specificity was: 75 %
You had 3 false positive(s) and 3 false negative(s).

Your accuracy was: 75 %
Your sensitivity was: 79 %
Your specificity was: 67 %
You had 4 false positive(s) and 5 false negative(s).

These are good "scores"; consider that the cardiologists didn't do much better at all.

Most false negatives I see are based on our current "understanding" of a STEMI being based on arbitrary millimeter criteria. If your false negatives included the phrase "subtle", that is par for the course.

5 or less false positives is consistent with a healthy STEMI system. The only way to be certain that your false positives are "Ok" is to analyze the ECG for *why* it isn't a STEMI and to see if your reasoning was sound. Artifact? Uncomplicated LVH? Uncomplicated LBBB? Those I'd say we shouldn't be mistaking for a STEMI. Other situations may not be so clear and are probably quite appropriate.

 

[teedubbyaw]

Part of the problem with current STEMI education is the misunderstanding as to what ST-depression actually means. Most textbooks will tell you localized depression is either (A) reciprocal change, or (B) focal ischemia.

The truth is only A is correct.

Ischemia instead produces diffuse ST-depression found in multiple leads. Localized ST-depression is a reciprocal change.

Many of these ECG's have a correct answer found by a careful analysis of ST-depression.

Another issue is that the extent of reciprocal changes I learned was inferior/lateral/posterior. I've always thought I was pretty good with 12 leads, but that's an eye opener that there's still a lot more to learn.

 

[Christopher]

Another issue is that the extent of reciprocal changes I learned was inferior/lateral/posterior. I've always thought I was pretty good with 12 leads, but that's an eye opener that there's still a lot more to learn.

I like to think of the changes in terms of a three dimensional continuous model of the heart...or in less engineery terms...a sphere of electrical tissue floating in space. Using exact definitions are unlikely to be anything but approximations of an approximation (aka. wild *** guess).

My favorite picture to use is this one from the Life in the Fast Lane crew. If you imagine yourself in space, floating with the heart, you can picture how something like an anteroseptal MI (perhaps due to a proximal or mid LAD lesion) could affect the inferior leads.

It also helps to realize that the frontal leads and the precordial leads are different in how they record electricity...but are in some ways equivalent. So yes, while the inferior leads technically follow a plane that cuts the body in two halves front-to-back, it isn't quite that simple.

Anyways...I came out of class knowing a lot about ECG's.

Then I hung out with some really smart folks and realized I came out of class knowing the bare minimum about ECG's....

 

[MonkeyArrow]

Do you remember roughly when you took it or what your accuracy was so I can exclude it?

Yeah. I took it yesterday around 8ish pm. I believe accuracy was somewhere around 53%. Sensitivity was around 70 and specificity right at 50 if I'm not mistaken.

Sorry :unsure:

 

[Christopher]

Yeah. I took it yesterday around 8ish pm. I believe acc. was somewhere around 53%. Sensitivity was around 70 and specificity right at 50 if I'm not mistaken.

Sorry :unsure:

No worries!

 

[9D4]

Well... I set myself up for failure on that one. Seeing all these guys in the 70's...
Considering I've only had 1 12 lead class, I guess i'll live with it...

Your accuracy was: 56 %
Your sensitivity was: 67 %
Your specificity was: 33 %
You had 8 false positive(s) and 8 false negative(s).

:unsure:

 

[Christopher]

Well... I set myself up for failure on that one. Seeing all these guys in the 70's...
Considering I've only had 1 12 lead class, I guess i'll live with it...

Your accuracy was: 56 %
Your sensitivity was: 67 %
Your specificity was: 33 %
You had 8 false positive(s) and 8 false negative(s).

:unsure:

Honestly you did fine. Look through the ECG's you "missed" and see how subtle they were or whether it was due to one of the more common causes of ST-elevation (of which AMI is an uncommon cause...although in this cohort it was).

 

[9D4]

Honestly you did fine. Look through the ECG's you "missed" and see how subtle they were or whether it was due to one of the more common causes of ST-elevation (of which AMI is an uncommon cause...although in this cohort it was).

One of them was a mis click if that matters

For the most part, the ones I missed, I think I rushed through. Half of them were very obvious looking back. The other half... Very,very subtle, so.... I guess there's no middle ground? Haha

 

[Jon]

Good Lord, Christopher! There's not a straight forward ECG in that entire test. Each was was difficult, IMHO. I would not feel comfortable calling any of them a STEMI without clinical correlation. My accuracy: 83% with 2 false positives and 4 false negatives (I answered conservatively). You could see many of these STEMI-like ECGs with other diagnoses (e.g., sepsis, etc.). I think paramedics and physicians alike would have extreme difficulty with this test.

Tom

If Tom only got an 83%, I feel slightly better with my score