What is it about the NIST data that you feel is flawed?
The science of science...
Science is observational and experimental study.
One of the reasons that a single study should never be used to determine practice is because of unintentional bias.
I see EMS people do this all the time. They find a single study that favors their opinion and they run with it like it is a definitive souce.
When choosing to do a study, the methods are chosen to support the hypothesis. It is not something sinister, it is just the limitations we face.
Many things are multifactoral and our observations are the summary of different causes. It is often impossible to set up a study to account for this. Especially in biological systems.
I use the example of soup. You literally select out part of the soup that you don't like or calls into question your corellations.
One of the most validating aspects of science is it stands up to scrutiny. BUt really, who is trying to reproduce or scrutinize fire and EMS studies who do not have a vested interest in them?
Nobody.
Even if they did, the public safety, in particularly, fire service propaganda machine will swing into effect to not discredit the results, but to elicit emotional response to those results.
Take for example sepsis research. If I come up with a conclusion, other people refute, I don't start an ad campaign talking about saving lives, what if it were you, heroes, life and death risks, etc.
I go back to the drawing board with this newly found information from the people who refuted my work and try to improve the process.
At the end of all studies is a bibliography. If you truly want to evaluate a study, you must also seek out all of those studies and read them as well. You will find a lot gets lost in translation as well as selectivity.
Research is time intensive. It also often raises more questions than it answers.
All good research explans "why" not just simple causation/correlation.
Some people also are seduced by the ease of "studies" to prove a point. If something is quantified, it does not qualify it. But they like to believe it does. They use research as a crutch similar to religion, to explain things in an easy to understand and absolute truth way.
Perhaps the biggest flaw in using research is the fact that it is based on observation but the principle being observed, especially in biological system, changes over time. The same can be said for public safety system.
The major problem with these studies is they assume all providers, firefighters, etc are equal. The individual knowledge/skill/experience and team dynamics are not quantifiable or reproducable.
What it leaves you with is a study, which supporters exclaim is definitive knowledge, that really isn't worth the paper its printed on.
Bad science is not better than no science. If you think that is not true, I urge you to look at all of the science around the 1800-1900s that demonstrated things like women were not as smart as men based on cranial vault volume or any plethora of "science" published and accepted demonstrating inferiority of various races or social groups.
Even for all of this, you need somebody to accurately and truthfully present the results of good studies to people who do not understand them and convince them of their validity.
Do you take the word of a used car salesman that the used car you want to buy is perfect?
Of course not, you go and get an opinion from a mechanic. As well as a history if you are smart, and you certainly test drive a few.
Last edited by a moderator: