|
***Click here for full pdf***
The recent Institute of Medicine (IOM) report about medical errors1 contains 2 different messages. One is cool and measured, a call for understanding the cause of errors in the health care system and for developing an approach using computerized and other "mechanical" support systems to decrease error rates. We like that messageindeed, we and our many colleagues at the Regenstrief Institute have been implementing it for the last 25 years.2-4
Furthermore, we agree that studying and understanding the causes of adverse events, preventable and otherwise, and developing mechanisms for reducing them are laudable goals that should be pursued. The other message in the IOM report is hot and shrill. It shouts about death and disability in US hospitals: "Preventable adverse events are a leading cause of death" and "at least 44,000, and perhaps as many as 98,000, Americans die in hospitals each year as a result of medical errors." The unstated corollaryreinforced by the death rate from motor vehicle collisions juxtaposed with the death rate from adverse eventsis that eliminating preventable adverse events also will eliminate the deaths.
Motor vehicle occupants do survive their ride if collisions are avoided. Unlike most people who step into motor vehicles, most patients admitted to hospitals have high disease burdens and high death risks even before they enter the hospital. Although some hospital deaths are preventable, most will occur no matter how many "accidents" we avoid. Of course, medical errors are never excusable, but the baseline death risk has to be known and factored out before drawing conclusions about the real effect of adverse reactions on death rates, preventable or otherwise.
The headline number of 98,000 deaths due to medical errors derives from the data reported in the Harvard Medical Practice Study, a groundbreaking study about adverse events, preventable and otherwise.5 This and subsequent articles6, 7 by the same group shed light on a subject that had never been systematically studied. These articles impressed us in 1991 and still do. The lower IOM figure (44,000) is derived from a 1992 study of Colorado and Utah data using the same methods as the Harvard Medical Practice Study.8 Both were observational studies and were not designed to describe causal relationships. The Harvard study authors included caveats, such as "lead to death" and "died at least in part as a result of adverse event." The authors of the Colorado-Utah study reported a proportion of patients who died in the adverse reaction group, but said nothing about the cause of these deaths. The IOM did not mention any of these limitations in its report.
In the Harvard study,5 investigators started with a sample of 31,429 patient admissions from the 1984 New York hospital admissions database. They applied screening criteria to find a subgroup of 7743 cases with a high likelihood of having experienced an adverse event. This group included all patients from the original sample who died, returned to intensive care, had excessive length of stay, or met other criteria. Because these selection criteria identify severely ill patients, we refer to it as the high-severity group. The Harvard authors targeted this high-severity subgroup for a detailed physician review in search of adverse events. The physicians discovered adverse events in 1278 of these 7743 cases. Of the 1278 high-severity patients for whom an adverse event was identified, the investigators reported that 173 patients (13.6%) died, at least in part because of an adverse event.
The Harvard study includes no information about the baseline risk of death in these patients or information about deaths in any comparison group. Therefore, it cannot be determined whether adverse events are correlated with, let alone whether they cause, death. Indeed, an assertion that adverse events caused death in 13.6% of the patients who experienced adverse events is tantamount to the assertion that there would be no deaths in a group with similar baseline risks who avoided all adverse events. Clinical experience tells us that this is not true.
How the 13.6% figure was obtained deserves careful consideration. Charts of all patients in the population of 31,429 who met 1 of the screening criteria (which included death) were reviewed for evidence of an adverse event. This approach is a logical way to obtain a high proportion of patients with adverse events. Among the subgroup of high-severity patients who had an adverse event, 13.6% died. What proportion of all patients in the high-severity group died? Using original data from the New York State Public Health Department, we learned that the New York State in-hospital acute care death rate was 3.4% in 1984. This should be a good approximation of the proportion of deaths in the Harvard initial sample of 31,429. By multiplying this figure times the sample size, we estimate that 1069 deaths occurred in the initial sample. Because death was a screening criterion for including a patient in the study sample,9 these 1069 deaths must have occurred in the high-severity group (n=7743), meaning that 13.8% of the patients in the high-severity group died overall. This figure is very similar to the 13.6% who died in the subset of this population with adverse events. This also means that the proportion of death in the group without adverse events must be very similar, unless many deaths occurred in the group with adverse events that were not attributed to adverse events. While the number of deaths in the adverse event group not attributed to adverse events is not known from the published data, the number is unlikely to be large since it was never referred to in any of the publications from this study.
However, neither the 13.8% of deaths for the overall group nor the 13.6% in the adverse event subset is a death rate in the usual sense of the word because of the sampling procedures used in the Harvard study (ie, death was a criterion for inclusion in the high-severity group). These figures are simply proportions of deaths calculated from the particular study sample; thus, one should not infer that they apply to any prespecified group of patients.
Furthermore, it is invalid for us to compare directly the proportion of deaths in the patients in the high-severity group who experienced or did not experience an adverse event because of many confounding factors, inherent selection bias in the death sample, and the possibility of initial differences in the baseline death risks of the 2 groups. However, it is also invalid for the IOM to assert that the observed deaths were caused by adverse events without presenting or defining the appropriateness of any comparison group whatsoever. Also, the sampling used in the study ensured identification of all deaths but not necessarily all adverse events, since the charts evaluated included all charts of patients who died but not all charts of patients who potentially experienced adverse events. This sampling method inflates the proportion of adverse events that lead to death.
The IOM report1 cited a number of other studies to support the argument that medical errors are a major cause of death. Most of these other studies also depended on physician chart review, qualified their claims with words like "possible cause," and lacked any kind of control or comparison group; however, the IOM did not emphasize these limitations. Furthermore, the IOM cited a study that claimed 7000 deaths were due to medication errors in 1993.10 However, this study miscounted deaths due to drug abuse as due to medication errors, according to a subsequent letter11; thus, in our opinion, the study should not have been cited.
The IOM figures focus on "preventable adverse events" rather than the total set of adverse events, but their estimates of preventable deaths depend on the relationship between adverse events in general and death. The number of preventable adverse events that caused death is calculated as a direct proportion of the total adverse events that cause death; therefore, if the relationship between overall adverse events and death declines, so does the relationship between preventable adverse events and death.
The Harvard study acknowledged that eliminating the adverse event (and even the negligence) would have little effect on the life expectancy of many terminally ill patients, but did not quantify the possible implications of this phenomenon on the death figures in the adverse event group. The 30-day posthospital admission mortality rate of 11.6%12 and the 30-day death risk up to 40% for some categories of Medicare patients13 suggest that an important proportion of hospitalized patients are at, or near, the end of their lives. If we are not careful about the mechanisms constructed to detect and reduce errors, end-of-life decisions could be distorted.
Given these facts, using available data and some reasonable assumptions, we believe that the increment in the published death rate due to adverse events above the baseline death rate could be very small. We also assert that the available data do not support IOM's claim of large numbers of deaths caused by adverse events (preventable or otherwise). The first author of the Harvard study recently raised other related concerns about the IOM report.14 Clearly, more study with careful attention to risk levels is needed to determine the true impact of adverse events on death rates among hospitalized patients. Until those results are available, the design of regulatory solutions is premature.
The IOM uses elaborate controls to ensure a careful balance of interests in the parties on the committees that produce reports and it uses extensive review to avoid errors in its reports. However, the reliance on studies without controls to make headline claims about huge numbers of preventable deaths was one error that it did not catchperhaps proving the point once again, in an Escherian way, that humans do err.
|
Subscribe to this comment's feed