| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Guest Access | Sign In via User Name/Password
|
|
|
|||||||
Guest Access | Sign In via User Name/Password |
|||||||||
Dr. Heffner is Professor of Medicine, Department of Medicine, Medical University of South Carolina.
Correspondence to: John E. Heffner, MD, FCCP, Division of Pulmonary and Critical Care Medicine, Medical University of South Carolina, 96 Jonathan Lucas St, PO Box 250623, Charleston, SC 29425; e-mail: heffnerj{at}musc.edu
I have developed a fondness for meta-analysis because of the jeopardy we physicians face in thinking we know more about some subjects than we actually do. With 33,000 new citations added to MEDLINE each month, it is no surprise that published reports of new clinical approaches stick in our minds and get entrenched into practice before sufficient evidence establishes their merit. Periodically, a good dose of meta-analysis purges our misperceptions and allows us to see the strength of the evidence—or lack thereof—that underlies our clinical habits. Meta-analysis fulfills its primary role by aggregating data from conflicting or underpowered studies in an attempt to establish statistical evidence of the value of an intervention.1 But to my way of thinking, meta-analysis provides a more important secondary benefit of critically appraising the quality of the data entered into its review. By detecting weaknesses of primary studies and identifying what we don’t know, but think we do, meta-analysis sets research agendas and keeps us from practicing "no-evidence"-based medicine.
I am delighted, therefore, that Freeman and colleagues have published in this edition of CHEST (see page 1412) a meta-analysis examining the relative risks in critically ill patients of percutaneous dilatational tracheotomy (PDT) by the Ciaglia technique2 compared to the standard surgical procedure. PDT has emerged within a short 15 years to become the principle method in many ICUs for placing a surgical airway. The technique has spawned a wealth of published case series, reviews, and conflicting editorial opinions but surprisingly few adequately designed, prospective, comparative studies examining its value relative to the standard technique. So what can we learn about PDT from this meta-analysis review?
First, we should look at how the study fulfilled the primary goal of meta-analysis in establishing statistically significant differences between competing interventions. The investigators observed similarities between PDT and surgical tracheotomy in three of the eight measured outcomes, but noted shorter operative times (9.84-min shorter), less postoperative and perioperative bleeding, less overall postoperative complications, and less postoperative stomal infections with PDT. Although nearly all of the complications favoring PDT appeared to be of minor clinical importance, we should critically appraise the meta-analysis itself before accepting PDT as the "procedure of choice."
Meta-analyses are most useful when they analyze aggregate data from high quality, randomized controlled trials (RCTs). As indicated by the authors, two of the five studies were nonrandomized. Inclusion of nonrandomized studies is risky in a meta-analysis because they usually overestimate the average effect of a treatment.3 Of the RCTs, one used an inadequate randomization technique (alternate weeks), and the randomization method was not mentioned in another. In the absence of appropriate subject allocation, I can’t help but suspect that patients assigned to standard tracheotomy were more critically ill, more technically challenging for airway placement, or affected by some other confounders. Consequently, the primary studies may have biased the results toward more complications in the standard tracheotomy group. This concern is especially relevant considering that none of the primary studies provided measures of disease severity. Meta-analyses can provide an estimate of the impact of selection bias by analyzing data for effect size separately for the randomized and nonrandomized studies, which was not done by Freeman and colleagues. It should be noted also that the Mantel-Haenszel fixed effect model used in this analysis may produce extremely misleading conclusions when confounders are not taken into account in the initial design of primary studies.
As an additional standard, meta-analyses should pool data from studies that are homogeneous.4 ) Homogeneous studies treat patients with similar interventions and measure similar outcomes. Meta-analyses that aggregate heterogeneous studies are pooling data from apples and oranges. The resulting conclusions are of dubious value except for concocting fruit cocktails. I am concerned that the five studies in the present analysis may have been too heterogeneous to allow pooling of their data. For instance, one study quantifies bleeding in milliliters and another by the ability to control blood loss with digital pressure. Even within the same study, different definitions of bleeding were used between tracheotomy groups. Heterogeneity of outcomes because of differing definitions is the only reason I can see that one of the studies observed stomal infections with standard tracheotomy in 0% of patients and another study in 63%; one study observed overall complications in 8% of patients, and another study noted complications in 87%. Freeman and colleagues established homogeneity of the primary studies with the Q statistic. Tests of homogeneity, however, have low power; test results that do not reject the hypothesis that primary studies are homogeneous and do not prove that the same outcomes are being measured.4 )
As their secondary purpose, meta-analyses should examine the quality of the primary studies and identify weaknesses in study design that could contribute to overgeneralized results. I would like to know, for example, how many of the primary studies used data collectors blinded to study purpose and group assignment, excluded from enrollment technically challenging patients (short necks, obesity), or experienced high dropout rates. Considering the subjective nature of many of the outcomes measured, lack of blinding is an important consideration. Without a measure of study quality, the suspicious reader might suspect that the zeal of the primary investigators for a new procedure could inflate estimates of the value of PDT. By not reporting quality criteria, I assume Freeman and colleagues found little evidence of rigorous study designs. My review of the primary studies found no use of blinding.
So what have we learned from this meta-analysis on the relative merits of PDT in our ICUs? I believe Freeman and colleagues have clearly demonstrated our unclear knowledge of the short-term risks of tracheotomy. As they discuss, we know even less about the long-term effects of PDT on the airway or the relative costs of the two procedures. Previous studies state that PDT has a cost advantage because it can be performed in the ICU rather than the operating room.5 6 In one of the primary studies, however, both procedures were done in the operating room,7 and in another both were done in the ICU.8 When performed in similar settings, large discrepancies in cost largely disappear.8 9
I believe Freeman and colleagues have done us a great service by establishing a research need for rigorously designed, randomized trials that examine standardized outcomes from tracheotomy. Considering how often we make decisions about modes of airway support, we should recognize—and quickly respond to—what we have now learned we don’t know about these commonly performed interventions.
But what do we do in the meantime in selecting between standard tracheotomy and PDT? Although insufficient data establish that PDT is safer than the standard procedure in general populations of critically ill patients, Freeman and associates adequately support that it is just as safe in skilled hands when clinically important end points are considered. Application of PDT should depend on local expertise and practice patterns within an ICU until better outcome data emerge. But should PDT be profiled as "simple to learn and perform," as stated in the meta-analysis introduction? Both PDT and standard surgical tracheotomy present considerable risks for acute and long-term complications. Their use for critically ill patients requires extensive procedural training, airway management skills, quality monitoring, and respect for the potential lethality of each of these invasive procedures. As taught to me by an old surgeon, there is no such thing as "simple" surgery. In the final analysis, this is one thing that we do know.
References
This article has been cited by other articles:
|
|
 |
|
  J E Heffner Management of the chronically ventilated patient with a tracheostomy Chronic Respiratory Disease, July 1, 2005; 2(3): 151 - 161. [Abstract] [PDF]
|
|
|
|
 |
|
 K. H. Polderman, J. J. Spijkstra, R. de Bree, H. M.T. Christiaans, H. P.M.M. Gelissen, J. P.J. Wester, and A. R.J. Girbes Percutaneous Dilatational Tracheostomy in the ICU: Optimal Organization, Low Complication Rates, and Description of a New Complication Chest, May 1, 2003; 123(5): 1595 - 1602. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. E. Heffner The Role of Tracheotomy in Weaning Chest, December 1, 2001; 120(6_suppl): 477S - 481S. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
