Advertisement

The vagaries of patient selection in cardiovascular surgery

Open ArchivePublished:March 28, 2016DOI:https://doi.org/10.1016/j.jtcvs.2016.03.032
      Patient selection is a great confounder in surgical outcomes research. This editorial argues that patient selection rather than technical excellence is the dominant driver of differential outcomes.
      See Articles page 677 and 832.
      See Editorial Commentary page 681.
      People who inject drugs face increased risk of dying from both acute and chronic illnesses—in a pooled analysis they were more than 14 times more likely to die per year than similar persons who did not inject drugs.
      • Mathers B.M.
      • Degenhardt L.
      • Bucello C.
      • Lemon J.
      • Wiessing L.
      • Hickman M.
      Mortality among people who inject drugs: a systematic review and meta-analysis.
      The most common causes of death are drug-related, unintentional injuries, and suicide (accounting for more than 85% of deaths).
      • Nambiar D.
      • Weir A.
      • Aspinall E.J.
      • Stoove M.
      • Hutchinson S.
      • Dietze P.
      • et al.
      Mortality and cause of death in a cohort of people who had ever injected drugs in Glasgow: 1982-2012.
      Infections (predominantly bloodborne viral illnesses, but including endocarditis and other bacterial infections) account for <10% of deaths. If injection of drugs confers such negative prognosis why, as Kim and colleagues report,
      • Kim J.B.
      Surgical outcomes of infective endocarditis among intravenous drug users.
      is the 10-year survival after surgery for endocarditis in patients who inject drugs similar to that of patients with endocarditis who did not inject drugs (70% vs 69%)? This occurred despite a high incidence of valve reinfection in the drug-injecting group (60% by 8 years). Very similar observations were recently reported in a cohort of patients operated in Cleveland.
      • Shrestha N.K.
      • Jue J.
      • Hussain S.T.
      • Jerry J.M.
      • Pettersson G.B.
      • Menon V.
      • et al.
      Injection drug use and outcomes after surgical intervention for infective endocarditis.
      Could a 10-year survival of 69% after surgery for endocarditis in patients who inject drugs be plausibly expected? The high recurrence rate of endocarditis after surgery on drug injectors reported by Kim and colleagues
      • Kim J.B.
      Surgical outcomes of infective endocarditis among intravenous drug users.
      likely symbolizes a high recidivism rate in their cohort, so one would expect most patients undergoing operation to remain with a long-term elevated hazard associated with drug injecting, independent of the valve surgery. There is no plausible mechanism whereby heart valve disease or heart valve surgery would offer a long-term protective effect from hazards of habitual drug injection, so these patients would still be expected to have a high mortality rate from drug-related, suicidal, accidental, and other causes. I argue that the short- and long-term outcomes in this, and many other surgical series, are primarily a reflection of selection bias (Figure 1).
      Figure thumbnail gr1
      Figure 1Forms of selection bias in cardiovascular surgery and examples. VSD, Ventricular septal defect; ECMO, extracorporeal membrane oxygenation.

      Catchment Area, Population, and Institutional Biases

      The question arises in any observational study of surgical operations as to whether patients reported are representative of all patients with an indication for the procedure, or whether they are a selected group. Were patients undergoing operation in the 2 major academic Boston hospitals that contributed to this report
      • Kim J.B.
      Surgical outcomes of infective endocarditis among intravenous drug users.
      reflective of a typical population of people who inject drugs? A report on drug use in Boston in a similar timeframe reported that 25% of drug injectors were women, 39% of nonwhite ethnicity, and 70% younger than age 40 years,

      Substance abuse in Boston. March 2007. Available at: http://www.bphc.org/healthdata/other-reports/Documents/Substance%20Abuse%20in%20Boston%202007.pdf. Accessed March 12, 2016.

      whereas in the current study 39% were women, only 13% nonwhite, and mean age was 36 years in patients undergoing operation for endocarditis who injected drugs.
      • Kim J.B.
      Surgical outcomes of infective endocarditis among intravenous drug users.
      This implies a bias toward predominantly older, white women. The outcomes of surgery could conceivably be different in younger male patients from ethnic minority groups. Disparity in application of tertiary health care may be reflective of social, economic, and logistic factors such as differential access to health care, social status, income, ability to pay for health care, and family support—all factors that can also influence outcomes of surgical therapies. In the United States, it is well recognized that uninsured patients are more frequently cared for in public city hospitals compared with academic and private hospitals,
      • Calman N.S.
      • Golub M.
      • Ruddock C.
      • Le L.
      • Hauser D.
      Separate and unequal care in New York City.
      so it is probable that cohorts undergoing operation for endocarditis in academic centers include a disproportionately higher representation of patients who have resources to pay for health care, and therefore may have overall better health compared with a typical patient who injects drugs. Consequently, a public city hospital performing operations for endocarditis on predominantly uninsured or indigent patients is unlikely to achieve similar success to that reported in academic centers. Such selection is invariable, and is beyond the control of individual surgeons. However, rarely do surgical studies report socioeconomic demographic characteristics, such as wealth, insurance, employment, or family support. These are all key elements in both patient selection and early and long-term survival. A disadvantaged social background has been associated with reduced long-term survival after valve replacement.
      • Bagger J.P.
      • Edwards M.B.
      • Taylor K.M.
      Influence of socioeconomic status on survival after primary aortic or mitral valve replacement.
      Although commonly reported patient demographic characteristics and computed risk profiles may seem similar across hospitals and surgical series, patient selection has invariably occurred for a patient to end up in 1 country, town, locality, or hospital as opposed to another. The forces that drive the selection are often unmeasured but also drive outcomes; for example, in the current study,
      • Kim J.B.
      Surgical outcomes of infective endocarditis among intravenous drug users.
      it is unknown whether the long-term survivors reflect a selected group of people who had the resources to seek and maintain good health despite continued drug use, which is a possible explanation for the good long-term survival.

      Referral Bias

      Within any given locality, socioeconomic group, or clinical subgroup, further patient selection occurs in referral patterns. Factors that drive referral to a certain surgeon or center as opposed to another may sometimes have a bearing on outcomes. Tertiary centers, by definition, benefit positively from referral bias. Other than those for whom the tertiary center is also the local hospital, to be treated in a tertiary center, patients must first be well enough to be transferred from the referring hospital or clinic, and they must survive the transfer. This process thereby excludes some of the sickest patients (who would either have to be treated locally, die in transfer, or be inoperable on arrival at the tertiary center). The tertiary center then deals with a selected subcohort of patients with higher likelihood of survival. The wider a tertiary center's catchment area the stronger this effect, because patients who can travel several hours by road or aircraft for surgery are, by definition, a superselected group of healthier—and often more affluent and educated—patients. It has long been appreciated that patients with life-threatening surgical diseases who have traveled to tertiary centers from outside towns and regions for surgical treatment do better than local patients treated for the same condition in the tertiary center. This distance bias effect was demonstrated decades ago at the Mayo Clinic, Rochester, where the survival of patients undergoing operation for aortic aneurysms was superior for patients who traveled to Rochester from other regions and states, compared with those who lived in Rochester
      • Melton III, L.J.
      Selection bias in the referral of patients and the natural history of surgical conditions.
      —this is because the former group is heavily selected by clinical condition, socioeconomic status, time (which forces survivor treatment bias), ability to pay, and means to be transferred, whereas the latter group is a comparatively unselected group of all comers presenting to the local emergency room. For patients who have a choice of centers, there is also bias in referral that generally results in academic centers treating proportionately more patients with better likelihood of positive outcome (for the specific stage and complexity of disease). Patients are often selected for referral to academic medical centers on the basis of age (younger patients are more likely to travel or be referred
      • Melton III, L.J.
      Selection bias in the referral of patients and the natural history of surgical conditions.
      • Machens A.
      • Hauptmann S.
      • Dralle H.
      Referral bias in thyroid cancer surgery: direction and magnitude.
      ), severity of disease (less or more severe forms of disease may be more likely to be referred), presence of complications (patients with more severe complications are less able to travel), amongst other factors. The net result is that for the same surgical operation on an immediately life-threatening disease, a tertiary center may report substantially better results than a nearby cardiac center (which does not have many outside referrals), despite similar treatment strategy and seemingly similar case mix and severity of disease.

      Treatment Selection Bias and Survivor Treatment Bias

      The process of selection continues after a patient has come into contact with his or her surgeon. For life-threatening conditions, surgeons can reasonably decline some patients as being too sick, or declare surgery as being of questionable benefit or futile. Indications and timing of surgery for complex cardiac conditions with systemic manifestations are sufficiently controversial such that a surgeon may cherry-pick those most likely to survive. A surgeon could, for example, decline to perform an intervention on a patient with indication for surgical treatment of endocarditis because he is homeless and has history of prior suicide attempts, but yet might operate on a drug-injecting company executive with similar indication and valve pathology. Such selection is not accounted for in clinical studies. Surgeons may defer surgery as a trial of life (ie, operate only if patient survives beyond the short term with clinical improvement). Published surgical series do not typically present data on patients who were declined for surgery, so it is impossible to know what selection underlies a series being reported. Delaying surgery is a powerful selection tool and introduces survivor treatment bias that results in overestimation of the benefits of surgery, because only patients who survived from diagnosis could have undergone operation, whereas those with severe forms of disease die early without surgery. If surgery is systematically delayed (eg, to allow antibiotic therapy, reduce risk of intracerebral complications, or for any logistic reason) then the surgeon operates on only those who survive the initial disease period (and hence are survivors of the disease). Survivor treatment bias is seen in many studies of surgical treatment of endocarditis and leads to overestimation of survival with surgery,
      • Tleyjeh I.M.
      • Ghomrawi H.M.
      • Steckelberg J.M.
      • Montori V.M.
      • Hoskin T.L.
      • Enders F.
      • et al.
      Conclusion about the association between valve surgery and mortality in an infective endocarditis cohort changed after adjusting for survivor bias.
      giving the impression that surgery is an exceedingly effective treatment for severe endocarditis. Kim and colleagues
      • Kim J.B.
      Surgical outcomes of infective endocarditis among intravenous drug users.
      do not provide information on the interval between diagnosis of endocarditis and surgical treatment, so it is uncertain the degree to which surgeon case selection and survivor treatment bias could have played a role in their outcomes.

      Lead Time Bias

      Lead time bias is a form of selection bias, more frequently seen in high volume reference centers. The larger a center's volume, the more likely that patients in early stages of disease are referred for opinion. Operating on asymptomatic patients, and those in early subclinical stage of disease will result in exaggerated midterm benefit of surgery, simply because patients have been intervened on months or years before there was any threat to life by disease, giving the erroneous impression that surgery has actually prolonged life (compared to the typical natural history of the clinical stages of disease). Lead time bias is best understood by considering that the best outcomes with a complex heart operation will generally be achieved in patients who do not need (or least need) surgery. For example, the most durable results of valve repair during aortic valve sparing root replacement are seen in patients who do not have major aortic insufficiency at time of valve repair.
      • Badiu C.C.
      • Eichinger W.
      • Bleiziffer S.
      • Hermes G.
      • Hettich I.
      • Krane M.
      • et al.
      Should root replacement with aortic valve-sparing be offered to patients with bicuspid valves or severe aortic regurgitation?.

      Inclusion of Patients Based on Completed Rather than Intended Procedure

      Surgeons may continue the selection in the operating room by performing different or supplementary procedures other than that initially planned, or providing operative descriptions that may change categorization of the patient. For example, the performance of a septal myectomy can move a patient out of a valve replacement surgery series for endocarditis. Because surgical series are generally defined by completed procedure, rather than intention to treat, the true denominator of patients who were treated for the indication of the procedure being studied is rarely known. The net result is abundance in the surgical literature of articles describing excellent outcomes with various surgical procedures; however, rarely are outcomes described with patients categorized by presentation or disease as opposed to the treatment received. What would have been of greater interest are the long-term outcomes of drug injectors with endocarditis who presented to the 2 Boston hospitals regardless of the treatment received, including those who did not undergo surgery. It is beyond the scope of most retrospective studies (the dominant form of surgical research) to provide such analyses, so readers are almost always left with an undefined selected cohort who received a procedure, as opposed to patients who presented with a specified disease.

      Research and Publication Bias

      Selection occurs in the hands of surgical researchers reporting the data. Researchers necessarily proscribe inclusion criteria for study entry. These may not be reflective of an entire experience, such that different conclusions could arise from differing choice of subpopulation, surgical procedure, or year of study. The endpoints chosen are also selected by the researchers, as are the risk stratification variables, so the conclusions from a study could vary with the variables chosen. Bias in follow-up may further lead to selection. In the study by Kim and colleagues,
      • Kim J.B.
      Surgical outcomes of infective endocarditis among intravenous drug users.
      late follow-up was unavailable in a quarter of patients. Such patients are typically censored at last follow-up, so they count as study successes regardless of outcome. If those patients lost to follow-up were those with high drug-related morbidity, then their omission could erroneously suggest superior effectiveness of surgery. Finally, surgical researchers and journal editors choose what to publish and are less likely to publish results that show below-average surgical outcomes. The literature therefore gives an impression of unrealistically low operative mortality compared with the general reality.
      • Anyanwu A.C.
      • Treasure T.
      Unrealistic expectations arising from mortality data reported in the cardiothoracic journals.
      Although it may seem intuitive that the randomized trial design or prospective cohort study would eliminate selection bias, a randomized trial is inherently selection-biased because of strict inclusion and exclusion criteria, and biased recruitment into the studies (eg, drug injectors who have a history of recent overdose are unlikely to be put into a clinical trial, even if they pass inclusion and exclusion criteria).

      Implications of Patient Selection

      The net result is that the cardiovascular surgical literature is necessarily dominated by selection bias. Selection bias alters the clinical spectrum of disease, treatments given, and outcomes of treatment, and therefore influences our perceptions of effectiveness of therapy. Although there is a general perception that use of risk models (such as the European System for Cardiac Operative Risk Evaluation
      • Roques F.
      • Michel P.
      • Goldstone A.R.
      • Nashef S.A.
      The logistic EuroSCORE.
      ), or statistical techniques like multivariate analysis and propensity score matching can balance or eliminate selection bias, these statistical methods are imperfect, do not exclude hidden bias (eg, unmeasured patient, team, and logistic factors that affect both decision to treat and the outcome of treatment), and may further propagate rather than eliminate selection bias.
      • Austin P.C.
      • Platt R.W.
      Survivor treatment bias, treatment selection bias, and propensity scores in observational research.
      Even with adjustment for known patient and treatment factors, more than 50% of variance in outcomes after cardiac surgery may remain unexplained,
      • Likosky D.S.
      • Goldberg J.B.
      • DiScipio A.W.
      • Kramer R.S.
      • Groom R.C.
      • Leavitt B.J.
      • et al.
      Variability in surgeons' perioperative practices may influence the incidence of low-output failure after coronary artery bypass grafting surgery.
      whereas surgeon factors may contribute to some unexplained variance, very likely there remains substantial amount of unstated selection bias. What implications does this have for readers of cardiovascular surgical literature? It is important that readers understand that selection bias is a strong determinant of outcome, and that outstanding surgical results are often more a triumph of the art of selection than of technical proficiency or excellence in patient care. This is particularly likely where studies report good outcomes with life-threatening conditions typically associated with poor prognosis, when studies report long-term survival mirroring that of a matched population without heart disease, and where studies report a near-zero incidence of major adverse effects. In deciding whether the results seen a particular study can be replicated in a surgeon's own practice, one must first understand the nuances of the selection that likely occurred in the study of interest, and then ascertain whether similar selection could be applicable to his or her own practice. The outstanding results achieved by Kim and colleagues
      • Kim J.B.
      Surgical outcomes of infective endocarditis among intravenous drug users.
      in treating endocarditis in drug injectors are applaudable; however, readers should not see the data as implying improvement in prognosis of endocarditis associated with recreational drug use. Very likely selection, rather than surgical strategy, is the dominant driver of the effectiveness of cardiac surgery in this and many other patient groups.

      References

        • Mathers B.M.
        • Degenhardt L.
        • Bucello C.
        • Lemon J.
        • Wiessing L.
        • Hickman M.
        Mortality among people who inject drugs: a systematic review and meta-analysis.
        Bull World Health Organ. 2013; 91: 102-123
        • Nambiar D.
        • Weir A.
        • Aspinall E.J.
        • Stoove M.
        • Hutchinson S.
        • Dietze P.
        • et al.
        Mortality and cause of death in a cohort of people who had ever injected drugs in Glasgow: 1982-2012.
        Drug Alcohol Depend. 2015; 147: 215-221
        • Kim J.B.
        Surgical outcomes of infective endocarditis among intravenous drug users.
        J Thorac Cardiovasc Surg. 2016; 152: 832-841
        • Shrestha N.K.
        • Jue J.
        • Hussain S.T.
        • Jerry J.M.
        • Pettersson G.B.
        • Menon V.
        • et al.
        Injection drug use and outcomes after surgical intervention for infective endocarditis.
        Ann Thorac Surg. 2015; 100: 875-882
      1. Substance abuse in Boston. March 2007. Available at: http://www.bphc.org/healthdata/other-reports/Documents/Substance%20Abuse%20in%20Boston%202007.pdf. Accessed March 12, 2016.

        • Calman N.S.
        • Golub M.
        • Ruddock C.
        • Le L.
        • Hauser D.
        Separate and unequal care in New York City.
        J Health Care Law Policy. 2006; 9: 105-120
        • Bagger J.P.
        • Edwards M.B.
        • Taylor K.M.
        Influence of socioeconomic status on survival after primary aortic or mitral valve replacement.
        Heart. 2008; 94: 182-185
        • Melton III, L.J.
        Selection bias in the referral of patients and the natural history of surgical conditions.
        Mayo Clin Proc. 1985; 60: 880-885
        • Machens A.
        • Hauptmann S.
        • Dralle H.
        Referral bias in thyroid cancer surgery: direction and magnitude.
        Eur J Surg Oncol. 2008; 34: 556-562
        • Tleyjeh I.M.
        • Ghomrawi H.M.
        • Steckelberg J.M.
        • Montori V.M.
        • Hoskin T.L.
        • Enders F.
        • et al.
        Conclusion about the association between valve surgery and mortality in an infective endocarditis cohort changed after adjusting for survivor bias.
        J Clin Epidemiol. 2010; 63: 130-135
        • Badiu C.C.
        • Eichinger W.
        • Bleiziffer S.
        • Hermes G.
        • Hettich I.
        • Krane M.
        • et al.
        Should root replacement with aortic valve-sparing be offered to patients with bicuspid valves or severe aortic regurgitation?.
        Eur J Cardiothorac Surg. 2010; 38: 515-522
        • Anyanwu A.C.
        • Treasure T.
        Unrealistic expectations arising from mortality data reported in the cardiothoracic journals.
        J Thorac Cardiovasc Surg. 2002; 123: 16-20
        • Roques F.
        • Michel P.
        • Goldstone A.R.
        • Nashef S.A.
        The logistic EuroSCORE.
        Eur Heart J. 2003; 24: 881-882
        • Austin P.C.
        • Platt R.W.
        Survivor treatment bias, treatment selection bias, and propensity scores in observational research.
        J Clin Epidemiol. 2010; 63: 136-138
        • Likosky D.S.
        • Goldberg J.B.
        • DiScipio A.W.
        • Kramer R.S.
        • Groom R.C.
        • Leavitt B.J.
        • et al.
        Variability in surgeons' perioperative practices may influence the incidence of low-output failure after coronary artery bypass grafting surgery.
        Circ Cardiovasc Qual Outcomes. 2012; 5: 638-644

      Linked Article