The vagaries of patient selection in cardiovascular surgery

The question arises in any observational study of surgical operations as to whether patients reported are representative of all patients with an indication for the procedure, or whether they are a selected group. Were patients undergoing operation in the 2 major academic Boston hospitals that contributed to this report reflective of a typical population of people who inject drugs? A report on drug use in Boston in a similar timeframe reported that 25% of drug injectors were women, 39% of nonwhite ethnicity, and 70% younger than age 40 years, whereas in the current study 39% were women, only 13% nonwhite, and mean age was 36 years in patients undergoing operation for endocarditis who injected drugs. This implies a bias toward predominantly older, white women. The outcomes of surgery could conceivably be different in younger male patients from ethnic minority groups. Disparity in application of tertiary health care may be reflective of social, economic, and logistic factors such as differential access to health care, social status, income, ability to pay for health care, and family support—all factors that can also influence outcomes of surgical therapies. In the United States, it is well recognized that uninsured patients are more frequently cared for in public city hospitals compared with academic and private hospitals, so it is probable that cohorts undergoing operation for endocarditis in academic centers include a disproportionately higher representation of patients who have resources to pay for health care, and therefore may have overall better health compared with a typical patient who injects drugs. Consequently, a public city hospital performing operations for endocarditis on predominantly uninsured or indigent patients is unlikely to achieve similar success to that reported in academic centers. Such selection is invariable, and is beyond the control of individual surgeons. However, rarely do surgical studies report socioeconomic demographic characteristics, such as wealth, insurance, employment, or family support. These are all key elements in both patient selection and early and long-term survival. A disadvantaged social background has been associated with reduced long-term survival after valve replacement. Although commonly reported patient demographic characteristics and computed risk profiles may seem similar across hospitals and surgical series, patient selection has invariably occurred for a patient to end up in 1 country, town, locality, or hospital as opposed to another. The forces that drive the selection are often unmeasured but also drive outcomes; for example, in the current study, it is unknown whether the long-term survivors reflect a selected group of people who had the resources to seek and maintain good health despite continued drug use, which is a possible explanation for the good long-term survival.

Within any given locality, socioeconomic group, or clinical subgroup, further patient selection occurs in referral patterns. Factors that drive referral to a certain surgeon or center as opposed to another may sometimes have a bearing on outcomes. Tertiary centers, by definition, benefit positively from referral bias. Other than those for whom the tertiary center is also the local hospital, to be treated in a tertiary center, patients must first be well enough to be transferred from the referring hospital or clinic, and they must survive the transfer. This process thereby excludes some of the sickest patients (who would either have to be treated locally, die in transfer, or be inoperable on arrival at the tertiary center). The tertiary center then deals with a selected subcohort of patients with higher likelihood of survival. The wider a tertiary center's catchment area the stronger this effect, because patients who can travel several hours by road or aircraft for surgery are, by definition, a superselected group of healthier—and often more affluent and educated—patients. It has long been appreciated that patients with life-threatening surgical diseases who have traveled to tertiary centers from outside towns and regions for surgical treatment do better than local patients treated for the same condition in the tertiary center. This distance bias effect was demonstrated decades ago at the Mayo Clinic, Rochester, where the survival of patients undergoing operation for aortic aneurysms was superior for patients who traveled to Rochester from other regions and states, compared with those who lived in Rochester—this is because the former group is heavily selected by clinical condition, socioeconomic status, time (which forces survivor treatment bias), ability to pay, and means to be transferred, whereas the latter group is a comparatively unselected group of all comers presenting to the local emergency room. For patients who have a choice of centers, there is also bias in referral that generally results in academic centers treating proportionately more patients with better likelihood of positive outcome (for the specific stage and complexity of disease). Patients are often selected for referral to academic medical centers on the basis of age (younger patients are more likely to travel or be referred^,  ), severity of disease (less or more severe forms of disease may be more likely to be referred), presence of complications (patients with more severe complications are less able to travel), amongst other factors. The net result is that for the same surgical operation on an immediately life-threatening disease, a tertiary center may report substantially better results than a nearby cardiac center (which does not have many outside referrals), despite similar treatment strategy and seemingly similar case mix and severity of disease.

The process of selection continues after a patient has come into contact with his or her surgeon. For life-threatening conditions, surgeons can reasonably decline some patients as being too sick, or declare surgery as being of questionable benefit or futile. Indications and timing of surgery for complex cardiac conditions with systemic manifestations are sufficiently controversial such that a surgeon may cherry-pick those most likely to survive. A surgeon could, for example, decline to perform an intervention on a patient with indication for surgical treatment of endocarditis because he is homeless and has history of prior suicide attempts, but yet might operate on a drug-injecting company executive with similar indication and valve pathology. Such selection is not accounted for in clinical studies. Surgeons may defer surgery as a trial of life (ie, operate only if patient survives beyond the short term with clinical improvement). Published surgical series do not typically present data on patients who were declined for surgery, so it is impossible to know what selection underlies a series being reported. Delaying surgery is a powerful selection tool and introduces survivor treatment bias that results in overestimation of the benefits of surgery, because only patients who survived from diagnosis could have undergone operation, whereas those with severe forms of disease die early without surgery. If surgery is systematically delayed (eg, to allow antibiotic therapy, reduce risk of intracerebral complications, or for any logistic reason) then the surgeon operates on only those who survive the initial disease period (and hence are survivors of the disease). Survivor treatment bias is seen in many studies of surgical treatment of endocarditis and leads to overestimation of survival with surgery, giving the impression that surgery is an exceedingly effective treatment for severe endocarditis. Kim and colleagues do not provide information on the interval between diagnosis of endocarditis and surgical treatment, so it is uncertain the degree to which surgeon case selection and survivor treatment bias could have played a role in their outcomes.

Lead time bias is a form of selection bias, more frequently seen in high volume reference centers. The larger a center's volume, the more likely that patients in early stages of disease are referred for opinion. Operating on asymptomatic patients, and those in early subclinical stage of disease will result in exaggerated midterm benefit of surgery, simply because patients have been intervened on months or years before there was any threat to life by disease, giving the erroneous impression that surgery has actually prolonged life (compared to the typical natural history of the clinical stages of disease). Lead time bias is best understood by considering that the best outcomes with a complex heart operation will generally be achieved in patients who do not need (or least need) surgery. For example, the most durable results of valve repair during aortic valve sparing root replacement are seen in patients who do not have major aortic insufficiency at time of valve repair.

Surgeons may continue the selection in the operating room by performing different or supplementary procedures other than that initially planned, or providing operative descriptions that may change categorization of the patient. For example, the performance of a septal myectomy can move a patient out of a valve replacement surgery series for endocarditis. Because surgical series are generally defined by completed procedure, rather than intention to treat, the true denominator of patients who were treated for the indication of the procedure being studied is rarely known. The net result is abundance in the surgical literature of articles describing excellent outcomes with various surgical procedures; however, rarely are outcomes described with patients categorized by presentation or disease as opposed to the treatment received. What would have been of greater interest are the long-term outcomes of drug injectors with endocarditis who presented to the 2 Boston hospitals regardless of the treatment received, including those who did not undergo surgery. It is beyond the scope of most retrospective studies (the dominant form of surgical research) to provide such analyses, so readers are almost always left with an undefined selected cohort who received a procedure, as opposed to patients who presented with a specified disease.

Selection occurs in the hands of surgical researchers reporting the data. Researchers necessarily proscribe inclusion criteria for study entry. These may not be reflective of an entire experience, such that different conclusions could arise from differing choice of subpopulation, surgical procedure, or year of study. The endpoints chosen are also selected by the researchers, as are the risk stratification variables, so the conclusions from a study could vary with the variables chosen. Bias in follow-up may further lead to selection. In the study by Kim and colleagues, late follow-up was unavailable in a quarter of patients. Such patients are typically censored at last follow-up, so they count as study successes regardless of outcome. If those patients lost to follow-up were those with high drug-related morbidity, then their omission could erroneously suggest superior effectiveness of surgery. Finally, surgical researchers and journal editors choose what to publish and are less likely to publish results that show below-average surgical outcomes. The literature therefore gives an impression of unrealistically low operative mortality compared with the general reality. Although it may seem intuitive that the randomized trial design or prospective cohort study would eliminate selection bias, a randomized trial is inherently selection-biased because of strict inclusion and exclusion criteria, and biased recruitment into the studies (eg, drug injectors who have a history of recent overdose are unlikely to be put into a clinical trial, even if they pass inclusion and exclusion criteria).

The net result is that the cardiovascular surgical literature is necessarily dominated by selection bias. Selection bias alters the clinical spectrum of disease, treatments given, and outcomes of treatment, and therefore influences our perceptions of effectiveness of therapy. Although there is a general perception that use of risk models (such as the European System for Cardiac Operative Risk Evaluation), or statistical techniques like multivariate analysis and propensity score matching can balance or eliminate selection bias, these statistical methods are imperfect, do not exclude hidden bias (eg, unmeasured patient, team, and logistic factors that affect both decision to treat and the outcome of treatment), and may further propagate rather than eliminate selection bias. Even with adjustment for known patient and treatment factors, more than 50% of variance in outcomes after cardiac surgery may remain unexplained, whereas surgeon factors may contribute to some unexplained variance, very likely there remains substantial amount of unstated selection bias. What implications does this have for readers of cardiovascular surgical literature? It is important that readers understand that selection bias is a strong determinant of outcome, and that outstanding surgical results are often more a triumph of the art of selection than of technical proficiency or excellence in patient care. This is particularly likely where studies report good outcomes with life-threatening conditions typically associated with poor prognosis, when studies report long-term survival mirroring that of a matched population without heart disease, and where studies report a near-zero incidence of major adverse effects. In deciding whether the results seen a particular study can be replicated in a surgeon's own practice, one must first understand the nuances of the selection that likely occurred in the study of interest, and then ascertain whether similar selection could be applicable to his or her own practice. The outstanding results achieved by Kim and colleagues in treating endocarditis in drug injectors are applaudable; however, readers should not see the data as implying improvement in prognosis of endocarditis associated with recreational drug use. Very likely selection, rather than surgical strategy, is the dominant driver of the effectiveness of cardiac surgery in this and many other patient groups.