Advertisement

Discussion

    Open ArchivePublished:February 14, 2019DOI:https://doi.org/10.1016/j.jtcvs.2018.12.074
        Dr Anelechi C. Anyanwu (New York, NY). This excellent analysis from the state of Massachusetts evaluates the trends and influence of public reporting. The hospital survival after coronary bypass surgery was consistently lower than the Society of Thoracic Surgeons (STS) average, as you have seen—that's why it's a higher predicted mortality—there were no obvious negative effects of public reporting, and only 2 hospitals were sanctioned in the time period for poor outcomes. None has been sanctioned in the past 10 years, none of them lost their market share, and this supports the positive value to the whole exercise. The analysis does, however, broach on several broad issues regarding public reporting, ranking, and comparison of centers and surgeons and evaluation of surgical outcomes, and I would like to develop some of these further.
        So my first question regards the low observed mortality in Massachusetts. In 2003 when public reporting started, you had a mortality that was the same as the STS database, but in 2005, 2 years later and every year since, you have had a mortality lower than the STS database and you have maintained an observed to expected ratio about 0.6 to 0.8 and a mortality rate consistently below 1.7% compared with 2% nationally. That implies someone's decision on public reporting. Indeed in New York State where we also have public reporting, our mortality rate for the last reportable year was 1.5%, also below the STS.
        So setting aside risk aversion, which we will come to, can you suggest mechanisms as to how public reporting could lead to better outcomes of surgery?
        Dr Shahian (Boston, Mass). That is an excellent question that the published literature does not answer conclusively.
        Before I answer your question, I would like to make a subtle but important clarification, which is the distinction between ranking and rating. Ranking implies a direct comparison of the results of 2 hospitals, and that is not permissible when using indirectly standardized results as we do in Massachusetts and nationally. Rather, we rate hospitals by comparing their risk-adjusted performance for their case mix compared with what would have been expected based on the national benchmark population of providers. This distinction seems arcane but it's actually very important and something we always try to emphasize.
        We don't have comparative data before 2002 to 2003, which is a limitation of our study. However, you are correct that in these first years of Massachusetts public reporting our mortality rates were close to national STS rates, then markedly diverged.
        As to why public reporting in general might improve outcomes, knowledge is power. With knowledge of your own results and the desire to excel, which most people in this room have, these data are a great stimulus and guide to performance improvement. They are also a lever for cardiac surgical programs to use with hospital administrations to get the resources they need to make their programs better.
        Finally, we have a very small number of programs in this country that consistently (not just 1 reporting period) underperform. Public reporting appropriately identifies such providers and increases the pressure on them to improve or close.
        Dr Anyanwu. So then should all states and countries embark on public reporting, in your opinion?
        Dr Shahian. If you can do it responsibly—and I would suggest that the Massachusetts and STS experiences are good models for mandatory and voluntary reporting—and with appropriate methodology, then I think it is an ethical imperative, It's a way to provide patients with additional objective information to aid them in the selection of a provider. It is certainly not the only factor they should consider, but it can be very useful in deciding where to go for surgery.
        Dr Anyanwu. But your positive experience has not been universal. As you know, in the United Kingdom, for example, there has been increasing outcry against public reporting with many cardiac surgeons reporting having observed or practiced risk aversion, surgeons have lost their jobs due to scrutiny of results, surgical training is suffering because the consultants are reluctant to allow trainees to operate. So these are likely consequences of surgeon-level reporting, which occurs in the United Kingdom and in the United States.
        But I know that in Massachusetts you have stopped doing surgeon-level reporting. Can you tell us why you stopped and whether as societies we should push against any further surgeon level reporting for future systems?
        Dr Shahian. Surgeon-level public reporting for coronary artery bypass grafting was discontinued in Massachusetts primarily because the interventional cardiology community did not want operator-specific reporting for percutaneous coronary intervention, and there was a desire to have similar approaches to these 2 initiatives.
        In general, what are the pros and cons of surgeon-specific reporting? On the pro side, we know that there is variation in surgeon outcomes, even within the same program. We are not a fungible commodity, and patients have a right to know the results of their prospective caregivers if they can be accurately determined. Secondly, patients and referring physicians often pick specific surgeons, not just hospitals, for elective or semielective procedures. A unique relationship is subsequently established between the patient and their surgeon, and the patient expects that that surgeon will oversee and optimize all aspects of the care required for a cardiac surgical patient in their institution. This is a very special relationship that doesn't exist between the patient and any other component of that team.
        Now, on the con side, it's more statistically challenging to estimate performance for a surgeon just because of the smaller number of cases, which is why we have developed an STS multiprocedural, multidomain composite measure for adult cardiac surgeons. The other potential problem with surgeon-level reporting is that it is surgeons, not hospitals, who make the decision about whether a patient is offered surgery or not. So if you are concerned about risk aversion, I think reporting at the surgeon level has more potential for harm. However, as a counter to that risk aversion argument, there are also data showing that in a public reporting environment, higher-risk patients tend to ultimately receive their operative care from more capable surgeons, although they may have been turned down by less experienced surgeons. This is exactly what you would like to have happen.
        Dr Anyanwu. So unintended and intended consequences of public reporting can lead to disruption of individual carriers, teams, hospitals, and while you have succeeded and have already taken those 2 outlier hospitals, that's not necessarily the experience elsewhere. So when we say a surgeon or a hospital is an outlier, we have to be pretty sure that that's the case.
        So that brings us to the tools that we use to measure them. In your study you took extra effort to validate a sample of cases. You adjudicated all deaths, you adjudicated all major risk factors, and that ensured a very accurate data set. In contrast, the STS, to which you compare, does none of the above, or in a very limited fashion. And I believe that the difference in prevalence of the key risk factors like emergent status and shock that you showed were because your data are very adjudicated.
        Do you think we should be using nonadjudicated data sources like the STS or administrative data sources to rate centers or surgeons or to rank centers or surgeons?
        Dr Shahian. I have to disagree with the premise of your statement. The data in the STS database are extraordinarily well audited, more so than virtually any other health care data of which I am aware. Using a highly respected external auditor, we now audit 10% of all programs every year. They review all major risk factors and outcomes.
        I just reviewed our audit data for the last several years. The agreement rates for particularly high-risk variables such as cardiogenic shock, emergency status, were all in the 95% to 99% range.
        Dr Anyanwu. Sorry. What I am saying is your data are more accurate, so that's the reason why you have a low incidence.
        Dr Shahian. I'm talking about the STS national data.
        Dr Anyanwu. The STS national data may not be as accurate as yours.
        Dr Shahian. No, I'm talking about the STS.
        Dr Anyanwu. Okay.
        Dr Shahian. Ten percent of STS programs are audited every year, about 120 programs last year and roughly 80 data elements from each chart, with 95% to 99% plus agreement rates for most high-risk variables. I am very confident about current STS data.
        STS also provides the opportunity for data managers and surgeons to access the leaders of the database, essentially on a real-time basis, to ask questions about coding, including conference calls every week addressing these questions. We have a national STS Advances in Quality and Outcomes conference annually devoted to teaching data managers how to code accurately. Overall, I'm quite confident with the accuracy of the STS database, although we are always trying to improve.
        Our additional peer adjudication of high-risk variables in Massachusetts has been a valuable adjunct to educate data managers and surgeons about the nuances of certain variables, and to reassure all stakeholders about the integrity and accuracy of the data. It could easily be reproduced by any state or regional cardiac surgical organization.
        Dr Oz Shapira (Jerusalem, Israel). It's humbling to give any comments on Dr Shahian's presentation, but I congratulate you and the authors for another excellent study. I was part of the Massachusetts database, and I want to ask you a specific question, and I fully believe in public reporting of a surgeon's performance. But you talk mostly about risk aversion. There is another very important risk in public reporting, which is teaching aversion, particularly in Massachusetts when you have teaching and nonteaching hospitals, not maybe a corollary to risk aversion.
        Are you looking to teaching aversion in your data and trying to separate from teaching and nonteaching hospitals and correlate it with risk aversion?
        Dr Shahian. A wonderful point, Oz. Unfortunately, I have no data on that issue. Although public reporting may impact the willingness of surgeons to take residents through tough cases, the increasing complexity and severity of the cases being encountered in cardiac surgery are also a challenging issue in this regard.
        Dr Benjamin Kozower (Charlottesville, Va). Thank you for an outstanding presentation and for all you have done in this field. I would like to ask a follow-up question about risk aversion. Around the same time that your data were collected, there were several important papers published using Massachusetts data. They showed that for acute cardiogenic shock where percutaneous coronary intervention might be most beneficial, it was used less frequently following public reporting of these high risk cases. So, it's hard for me to reconcile this and I remain concerned that risk aversion exits and is related to public reporting.
        Dr Shahian. Good point, Benj. I have had an opportunity over the past year to try to ferret out all the available information in the literature about risk aversion. The evidence for this behavior in interventional cardiology is much stronger and much more consistent, probably because a lot of the cardiogenic shock, postmyocardial infarction patients who might have been operated on in the past are all going to the cath lab now. So the interventional cardiologists have to deal with these very challenging ethical questions. Based on my review of the evidence, there is no question that there is risk aversion in the percutaneous coronary intervention community.
        In cardiac surgery I would say the evidence is mixed. There are surveys asking surgeons if they have practiced risk aversion, and they will admit to it. However, if you look at aggregate data, such as the Massachusetts or national STS data, it's hard to conclusively demonstrate risk aversion. Of course, those aggregate data do not tell you what's going on at the level of an individual patient decision.
        Dr Kozower. And in your state data source, do you have any way to link the short-term outcomes to long-term outcomes?
        Dr Shahian. We don't, Benj, but that is something that we are actually very close to being able to do in STS through linkages with the National Death Index. Also, as short-term mortality rates have fallen to the 1% range, mortality alone is not an adequate discriminator of quality. That's one reason why we look at complications as well in the STS composite measures. But I agree we need to look at the long-term durability of what we do—how are those patients doing 5 years and 10 years out.
        Dr Marian Zembala (Zabrze, Poland). Thank you so much for this message, but when you look for the trends in patients who are in cardiogenic shock for the past 10 to 12 years, you will be surprised. We are still almost at the same level like 10 to 15 years ago despite the assist devices, echo, and other points. And the lessons, what we have learned and we realize in Poland, we separate the group of cardiogenic patients and patients as postinfarction like ST-elevation myocardial infarction, for example, and present separate death. In this situation you save the patient, you are active for teaching and active to accept this patient for also surgical and cardiac revascularization, but you are not afraid in a very obsessive way to disturb your average very good results. I think you will always compromise, because patient is first and you can keep quality. That is my message, that is my intention.
        Dr Shahian. So if I understand you correctly, are you arguing for exclusion of very high-risk patients from public reporting?
        Dr Zembala. Not exclusion but just present separately. That's different. Then you can present deaths, because you have centers that avoid this type of patient and you have centers which 24 hours work on all types of patients, including cardiogenic shock, and then this situation you can compromise both and the patient's life is first.
        Dr Shahian. We have not done that in Massachusetts or STS nationally for cardiac surgery, but it's a reasonable suggestion. It is done in Massachusetts percutaneous coronary intervention public reporting, where they have separate reports for more elective cases versus the shock or ST-elevation myocardial infarction cases.
        Dr Joanna Chikwe (New York, NY). A slightly broader question about risk aversion driven by this laser focus on 30-day outcomes. Would either of the panelists speak to innovation aversion caused by this laser focus on 30-day outcomes? We heard this morning about data from an entire state where only 5.7% of coronary artery bypass grafting patients had more than a single arterial conduit. How can we drive practice and innovation that is more focused on our patients' long-term outcomes and not just on the 30-day results?
        Dr Shahian. As I mentioned previously, as short-term outcomes progressively improve, getting long-term data to share with patients will be increasingly important, not just survival but also freedom from reinterventions and late complications.
        Regarding the stifling of innovation by public reporting, that is a legitimate concern. In Dr Bavaria's STS presidential address 2 years ago, he expressed concerns about how reporting could stifle innovation, because initial results with any new procedure or device may not be optimal. I understand that argument, but at the same time we need to think about this issue from the patient's perspective. We cannot in good conscience utilize new procedures or technologies unless patients understand the potential risks of a suboptimal result. I want to make sure that patients are fully informed and protected as we apply innovative new approaches.
        Dr John D. Puskas (New York, NY). David, fascinating presentation. I am going to ask a very pointed question, my friend. When will the STS make a second arterial graft a quality metric?
        Dr Shahian. We are seriously considering that, John. We are now at about 98% to 99% utilization for a single arterial graft, but our multiarterial grafting percentages are abysmal.
        Dr Anyanwu. I think we have to differentiate risk aversion. There is always risk awareness, and I think in public reporting states you have risk awareness. I would be interested to know in Massachusetts whether there were other practice changes in those 10 years. For example, were patients with coronary disease getting a mitral valve repaired to move them away from an isolated coronary category? Were people getting more percutaneous coronary interventions? What happened to all the shock patients who had acute myocardial infarctions in Massachusetts? Did they get left ventricular assist devices? Did they just die? What happened to them?
        I think one has to look at risk awareness and look at what happens in the whole big picture as opposed to just in the surgical arena.
        Dr Shahian. To answer your specific questions, we have typically reviewed coronary artery bypass grafting + mitral repair operative notes and charts in Massachusetts to be sure these repairs were not being done to remove the patient from the publicly reported, isolated coronary artery bypass graft category. Regarding percutaneous coronary intervention for shock patients, there was a drop in the percentage of such cases when percutaneous coronary intervention public reporting was instituted but this was reversed when a compassionate use variable was added to our risk models to account for extremely high-risk patient presentations.
        Just one parting comment on risk aversion. There are 3 things that can happen in a so-called risk averse environment, which I think is better characterized as risk aware—2 of those 3 outcomes are positive. The first consequence is the negative consequence we worry most about, which is inappropriate denial of interventions to very high-risk patients. The second consequence, demonstrated in multiple studies, is that in a public reporting environment you get better matching of high-risk patients to programs and surgeons who are more capable, more experienced, have better outcomes, and are generally more suited to taking care of those patients. This is exactly what you would like to have happen. And the third outcome is also positive. The patient with virtually no chance of survival, the hopeless case, is less likely to go to the operating room, which is often the best decision for them and their loved ones.
        Thank you very much.