Advertisement

Fetal growth and gestational age improve outcome predictions in neonatal heart surgery

Open AccessPublished:May 27, 2022DOI:https://doi.org/10.1016/j.jtcvs.2022.05.022

      Abstract

      Objective

      Current risk adjustment models for congenital heart surgery do not fully incorporate multiple factors unique to neonates such as granular gestational age (GA) and birth weight (BW) z score data. This study sought to develop a Neonatal Risk Adjustment Model for congenital heart surgery to address these deficiencies.

      Methods

      Cohort study of neonates undergoing cardiothoracic surgery during the neonatal period captured in the Pediatric Cardiac Critical Care Consortium database between 2014 and 2020. Candidate predictors were included in the model if they were associated with mortality in the univariate analyses. GA and BW z score were both added as multicategory variables. Mortality probabilities were predicted for different GA and BW z scores while keeping all other variables at their mean value.

      Results

      The C statistic for the mortality model was 0.8097 (95% confidence interval, 0.7942-0.8255) with excellent calibration. Mortality prediction for a neonate at 40 weeks GA and a BW z score 0 to 1 was 3.5% versus 9.8% for the same neonate at 37 weeks GA and a BW z score −2 to −1. For preterm infants the mortality prediction at 34 to 36 weeks with a BW z score 0 to 1 was 10.6%, whereas it was 36.1% for the same infant at <32 weeks with a BW z score of −2 to −1.

      Conclusions

      This Neonatal Risk Adjustment Model incorporates more granular data on GA and adds the novel risk factor BW z score. These 2 factors refine mortality predictions compared with traditional risk models. It may be used to compare outcomes across centers for the neonatal population.

      Graphical abstract

      Key Words

      Abbreviations and Acronyms:

      BW (birth weight), CHD (congenital heart disease), CICU (cardiac intensive care unit), GA (gestational age), PC4 (Pediatric Cardiac Critical Care Consortium), STAT Mortality Categories (Society of Thoracic Surgeons–European Association for Cardio-thoracic Surgery Congenital Heart Surgery Mortality Score and associated Categories), STS (Society of Thoracic Surgeons)
      Figure thumbnail fx2
      Refinement of outcome prediction with neonatal PC4 risk adjustment model.
      Incorporating granular data on gestational age and birth weight z score into a specific neonatal risk model for congenital heart disease outcomes leads to improved mortality predictions.
      Currently, several risk adjustment models exist for children with congenital heart disease. They only distinguish between term and preterm neonates. We developed a model specifically for the neonatal population. We show a large variation in individual outcome prediction based on different gestational ages and birth weight z scores.
      Although survival for neonates undergoing congenital heart surgery has improved over recent decades,
      • Jacobs J.P.
      • He X.
      • Mayer Jr., J.E.
      • Austin III, E.H.
      • Quintessenza J.A.
      • Karl T.R.
      • et al.
      Mortality trends in pediatric and congenital heart surgery: an analysis of the Society of Thoracic Surgeons Congenital Heart Surgery Database.
      ,
      • Oster M.E.
      • Lee K.A.
      • Honein M.A.
      • Riehle-Colarusso T.
      • Shin M.
      • Correa A.
      Temporal trends in survival among infants with critical congenital heart defects.
      congenital heart disease (CHD) remains among the leading causes of neonatal mortality in high resource countries.
      • Dolk H.
      • Loane M.
      • Garne E.
      European Surveillance of Congenital Anomalies (EUROCAT) working group. Congenital heart defects in Europe.
      Two risk adjustment tools were developed using best current methodology in children undergoing cardiac surgery to account for case mix and compare outcomes within and across centers: The Society of Thoracic Surgeons (STS)–European Association for Cardio-Thoracic Surgery Congenital Heart Surgery Mortality Score and Associated Categories (STAT Mortality Categories)
      • O’Brien S.M.
      • Clarke D.R.
      • Jacobs J.P.
      • Jacobs M.L.
      • Lacour-Gayet F.G.
      • Pizarro C.
      • et al.
      An empirically based tool for analyzing mortality associated with congenital heart surgery.
      and the Pediatric Cardiac Critical Care Consortium (PC4) Surgical Risk Adjustment Model.
      • Pasquali S.K.
      • Gaies M.
      • Banerjee M.
      • Zhang W.
      • Donohue J.
      • Russell M.
      • et al.
      The quest for precision medicine: unmeasured patient factors and mortality after congenital heart surgery.
      Both risk adjustment tools are empirically derived in contrast to the older Risk Adjustment in Congenital Heart Surgery
      • Jenkins K.J.
      • Gauvreau K.
      • Newburger J.W.
      • Spray T.L.
      • Moller J.H.
      • Iezzoni L.I.
      Consensus-based method for risk adjustment for surgery for congenital heart disease.
      classification that was based on expert opinion. Neither of these tools was specifically developed for neonatal populations.
      Several factors unique to neonates have been identified as risk factors for mortality that have not been fully incorporated in the STS Congenital Heart Surgery Database Mortality Risk or PC4 Surgical Risk Adjustment Model. Both models use both gestational age (GA) and birth weight (BW) z score as dichotomous variables; term (≥37 weeks' GA) versus preterm (<37 weeks' GA) and BW z score <−2 versus ≥−2. However, several studies have demonstrated that not only preterm (<37 weeks' GA) but also early-term infants (37-38 weeks' GA) are at higher risk of mortality compared with their full-term counterparts (≥39 weeks' GA).
      • Costello J.M.
      • Pasquali S.K.
      • Jacobs J.P.
      • He X.
      • Hill K.D.
      • Cooper D.S.
      • et al.
      Gestational age at birth and outcomes after neonatal cardiac surgery.
      ,
      • Steurer M.A.
      • Baer R.J.
      • Keller R.L.
      • Oltman S.
      • Chambers C.D.
      • Norton M.E.
      • et al.
      Gestational age and outcomes in critical congenital heart disease.
      More recently, we showed that BW z scores slightly below average are independent risk factors for mortality and morbidity in neonates who undergo cardiac surgery.
      • Steurer M.A.
      • Peyvandi S.
      • Costello J.M.
      • Moon-Grady A.J.
      • Habib R.H.
      • Hill K.D.
      • et al.
      Association between Z score for birth weight and postoperative outcomes in neonates and infants with congenital heart disease.
      Thus, the aim of this study was to develop a risk adjustment model specifically for neonates undergoing congenital heart surgery. We hypothesize that adding granular data on GA and BW z score improves outcome predictions in neonatal CHD populations.

      Methods

      The PC4 is a quality improvement collaborative that collects data on all patients with primary cardiac disease admitted to the cardiac intensive care unit (CICU) of participating centers and maintains a clinical registry to support research and quality science.
      • Gaies M.
      • Cooper D.S.
      • Tabbutt S.
      • Schwartz S.M.
      • Ghanayem N.
      • Chanani N.K.
      • et al.
      Collaborative quality improvement in the cardiac intensive care unit: development of the Paediatric Cardiac Critical Care Consortium (PC4).
      Data managers of each participating center collect and enter data in accordance with the standardized PC4 Data Definitions Manual.
      • Gaies M.
      • Donohue J.E.
      • Willis G.M.
      • Kennedy A.T.
      • Butcher J.
      • Scheurer M.A.
      • et al.
      Data integrity of the Pediatric Cardiac Critical Care Consortium (PC4) clinical registry.
      The University of Michigan Institutional Review Board provides oversight for the PC4 Data Coordinating Center; this study was reviewed and approved with waiver of informed consent (approval No.: REP00000029; approval date: November 17, 2021).
      All neonates undergoing cardiothoracic surgery with or without cardiopulmonary bypass during the neonatal period (≤30 days of chronological age) who were captured in the PC4 database between August 1, 2014, and July 31, 2020, were included. We excluded patients with BW <2.5 kg without CHD only receiving surgical patent ductus arteriosus ligation and neonates who received pacemaker/implantable cardioverter defibrillator placement only. We also excluded centers with fewer than 50 neonates over the entire study period. The outcome assessed was in-hospital mortality.
      Our candidate predictor variables included GA at birth, BW z score, sex, age at surgery, race/ethnicity, STAT Mortality Categories, extracardiac abnormalities, chromosomal abnormalities or genetic syndromes, insurance type, born at operative center (vs transferred after birth), antenatal diagnosis of CHD, high-risk preoperative neonate (defined as either preoperative cardiac arrest or mechanical circulatory support), preoperative end organ dysfunction (defined as any of the following conditions: preoperative hepatic dysfunction, renal failure, necrotizing enterocolitis, coagulopathy, or bronchopulmonary dysplasia), preoperative mechanical ventilation, venous line, arterial line, sepsis, seizure, neurological condition, vasoactive inotropic support, and venue of immediate preoperative care such as CICU, neonatal intensive care unit, ward, or direct admission to the operating room from the delivery room. For neonates who were not in the CICU preoperatively, mechanical ventilation was only captured if patients were intubated for either respiratory or cardiac reasons or still intubated when going to the operating room; those who were intubated and extubated before the date of the operating room for other reasons were not captured. Similarly for preoperative venous and arterial lines, these were only captured if they were still present at the time of the operation if the patient was not in the CICU preoperatively. To adjust for the complexity of the cardiac operation, we used STAT Mortality Categories. STAT Mortality Categories classify operations into 5 different groups based on operative mortality risk.
      • O’Brien S.M.
      • Clarke D.R.
      • Jacobs J.P.
      • Jacobs M.L.
      • Lacour-Gayet F.G.
      • Pizarro C.
      • et al.
      An empirically based tool for analyzing mortality associated with congenital heart surgery.
      ,
      • Jacobs M.L.
      • Jacobs J.P.
      • Hill K.D.
      • Hornik C.
      • O’Brien S.M.
      • Pasquali S.K.
      • et al.
      The Society of Thoracic Surgeons Congenital Heart Surgery Database: 2017 update on research.
      STAT Mortality Categories are not risk models but tools for risk stratification.
      GA at birth is captured as completed weeks in PC4; that is, 34 and five-sevenths weeks is recorded as 34 weeks. BW z score was calculated using the LMS (ie, lambda for the skew, mu for the median, and sigma for the generalized coefficient of variation) method with GA and sex-specific data provided by Fenton and Sauve.
      • Fenton T.R.
      • Sauve R.S.
      Using the LMS method to calculate z scores for the Fenton preterm infant growth chart.
      For each week of GA at birth the distribution of BWs for that specific week was considered and the BW z score for BW was calculated from this distribution. This method eliminates concern for collinearity between BW and GA. Both GA and BW z score showed a nonlinear relationship with the outcomes. The model fit was similar when categorizing the variables and when using them as splines. For ease of interpretation of the model, the decision was made to use both predictors as categorical variables. GA was categorized according to convention into the following gestational age groups: <32 weeks (very preterm infants), 32 to 33 weeks (moderately preterm infants), 34 to 36 weeks (late preterm infants), 37 to 38 weeks (early-term infants) and >38 weeks (full-term infants). Similarly, BW z score was categorized into the following groups: <−2, ≥−2 to <−1, ≥−1 to <0, ≥0 to <+1, ≥+1 to <+2 and ≥+2.
      The association between each predictor and the primary outcome was assessed in descriptive analyses. The χ2 or Fisher exact test was used for categorical variables, and Wilcoxon rank-sum test was used for continuous variables, as appropriate. All predictors with a P < .05 in these univariate analyses were entered into the 2-level multivariable logistic regression model with center as a random effect. Collinearities between variables were assessed by calculating variance inflation factors. The model was checked for interactions between age at surgery and STAT Mortality Categories, age at surgery, GA at birth and BW z score, and STAT Mortality Category. Details on missing data points are listed in Table E1.
      Ten-fold cross-validation was performed to avoid overfitting. C statistics with CIs are presented to assess discrimination of the models. Calibration of the model was assessed with calibration plots, Sanders-modified Brier score, and concordance correlation coefficient.
      To compare the Neonatal Risk Adjustment Model to the PC4 Surgical Risk Adjustment Model, we calculated the C statistics for the PC4 Surgical Risk Adjustment Model restricted to the neonatal population. We also calculated the C statistics with and without our 2 main predictors of interest: GA and BW z score. Given the limitations of the C statistic, we also calculated predicted mortality and 95% CI for neonates with different GA and BW z score categories while keeping all other variables included in the respective models at their mean value. Observed-to-expected mortality ratios (O/E) at 95%, 90%, 85% and 80% CIs were calculated for each participating center. All analyses were performed either using SAS version 9.4 (SAS Institute) or STATA version 15 (Stata Corp).

      Results

      There were 9636 neonates included in this study. There were 13.7% (1325 out of 9636) born preterm (<37 weeks GA), and median age of surgery was 7 days (interquartile range, 4-11 days). 50.6% (4871 out of 9636) were STAT mortality category 4. Other baseline characteristics are shown in Table 1. In-hospital mortality was 7.3% (703 out of 9636).
      Table 1Predictor variables and univariate associations
      VariableSurvived (n = 8933)Died (n = 703)P value
      Candidate predictor
      Gestational age (wk)39 (37-39)38 (36-39).000
      Gestational age category (wk)
       <3256 (0.6)22 (3.1).000
       32-33116 (1.3)34 (4.8)
       34-36953 (10.7)144 (20.5)
       37-383124 (35)255 (36.3)
       ≥394684 (52.4)248 (35.3)
      BW (kg)3.2 (2.8-3.5)2.9 (2.4-3.3).000
      BW z score category.000
       <−2365 (4.1)50 (7.1)
       ≥−2 to <−11617 (18.1)164 (23.3)
       ≥−1 to <03459 (38.7)271 (38.5)
       ≥0 to <+12541 (28.4)141 (20.1)
       ≥+1 to <+2754 (8.4)62 (8.8)
       ≥+2197 (2.2)15 (2.1)
      Male sex5323 (59.6)386 (54.9).015
      Age at surgery (d)7 (4-11)6 (4-10).000
      Race/ethnicity.002
       Non-Hispanic White4764 (53.3)331 (47.1)
       Non-Hispanic Black1065 (11.9)111 (15.8)
       Hispanic1492 (16.7)132 (18.8)
       Other1612 (18)129 (18.3)
      STAT Mortality Category.000
       1278 (3.1)4 (0.6)
       21254 (14)34 (4.8)
       31282 (14.4)24 (3.4)
       44473 (50.1)398 (56.6)
       51638 (18.3)243 (34.6)
       Missing8 (0.1)0 (0)
      Extracardiac abnormalities1529 (17.1)215 (30.6).000
      Chromosomal abnormality or genetic syndrome1641 (18.3)218 (31.0).000
      Insurance type.000
       Public4144 (46.4)384 (54.6)
       Private3988 (44.6)252 (35.8)
       Non-US insurance21 (0.2)2 (0.3)
       None/self216 (2.4)28 (4)
       Unknown564 (6.3)37 (5.3)
      Inborn.198
       No5613 (62.8)419 (59.6)
       Yes2834 (31.7)239 (34.0)
       Unknown486 (5.4)45 (6.4)
      Venue of preoperative care immediately preceding surgery.000
       CICU6448 (72.2)531 (75.5)
       NICU1871 (20.9)108 (15.4)
       Ward307 (3.4)13 (1.8)
       Operating room
      This group consisted of total anomalous pulmonary venous return (n = 155 [41.3%]), hypoplastic left heart syndrome, or other single ventricle (n = 56 [15.4%]) and others (n = 147 [43.3%]). It includes neonates who went straight to the operating room from the delivery room or were transferred from an outside hospital and admitted directly to the operating room.
      307 (3.4)51 (7.3)
      Antenatal diagnosis of CHD5498 (61.5)534 (76.0).000
      High risk preoperative neonate
      Defined as either preoperative cardiac arrest or mechanical circulatory support.
      95 (1.1)59 (8.4)<.001
       Cardiac arrest72 (0.8)20 (2.8).000
       Mechanical circulatory support29 (0.3)42 (6).000
      Preoperative end organ dysfunction678 (7.6)116 (16.5).000
       Hepatic dysfunction72 (0.8)10 (1.4).087
       Renal failure208 (2.3)44 (6.3).000
       NEC13 (0.1)3 (0.4).106
       Coagulopathy453 (5.1)70 (10).000
       BPD35 (0.4)12 (1.7).000
      Preoperative mechanical ventilation2631 (29.5)314 (44.7).000
      Preoperative venous line7130 (79.8)601 (85.5).000
      Preoperative arterial line4526 (50.7)443 (63).000
      Preoperative sepsis32 (0.4)2 (0.3)>.999
      Preoperative seizure82 (0.9)8 (1.1).559
      Preoperative neurological condition130 (1.6)14 (2.0).374
      Preoperative neurological condition within 48 h17 (0.2)2 (0.3).645
      Preoperative vasoactive support1251 (14.0)233 (33.1).000
      Values are presented as median (interquartile range) or n (%). BW, Birth weight; STAT, The Society of Thoracic Surgeons (STS)–European Association for Cardio-Thoracic Surgery Congenital Heart Surgery Mortality Score and Associated Categories; CICU, cardiac intensive care unit; NICU, neonatal intensive care unit; CHD, congenital heart disease; NEC, necrotizing enterocolitis; BPD, bronchopulmonary dysplasia.
      This group consisted of total anomalous pulmonary venous return (n = 155 [41.3%]), hypoplastic left heart syndrome, or other single ventricle (n = 56 [15.4%]) and others (n = 147 [43.3%]). It includes neonates who went straight to the operating room from the delivery room or were transferred from an outside hospital and admitted directly to the operating room.
      Defined as either preoperative cardiac arrest or mechanical circulatory support.
      Table 1 shows the univariate analyses of all assessed predictors with our primary outcome of in-hospital mortality. Seventeen out of 22 predictors were significantly associated with in-hospital mortality and included in the final model. None of the tested interaction terms significantly improved the C statistic. There was no concerning collinearity between the predictors with all variance inflation factors <10. Discrimination of the model for in-hospital mortality was good with a bias corrected C statistic of 0.8097 (95% CI, 0.7942-0.8255). The 10-fold cross validated C statistic was slightly lower at 0.7903 (95% CI, 0.7720-0.8064). Calibration of the model was excellent up to an observed mortality of 40%. At a higher observed mortality, the model overestimated the outcome slightly (Figure 1). In comparison, the bias corrected C statistics for the standard PC4 Surgical Risk Adjustment Model restricted to the neonatal population was lower at 0.7824. The Sanders-modified Brier score was 0.0606. The concordance correlation coefficient was 0.997.
      Figure thumbnail gr1
      Figure 1Calibration plot for mortality model. The calibration plot shows the observed mortality by deciles on the y-axis and the corresponding expected mortality on the x-axis.
      Table 2 shows the multivariable prediction model for in-hospital mortality. GA was an important predictor: GA <32 weeks had an odds ratio for mortality of 10.01 (95% CI, 5.32-18.82) compared with GA >38 weeks. Low-BW z score categories of <−1, male sex, age at surgery, extracardiac abnormalities, chromosomal abnormalities or syndrome, direct admission from the operating room, antenatal diagnosis of CHD, high-risk preoperative neonate, preoperative end organ dysfunction, preoperative mechanical ventilation, and preoperative vasoactive inotropic support were all independently associated with in-hospital mortality in the multivariable model.
      Table 2Multivariable logistic regression model for mortality
      VariableOdds ratio (95% CI)P value
      Gestational age (wk)
       < 3210.01 (5.32-18.82)<.001
       32-336.18 (3.91-9.79)<.001
       34-362.63 (2.06-3.34)<.001
       37-381.30 (1.07-1.58).007
       ≥39Reference
      BW z score category
       <−21.75 (1.24-2.49).002
       ≥−2 to <−11.23 (0.99-1.54).061
       ≥−1 to <0Reference
       ≥0 to <+10.72 (0.58-0.90).005
       ≥+1 to < +21.06 (0.78-1.44).709
       ≥+20.56 (0.31-1.04).069
      Male sex0.88 (0.74-1.04).137
      Age at surgery (d)0.97 (0.96-0.99)<.001
      Race/ethnicity
       Non-Hispanic WhiteReference
       Non-Hispanic Black1.16 (0.90-1.50).250
       Hispanic1.26 (0.99-1.61).059
       Other1.28 (1.00-1.65).053
      STAT Mortality Category
       1Reference
       21.42 (0.49-4.13).523
       31.11 (0.37-3.33).849
       44.53 (1.64-12.51).004
       58.19 (2.93-22.88).000
      Extracardiac abnormalities1.57 (1.29-1.91)<.001
      Chromosomal abnormality or genetic syndrome1.58 (1.30-1.92)<.001
      Insurance type
       PublicReference
       Private0.80 (0.66-0.97).026
       Non-US insurance0.96 (0.21-4.45).960
       None/self1.81 (1.15-2.86).010
       Unknown0.93 (0.58-1.50).773
      Venue of preoperative care immediately preceding surgery
       CICUReference
       NICU1.19 (0.90-1.58).225
       Ward1.09 (0.60-1.98).787
       Operating room3.51 (2.33-5.29).000
      Antenatal diagnosis of CHD1.63 (1.33-1.99)<.001
      High-risk preoperative neonate
      Defined as either preoperative cardiac arrest or mechanical circulatory support.
      6.05 (4.07-8.99)<.001
      Preoperative end organ dysfunction1.57 (1.21-2.02).001
      Preoperative mechanical ventilation1.64 (1.33-2.01)<.001
      Preoperative venous line1.08 (0.81-1.44).607
      Preoperative arterial line1.20 (0.98-1.46).081
      Preoperative vasoactive support1.79 (1.46-2.20)<.001
      BW, Birth weight; STAT, The Society of Thoracic Surgeons (STS)–European Association for Cardio-Thoracic Surgery Congenital Heart Surgery Mortality Score and Associated Categories; CICU, cardiac intensive care unit; NICU, neonatal intensive care unit; CHD, congenital heart disease.
      Defined as either preoperative cardiac arrest or mechanical circulatory support.
      The cross-validated C statistic without our 2 key factors—BW z score and GA—was 0.7656 (95% CI, 0.7514-0.7865), when these 2 factors were added to the model, the cross-validated C statistic rose to 0.7903 (95% CI, 0.7720-0.8064). However, the value of adding these 2 factors to the model becomes more apparent when looking at predicted probabilities for mortality; Table 3 shows the more refined mortality predictions of the Neonatal Risk Adjustment Model compared to the PC4 Surgical Risk Adjustment Model based on GA and BW z score while keeping all other predictors in the respective model at their mean value.
      Table 3Predicted probabilities for the surgical and Neonatal Risk Adjustment Model
      PredictorSurgical Risk Adjustment ModelNeonatal Risk Adjustment Model
      Term/early-term infant
       40 wk GA, BW z score 0 to 16.1 (5.6-6.6)3.5 (2.8-4.1)
       40 wk GA, BW z score −2 to −16.5 (5.4-7.6)
       37 wk GA, BW z score 0 to 15.3 (4.4-6.3)
       37 wk GA, BW z score −2 to −19.8 (8.2-11.3)
      Preterm infant
       34-36 wk GA, BW z score 0 to 115.1 (13.3-16.9)10.6 (8.5-12.8)
       34-36 wk GA, BW z score −2 to −114.7 (12.1-17.3)
       32-33 wk GA, BW z score 0 to 122.6 (16.0-29.2)
       32-33 wk GA, BW z score −2 to −126.2 (18.9-33.6)
       <32 wk GA, BW z score 0 to 123.5 (16.0-31.0)
       <32 wk GA, BW z score −2 to −136.1 (25.7-46.5)
      Values are presented as predicted probability (%) (95% CI). All other predictors included in the respective models are kept at their mean value. GA, Gestational age; BW, birth weight.
      Figure 2, A, shows the O/E ratio in-hospital mortality ratio sorted by hospital center in a caterpillar plot. The lowest O/E ratio was 0.51 (95% CI, 0.29-0.78) and the highest was 1.9 (90% CI, 1.40-2.35). Figure 2, B, shows the lack of association between O/E ratio and center volume (Spearman ρ = −0.0551; P = .75). Table 4 shows the performance of center for different levels of CIs: at 80% CI, 23 out of 35 (65.7%) hospitals performed as expected, whereas at 95% CI, 30 out of 35 (85.7%) hospitals performed as expected.
      Figure thumbnail gr2
      Figure 2Center-to-center variability. A, Caterpillar plot for observed to expected (O/E) mortality ratio for each center. O/E for mortality is shown on the y-axis, the x-axis shows the order number of each hospital. With upper and lower 90% CI. B, Center volume. O/E for mortality is shown on the y-axis, the x-axis shows center volume. The line depicts linear predictions. Spearman ρ is −0.0551 (P = .75).
      Table 4Performance of hospitals (N = 35) for different CIs
      CIPerforming worse than expectedPerforming as expectedPerforming better than expected
      80%5 (14.3)23 (65.7)7 (20)
      85%4 (11.4)25 (71.4)6 (17.1)
      90%3 (8.6)27 (77.1)5 (14.3)
      95%3 (8.6)30 (85.7)2 (5.7)
      Values are presented as n (%).

      Discussion

      We present a risk adjusted mortality model specifically for neonates undergoing congenital heart surgery within the first 30 days of life that demonstrates high discrimination even after cross-validation and good calibration. The incorporation of more granular data on GA and BW z score leads to a more refined mortality prediction. This Neonatal Risk Adjustment Model confirms the important influence of fetal growth restriction on mortality that we have described in other populations of neonates with CHD.
      • Steurer M.A.
      • Peyvandi S.
      • Costello J.M.
      • Moon-Grady A.J.
      • Habib R.H.
      • Hill K.D.
      • et al.
      Association between Z score for birth weight and postoperative outcomes in neonates and infants with congenital heart disease.
      ,
      • Jacobs M.L.
      • Jacobs J.P.
      • Hill K.D.
      • Hornik C.
      • O’Brien S.M.
      • Pasquali S.K.
      • et al.
      The Society of Thoracic Surgeons Congenital Heart Surgery Database: 2017 update on research.
      Additionally, we show that significant variation in center outcome exists (Figure 3).
      Figure thumbnail gr3
      Figure 3The study's methods, results, and implications. PC4, Pediatric Cardiac Critical Care Consortium.
      The PC4 Surgical Risk Adjustment Model and the STS Congenital Heart Surgery Database Mortality Risk Adjustment Model represent the current gold standards in pediatric cardiac surgical mortality case mix adjustment.
      • Pasquali S.K.
      • Gaies M.
      • Banerjee M.
      • Zhang W.
      • Donohue J.
      • Russell M.
      • et al.
      The quest for precision medicine: unmeasured patient factors and mortality after congenital heart surgery.
      ,
      • O’Brien S.M.
      • Jacobs J.P.
      • Pasquali S.K.
      • Gaynor J.W.
      • Karamlou T.
      • Welke K.F.
      • et al.
      The Society of Thoracic Surgeons Congenital Heart Surgery Database mortality risk model: part 1—statistical methodology.
      ,
      • Jacobs J.P.
      • O’Brien S.M.
      • Pasquali S.K.
      • Gaynor J.W.
      • Mayer Jr., J.E.
      • Karamlou T.
      • et al.
      The Society of Thoracic Surgeons Congenital Heart Surgery Database mortality risk model: part 2—clinical application.
      Both incorporate several risk factors that apply to neonates such as prematurity, presence of chromosomal abnormalities, and certain preoperative conditions such as shock or mechanical ventilation for cardiorespiratory failure. However, in both, prematurity is used as a dichotomous variable (<37 vs ≥ 37 weeks' gestation). Consistent with prior literature,
      • Costello J.M.
      • Pasquali S.K.
      • Jacobs J.P.
      • He X.
      • Hill K.D.
      • Cooper D.S.
      • et al.
      Gestational age at birth and outcomes after neonatal cardiac surgery.
      ,
      • Steurer M.A.
      • Baer R.J.
      • Keller R.L.
      • Oltman S.
      • Chambers C.D.
      • Norton M.E.
      • et al.
      Gestational age and outcomes in critical congenital heart disease.
      this model identifies early-term infants of 37 to 38 weeks’ GA at higher risk of mortality after congenital heart surgery than full-term neonates.
      Another risk factor that is not included in the PC4 Surgical Risk Adjustment Model or the STS Congenital Heart Surgery Database Mortality Risk Adjustment Model is BW z score—a surrogate marker for fetal growth. The influence of BW z score on outcomes in neonates with CHD is a relatively new finding but has been shown to be predictive in different cohorts of neonates with CHD
      • Steurer M.A.
      • Peyvandi S.
      • Costello J.M.
      • Moon-Grady A.J.
      • Habib R.H.
      • Hill K.D.
      • et al.
      Association between Z score for birth weight and postoperative outcomes in neonates and infants with congenital heart disease.
      ,
      • Steurer M.A.
      • Baer R.J.
      • Burke E.
      • Peyvandi S.
      • Oltman S.
      • Chambers C.D.
      • et al.
      Effect of fetal growth on 1-year mortality in neonates with critical congenital heart disease.
      and was validated in our model. Moving forward, BW z score should be included as a multicategorical term when assessing case mix and performing benchmarking in the neonatal population with CHD.
      When comparing the C statistics between the current neonatal model and the standard PC4 Surgical Risk Adjustment model we see only a slight increase in the C statistics (from 0.7824 to 0.8097). However, it is important to recognize that the C statistics solely assesses overall model discrimination and is not necessarily a good measure of prognostic ability for certain subgroups of patients. For this purpose, we calculated predicted mortalities for individual GA and BW categories. In the traditional Surgical Risk Adjustment model, a term neonate has a predicted mortality of 6.1% irrespective if the baby is born at 37 weeks' or at 40 weeks' gestation, and irrespective of z score for BW. The current neonatal risk model is able to refine this prognosis: a term infant born at 40 weeks’ GA with a z score for BW from 0 to 1 has a 3.5% predicted mortality, whereas an infant born at gestational age 37 weeks with a z score for BW −2 to −1, has a predicted mortality of 9.8%. Even larger differences in predicted mortality were seen for preterm infants (Table 3). This illustrates nicely the importance and value of adding more refined neonatal predictor variable.
      Unfortunately, we were not able to directly compare our model to the STS Congenital Heart Disease Database Mortality Risk Adjustment Model because we did not have access to the STS dataset and model details. A further investigation should incorporate the more granular GA and BW z score data into the STS Congenital Heart Disease Database Mortality Risk Adjustment Model and assess model performance and individual predictions.
      Despite accounting for the many risk factors available in the PC4 database, we found significant center-to-center variability for mortality in neonates with CHD with center-specific O/E ratios ranging from 0.51 to 1.9. At a CI level of 95%, 85.7% of all centers perform as expected, whereas only about two-thirds of all centers perform as expected at an 80% CI level. Center-to-center variability has been described across multiple settings in surgical outcomes for congenital heart disease.
      • Pasquali S.K.
      • Li J.S.
      • Burstein D.S.
      • Sheng S.
      • O’Brien S.M.
      • Jacobs M.L.
      • et al.
      Association of center volume with mortality and complications in pediatric heart surgery.
      • Hirsch J.C.
      • Gurney J.G.
      • Donohue J.E.
      • Gebremariam A.
      • Bove E.L.
      • Ohye R.G.
      Hospital mortality for Norwood and arterial switch operations as a function of institutional volume.
      • Tabbutt S.
      • Schuette J.
      • Zhang W.
      • Alten J.
      • Donohue J.
      • Gaynor J.W.
      • et al.
      A novel model demonstrates variation in risk-adjusted mortality across pediatric cardiac ICUs after surgery.
      Although previous studies have demonstrated a volume-outcome relationship with lower surgical mortality at higher volume centers,
      • Pasquali S.K.
      • Li J.S.
      • Burstein D.S.
      • Sheng S.
      • O’Brien S.M.
      • Jacobs M.L.
      • et al.
      Association of center volume with mortality and complications in pediatric heart surgery.
      ,
      • Hirsch J.C.
      • Gurney J.G.
      • Donohue J.E.
      • Gebremariam A.
      • Bove E.L.
      • Ohye R.G.
      Hospital mortality for Norwood and arterial switch operations as a function of institutional volume.
      our data did not find an association between the O/E ratio and volume of neonatal cases. The current study only included about one-third of the congenital heart surgeon centers in the United States, which were all relatively early adopters of PC4 and thus highly invested in improving quality, thus it may be underpowered to detect any volume–outcome relationship. As a quality improvement collaborative, PC4 strives to identify high-performing centers with a goal of sharing best practices and care models, and future projects should focus on these important questions. By utilizing a collaborative, unblinded platform (like PC4) higher-performing centers may be able to share clinical practices to improve outcomes at lower-performing centers.
      This study has several limitations. First and foremost is the need for external validation. We performed cross-validation of our C statistics to avoid overfitting; however, validation on a different cohort of neonates is warranted. Additionally, because the preoperative venue was not standardized with nearly 28% of the neonates going to the operating room from a non-CICU, a small number of preoperative data points that were only collected in the CICU were not included. Lastly, there is a relatively low number of neonates <32 weeks’ gestation, so the results for this age group should be interpreted with caution. This emphasizes the importance of creating databases that collect information on this patient group even if they are not admitted to a CICU.

      Conclusions

      We show that developing a risk adjustment model specifically for neonates undergoing congenital heart surgery is feasible with high discrimination and good calibration. However, it has to be noted that at higher outcome probabilities the model is slightly overestimating mortality. Incorporating granular data on 2 of the most important neonatal risk factors, GA and BW z score, leads to improved outcome predictions (Video 1). In addition, we show wide center variation in risk-adjusted mortality following neonatal heart surgery suggesting opportunities for improvement through collaboration.

      Conflict of Interest Statement

      The authors reported no conflicts of interest.
      The Journal policy requires editors and reviewers to disclose conflicts of interest and to decline handling or reviewing manuscripts for which they may have a conflict of interest. The editors and reviewers of this article have no conflicts of interest.

      Supplementary Data

      Appendix

      Table E1Approach to missing data
      Risk FactorN%Solution
      Race8749.1%Listed as “other”
      Ethnicity5896.1%Listed as “other”
      Insurance type6016.2%Listed as “unknown”
      Inborn4865.4%Listed as “unknown”
      Birth weight2782.9%7 observations had surgery at age day 1 of life and surgical weight was used as birth weight
      For the remaining 271 observations, ordinary least square regression was used to predict birth weight (outcome, birth weight; predictors, surgical weight and surgical age). To check the accuracy of the imputation model, the predicted and observed birth weight correlation was calculated in all neonates with nonmissing birth weight (correlation = 0.928).
      Gestational age (wk)1231.3%Dichotomous variable for prematurity indicated that none of these neonates were born preterm. All missing observations were set to ≥ 39 weeks.
      STAT Mortality Category80.1%Listed as “missing” in univariate analysis, excluded from multivariable regression.
      STAT, The Society of Thoracic Surgeons (STS)–European Association for Cardio-Thoracic Surgery Congenital Heart Surgery Mortality Score and Associated Categories.

      References

        • Jacobs J.P.
        • He X.
        • Mayer Jr., J.E.
        • Austin III, E.H.
        • Quintessenza J.A.
        • Karl T.R.
        • et al.
        Mortality trends in pediatric and congenital heart surgery: an analysis of the Society of Thoracic Surgeons Congenital Heart Surgery Database.
        Ann Thorac Surg. 2016; 102: 1345-1352https://doi.org/10.1016/j.athoracsur.2016.01.071
        • Oster M.E.
        • Lee K.A.
        • Honein M.A.
        • Riehle-Colarusso T.
        • Shin M.
        • Correa A.
        Temporal trends in survival among infants with critical congenital heart defects.
        Pediatrics. 2013; 131: e1502-e1508https://doi.org/10.1542/peds.2012-3435
        • Dolk H.
        • Loane M.
        • Garne E.
        European Surveillance of Congenital Anomalies (EUROCAT) working group. Congenital heart defects in Europe.
        Circulation. 2011; 123: 841-849https://doi.org/10.1161/circulationaha.110.958405
        • O’Brien S.M.
        • Clarke D.R.
        • Jacobs J.P.
        • Jacobs M.L.
        • Lacour-Gayet F.G.
        • Pizarro C.
        • et al.
        An empirically based tool for analyzing mortality associated with congenital heart surgery.
        J Thorac Cardiovasc Surg. 2009; 138: 1139-1153https://doi.org/10.1016/j.jtcvs.2009.03.071
        • Pasquali S.K.
        • Gaies M.
        • Banerjee M.
        • Zhang W.
        • Donohue J.
        • Russell M.
        • et al.
        The quest for precision medicine: unmeasured patient factors and mortality after congenital heart surgery.
        Ann Thorac Surg. 2019; 108: 1889-1894https://doi.org/10.1016/j.athoracsur.2019.06.031
        • Jenkins K.J.
        • Gauvreau K.
        • Newburger J.W.
        • Spray T.L.
        • Moller J.H.
        • Iezzoni L.I.
        Consensus-based method for risk adjustment for surgery for congenital heart disease.
        J Thorac Cardiovasc Surg. 2002; 123: 110-118https://doi.org/10.1067/mtc.2002.119064
        • Costello J.M.
        • Pasquali S.K.
        • Jacobs J.P.
        • He X.
        • Hill K.D.
        • Cooper D.S.
        • et al.
        Gestational age at birth and outcomes after neonatal cardiac surgery.
        Circulation. 2014; 129: 2511-2517https://doi.org/10.1161/circulationaha.113.005864
        • Steurer M.A.
        • Baer R.J.
        • Keller R.L.
        • Oltman S.
        • Chambers C.D.
        • Norton M.E.
        • et al.
        Gestational age and outcomes in critical congenital heart disease.
        Pediatrics. 2017; 140e20170999https://doi.org/10.1542/peds.2017-0999
        • Steurer M.A.
        • Peyvandi S.
        • Costello J.M.
        • Moon-Grady A.J.
        • Habib R.H.
        • Hill K.D.
        • et al.
        Association between Z score for birth weight and postoperative outcomes in neonates and infants with congenital heart disease.
        J Thorac Cardiovasc Surg. 2021; 162: 1838-1847https://doi.org/10.1016/j.jtcvs.2021.01.065
        • Gaies M.
        • Cooper D.S.
        • Tabbutt S.
        • Schwartz S.M.
        • Ghanayem N.
        • Chanani N.K.
        • et al.
        Collaborative quality improvement in the cardiac intensive care unit: development of the Paediatric Cardiac Critical Care Consortium (PC4).
        Cardiol Young. 2015; 25: 951-957https://doi.org/10.1017/s1047951114001450
        • Gaies M.
        • Donohue J.E.
        • Willis G.M.
        • Kennedy A.T.
        • Butcher J.
        • Scheurer M.A.
        • et al.
        Data integrity of the Pediatric Cardiac Critical Care Consortium (PC4) clinical registry.
        Cardiol Young. 2016; 26: 1090-1096https://doi.org/10.1017/s1047951115001833
        • Jacobs M.L.
        • Jacobs J.P.
        • Hill K.D.
        • Hornik C.
        • O’Brien S.M.
        • Pasquali S.K.
        • et al.
        The Society of Thoracic Surgeons Congenital Heart Surgery Database: 2017 update on research.
        Ann Thorac Surg. 2017; 104: 731-741https://doi.org/10.1016/j.athoracsur.2017.07.001
        • Fenton T.R.
        • Sauve R.S.
        Using the LMS method to calculate z scores for the Fenton preterm infant growth chart.
        Eur J Clin Nutr. 2007; 61: 1380-1385https://doi.org/10.1038/sj.ejcn.1602667
        • O’Brien S.M.
        • Jacobs J.P.
        • Pasquali S.K.
        • Gaynor J.W.
        • Karamlou T.
        • Welke K.F.
        • et al.
        The Society of Thoracic Surgeons Congenital Heart Surgery Database mortality risk model: part 1—statistical methodology.
        Ann Thorac Surg. 2015; 100: 1054-1062https://doi.org/10.1016/j.athoracsur.2015.07.014
        • Jacobs J.P.
        • O’Brien S.M.
        • Pasquali S.K.
        • Gaynor J.W.
        • Mayer Jr., J.E.
        • Karamlou T.
        • et al.
        The Society of Thoracic Surgeons Congenital Heart Surgery Database mortality risk model: part 2—clinical application.
        Ann Thorac Surg. 2015; 100: 1063-1070https://doi.org/10.1016/j.athoracsur.2015.07.011
        • Steurer M.A.
        • Baer R.J.
        • Burke E.
        • Peyvandi S.
        • Oltman S.
        • Chambers C.D.
        • et al.
        Effect of fetal growth on 1-year mortality in neonates with critical congenital heart disease.
        J Am Heart Assoc. 2018; 7e009693https://doi.org/10.1161/jaha.118.009693
        • Pasquali S.K.
        • Li J.S.
        • Burstein D.S.
        • Sheng S.
        • O’Brien S.M.
        • Jacobs M.L.
        • et al.
        Association of center volume with mortality and complications in pediatric heart surgery.
        Pediatrics. 2012; 129: e370-e376https://doi.org/10.1542/peds.2011-1188
        • Hirsch J.C.
        • Gurney J.G.
        • Donohue J.E.
        • Gebremariam A.
        • Bove E.L.
        • Ohye R.G.
        Hospital mortality for Norwood and arterial switch operations as a function of institutional volume.
        Pediatr Cardiol. 2008; 29: 713-717https://doi.org/10.1007/s00246-007-9171-2
        • Tabbutt S.
        • Schuette J.
        • Zhang W.
        • Alten J.
        • Donohue J.
        • Gaynor J.W.
        • et al.
        A novel model demonstrates variation in risk-adjusted mortality across pediatric cardiac ICUs after surgery.
        Pediatr Crit Care Me. 2019; 20: 136-142https://doi.org/10.1097/pcc.0000000000001776