Amniotic Fluid Embolism: A Case Study and Literature Review

Ryan J Elsey DO^1*, Mary K Moats-Biechler OMS-IV², Michael W Faust MD³, Jennifer A Cooley CRNA-APRN⁴, Sheela Ahari MD⁴, and Douglas T Summerfield MD¹

Departments of Internal Medicine¹,Obstetrics and Gynecology³,and Anesthesia⁴

¹Mercy Medical Center—North Iowa

Mason City, IA USA

²A.T. Still University

Kirksville, MO USA

Abstract

Amniotic fluid embolus is a rare and life threatening peripartum complication that requires quick recognition and emergent interdisciplinary management to provide the best chance of a positive outcome for the mother and infant. The following case study demonstrates the importance of quick recognition as well as an interdisciplinary approach in caring for such a condition.  A literature review regarding the current recommendations for management of this condition follows as well as a proposed treatment algorithm.

Introduction

Amniotic fluid embolus (AFE) is a rare and life-threatening complication of pregnancy; a recent population-based review found an estimated incidence ranging from 1 in 15,200 deliveries in North America and 1 in 53,800 deliveries in Europe (1). Mortality rates vary but have been reported to range from 11% to more than 60%, with the most recent population-based studies in the United States reporting a 21.6% fatality rate (1-4).  Despite best efforts, it remains one of the leading causes of maternal death (1,5,6). However, rapid diagnosis of AFE and immediate obstetric and intensive care has proven to play a decisive role in maternal prognosis and survival (7-9).

In 2016, uniform diagnostic criteria were proposed for reporting on cases of AFE. First, a report of AFE requires a sudden onset of cardiorespiratory arrest, which consists of both hypotension (systolic blood pressure < 90 mmHg) and respiratory compromise (dyspnea, cyanosis, or SpO2 < 90%). Secondly, overt disseminated intravascular coagulation (DIC) must be documented following the appearance of signs or symptoms using a standardized scoring system. Coagulopathy must be detected prior to a loss of sufficient blood to account for dilutional or shock-related consumptive coagulopathy. Third, the clinical onset must occur during labor or within 30 minutes of delivery of the placenta. Fourth, no fever ≥ 38.0° C during labor can occur (10).

The following case study qualifies as a reportable incidence of an AFE under the above criteria and further demonstrates the ability to successfully stabilize a patient with AFE due to quick recognition, interdisciplinary cooperation, and effective supportive management.

Case Presentation

A 34-year-old gravida 5, para 1-1-2-2, presented at 36 weeks and 1-day gestation for induction of labor. Her past medical history included esophageal atresia at birth and a past pregnancy complicated by preterm, premature rupture of the membranes. Initial labs at admission were significant for a hemoglobin of 12.2 g/dL and a platelet count of 234 x10³ u/L. The patient was subsequently started on lactated ringers at 125 ml/hr. As the patient's labor progressed, an epidural was placed 3 hours after admission. Four hours and 42 minutes after admission, an artificial rupture of the membranes was performed.

Eighteen minutes after the artificial rupture of the membranes was performed, the patient was noted to have seizure-like activity. She was given an intravenous (IV) fluid bolus and ephedrine, and the anesthesia provider was emergently contacted. When anesthesia arrived, the patient was noted to be cyanotic in bed. Patient vitals and exam were significant for emesis, a heart rate of 50 beats per minute (bpm), systolic blood pressure in the low 70s (mmHg), and a fetal heart rate in the 70s.

The differential diagnosis at this time was broad and included anesthesia drug reactions such as an intravascular epidural migration, pulmonary thromboembolism, eclampsia, or even an aortic dissection. A pulmonary embolism was felt to be unlikely due to the patient's bradycardia and sudden neurologic changes. Eclampsia was less likely at the time due to no signs of pre-eclampsia in the patient as well as the patient's current bradycardia and hypotension. Given the patient's absence of Marfan syndrome, aortic dissection was not considered to be a high probability. The patient did have signs consistent with an intravascular epidural including altered mental status, cyanosis, bradycardia, hypotension, vomiting, and a low fetal heart rate. However, at the time anesthesia felt she was more likely suffering from an acute embolic process given the timeframe between the artificial rupture of the membranes and the onset of her symptoms.

Given the patient's instability, she was emergently taken for a cesarean section and intubated to provide airway stabilization. The cesarean section began 15 minutes after seizure like symptoms started and upon delivery, the infant was subsequently transferred to a tertiary center for therapeutic hypothermia.

Intraoperatively, the patient was noted to maintain a peripheral capillary oxygen saturation (SpO2) of >90%. However, end tidal C02 was elevated to 54 mmHg despite hyperventilation and peak airway pressures were elevated to 38 cmH₂O. Albuterol and sevoflurane were subsequently utilized in an attempt to increase bronchodilation. Following completion of the caesarian section, peak airway pressures normalized to less than 30 cmH₂O but end tidal CO2 levels remained as high as 52 mmHg despite hyperventilation. Blood pressure was significant for systolic pressure of 80 mmHg.  IV phenylephrine was administered. Additionally, uterine massage was performed to aid in hemorrhage control and the patient was administered IV oxytocin, methylergonovine maleate, carboprost, and vaginal misoprostol.  A repeat complete blood count was performed one hour after symptom onset which showed a hemoglobin of 10.3 g/dL and a platelet count of 103 x10³ u/L.

In this case, the patient’s care team had a high suspicion of an AFE with symptoms that followed the uniform diagnostic criteria for an AFE. The patient had hemodynamic instability, coinciding with the recent rupture of membranes. Her systolic blood pressure was < 90 mmHg and her end tidal C02 levels (in mmHg) were elevated to the high 40s and low 50s. The critical care team was notified of her condition and the patient was subsequently transferred to the Intensive Care Unit (ICU) on mechanical ventilation and sedated with fentanyl and versed.

Upon arrival to the ICU, a DIC panel was performed revealing DIC. Labs showed a fibrinogen level of 52 mg/dL, A D-dimer greater than 128,000 ng/mL, and a platelet count of 80,000 u/L despite the administration of one pooled unit of platelets. The patient's international normalized ratio (INR) was 1.3 with a baseline INR of 0.9. Due to multiple laboratory abnormalities and a clinical condition consistent with DIC, aggressive transfusions were performed per the standard of care for patients suffering with DIC. A peripheral smear was obtained revealing schistocytes (Figure 1) which verified the DIC diagnosis.

Figure 1. The patient's peripheral blood smear four hours after onset of symptoms which demonstrates schistocytes indicative of DIC.

Hematology was emergently consulted and it was recommended to avoid additional platelet transfusions unless platelet counts dropped below 10,000 to 20,000 u/L. One milligram (mg) of subcutaneous phytonadione was also given five hours after symptom onset in an effort to decrease bleeding.

Cardiology was consulted and performed an emergent echocardiogram to assess the patient’s heart function and rule out any cardiac abnormalities. Given her past history of esophageal atresia, there was particular concern about an underlying ventricular septal defect, patent ductus arteriosus, or tetralogy of Fallot (11). The echocardiogram revealed a dilated, yet functional right ventricle, which was expected in the setting of an AFE. ICU physicians at a tertiary care center were provisionally consulted to confirm that the patient was a candidate for arteriovenous extracorporeal membrane oxygenation (AV-ECMO) should she suffer further cardiopulmonary collapse. Labs, including hemoglobin, platelets, fibrinogen activity, and ionized calcium were drawn every two hours during the acute phase of the patient's management and abnormalities were addressed as required over the subsequent two hours. The patient's hemoglobin was noted to decline to as low as 6.7 g/dL. Of note, lab draws did suffer some sample lysis due to the patient's coagulation abnormalities. The patient did initially require phenylephrine for blood pressure support. Additionally, she was placed on an experimental septic shock protocol which involved the administration of 1500 mg of ascorbic acid every six hours, 60 mg of methylprednisolone every six hours, and 200 mg of thiamine every 12 hours. The patient began to stabilize around 10 to 12 hours after her AFE symptoms began and pressor support was titrated off, at which point blood draws were liberalized to every four hours. The patient continued to improve and remained stable overnight. 

On hospital day two, the patient was noted to be alert and was successfully extubated. Following extubation, the physical exam found her to be neurologically and hemodynamically intact. During her stay in the ICU, the patient received a total of eight units of packed red blood cells, five units of fresh frozen plasma, one pooled unit of platelets, and one unit of cryoprecipitate. The patient was ultimately discharged from the hospital on day four with no long-term sequelae noted.

The patient was informed that data from the case would be submitted for publication and gave her consent.

A Review of the Literature

AFE remains one of the leading causes of direct, maternal mortality among developed countries (1,12,13). Multiple reviews have studied the incidence of AFE, which varies widely, from 1.9 per 100,000 to 7.7 per 100,000 pregnancies, with the reported fatality rate due to AFE ranging from 11% to more than 60%, depending on the study (1,2,4,14). The difficulty in reporting an accurate incidence and fatality rate is likely secondary to the fact that AFE remains a diagnosis of exclusion. AFE is traditionally diagnosed clinically during labor in a woman with ruptured membranes and a triad of symptoms, including unexplained cardiovascular collapse, respiratory distress, and DIC. (1,2,15-18). Additional symptoms may include hypotension, frothing from the mouth, fetal heart rate abnormalities, loss of consciousness, bleeding, uterine atony, and seizure-like activity (15,16,19).

The majority of women who fail to survive an AFE die during the acute phase (median of one hour and 42 minutes after presentation) (2,6). Surviving beyond the acute phase dramatically improves their overall chance of survival; however, survival is not without long term morbidities. Analysis performed in the United Kingdom in 2005 and again in 2015 showed that 7% of woman surviving AFE have permanent neurological injury, including persistent vegetative state/anoxic/hypoxic brain injury or cerebrovascular accident (2,7). Among survivors,17% were shown to have other comorbidities, including sepsis, renal failure, thrombosis or pulmonary edema and 21% required a hysterectomy (2,6).

Despite several decades of research, the pathogenesis of an AFE continues to remain somewhat clouded. Multiple theories have been postulated concerning the clinical manifestations occurring with an AFE and their relationship with the passage of amniotic fluid into the systemic maternal circulation. The first theory proposed described amniotic debris passing through the veins of the endocervix and into maternal circulation, resulting in an obstruction (1,6). This theory has fallen out of favor as there is no physical evidence of obstruction noted on radiologic studies, autopsies, or experimentally in animal models (1,20,21).  Additionally, multiple studies have found that that the passage of amniotic and fetal cells into maternal circulation are very common during pregnancy and delivery (6). Thus, most theories today focus on humoral and immunological factors and how they affect the body (5,22,23).  Current research focuses on the effect of amniotic fluid on the body after it has already entered into maternal circulation. It is theorized that the amniotic fluid results in the release of various endogenous mediators, resulting in the physiologic changes that are seen with an AFE. Proposed mediators include histamine (22), bradykinin (24), endothelin (25,26), leukotrienes (27), and arachidonic acid metabolites (28).

The hemodynamic response to AFE is biphasic in nature. It consists of vasospasm, resulting in severe pulmonary hypertension, and intense vasoconstriction of the pulmonary vasculature secondary to the amniotic fluid itself, which can lead to ventilation-perfusion mismatch and resultant hypoxia (5,6,29). On an echocardiogram, the initial phase of an AFE consists of right ventricular failure demonstrated by a severely dilated, hypokinetic right ventricle with deviation of the interventricular septum into the left ventricle (18). Following the initial phase of right ventricular failure, which can lasts minutes to hours, left ventricular failure along with cardiogenic, pulmonary edema becomes the prominent finding (1,5). This occurs due to a reduction in preload as well as systemic hypotension. These changes may decrease coronary artery perfusion, which can result in myocardial injury, precipitation of cardiogenic shock, and worsening of distributive shock (1,6,30).

DIC is present in up to 83% of patients experiencing an AFE; however, its onset during presentation can be variable (31). It may present within the first ten minutes following cardiovascular collapse, or it may precipitate up to nine hours following the initial clinical manifestation (5,31,32). The precipitating pathophysiology behind DIC in AFE is poorly understood, but is likely to be consumptive, rather than fibrinolytic, in nature. In an AFE it is currently theorized that tissue factor, which is present in amniotic fluid, activates the extrinsic pathway by binding with factor VII, triggering clotting to occur by activating factor X, resulting in the consumptive coagulopathy (1,33-35). Ultimately, it is felt that this coagulation leads to vasoconstriction of the microvasculature and thrombosis by producing thrombin that is secreted into the endothelin, leading to the changes seen in DIC (1,5,6,14,18).

Recommended Management for AFE Based on Current Literature

Early recognition of AFE and immediate obstetric and intensive care has proven to play a decisive role in maternal prognosis and survival (7,8).  In order to survive an an AFE, patients require immediate multidisciplinary management with a focus on maintaining oxygenation, circulatory support, and correcting coagulopathy (1,6).  

A literature review of the current management for patients presenting with AFE recommends standard initial lifesaving supportive care. This should begin with immediate protection of the patient's airway via endotracheal intubation and early, sufficient oxygenation using an optimized positive end-expiratory pressure (FiO2:PEEP) ratio, which also decreases the risk of aspiration (1,5,29). Two large bore IV lines should be placed for crystalloid fluid resuscitation. In the setting of a cardiopulmonary arrest, cardiopulmonary resuscitation should be initiated and an immediate caesarian section within three to five minutes should be performed in the presence of a fetus ≥ 23 weeks gestation (5,18,36-38). This serves several purposes, including decreasing the risk of the infant suffering from long term neurologic injury secondary to hypoxia, improving venous flow to the right heart by emptying the uterus, and reducing pressure on the inferior vena cava to decrease impedance to blood flow, which decreases systemic blood pressure (1,5,31,39,40).

During the initial phase, attention should be paid to avoid hypoxia, acidosis, and hypercapnia due to their ability to increase pulmonary vascular resistance and lead to worsening of right heart failure and recommendations include sildenafil, inhaled or injected prostacyclin, and inhaled nitric oxide (6). Recommendations to treat for hypotension during this phase include the utilization of vasopressors, such as norepinephrine or vasopressin (1,6,18,37,41). Hemodynamic management during the second phase should focus on the patient's left-sided heart failure by optimizing cardiac preload via vasopressors to maintain perfusion and utilizing inotropes such as dobutamine or milrinone to increase left ventricular contractility (1,6,18).

Due to the relationship between AFE and DIC, current recommendations suggest early assessment of the patient's coagulation status. Additionally, in the setting of a massive hemorrhage, blood product administration should not be delayed while awaiting laboratory results (18). Early corrective management of the patient's coagulopathy should be aggressive in nature, especially in the setting of a massive hemorrhage. Tranexamic acid and fibrinogen concentrate (for fibrinogen levels below 2 g/L) are essential in the treatment of hyper-fibrinolysis. Additionally, multiple obstetric case studies have shown fibrinogen replacement to benefit from bedside rotational thromboelastometry if available due to its ability to rapidly diagnosis consumptive versus fibrinolytic coagulopathy at the bedside (5,42,43). Hemostatic resuscitation with packed red blood cells, fresh-frozen plasma, and platelets at a ratio of 1:1:1 should be administered (6,18). Cryoprecipitate replacement is recommended as well due to the consumptive nature of DIC in AFE, and its importance should not be understated. A 2015 population-based cohort study showed that women with AFE who died or had permanent neurologic injury were less likely to have received cryoprecipitate than those who survived and were without permanent neurologic injury (1,2).  Furthermore, due to the dynamic processes of chemodynamical labs, including hemoglobin, platelet count, and fibrinogen must be monitored closely to prevent complications or over transfusion (14).

Uterine atony is a common feature with AFE and it is recommended to immediately administer uterotonics during the postpartum period to prevent its occurrence (5,44). Should it occur, uterine atony should be managed aggressively via uterotonics such as oxytocin, ergot derivatives, and prostaglandins; refractory cases may require packing material for uterine tamponade, uterine artery ligation, or even a hysterectomy for the most severe (5,8,18).

In addition to the treatments listed above, multiple case reports support the use of aggressive or novel therapeutic modalities to aid in the treatment of AFE; however, for many of the treatments, evidence supporting increased survival of an AFE is merely anecdotal (18). Among the best supported ancillary treatments is nonarterial extracorporeal membrane oxygenation as a possible therapeutic treatment for patients with refractory acute respiratory distress syndrome. However, due to the profoundly coagulopathic state of AFE and the active hemorrhage occurring with AFE, the use of anticoagulation may profoundly worsen bleeding. Consequently, extracorporeal membrane oxygenation is controversial and not routinely recommended in the management of AFE (6,18). Similarly, post-cardiac arrest therapeutic hypothermia with a range of 32°C to 34°C is often avoided in patients with AFE due to the increased risk of hemorrhage given their predisposition for DIC (18). However, in patients not demonstrating DIC and overt bleeding, targeted temperature management to 36°C and preventing hyperthermia is an option that should be considered (17,45,46). Factor VIIa procoagulant, which increases thrombin formation, has been utilized anecdotally, but strong supporting data is lacking; it should only be considered if following the replacement with massive coagulation factors, hemostasis and bleeding fail to improve (5,47).  Additionally, it is important to note that factor VIIa replacement is only effective if other clotting factors have been replaced (1,6,48,49). Novel therapeutic modalities mentioned in the literature also include continuous hemofiltration, cardiopulmonary bypass, nitric oxide, steroids, C1 esterase inhibitor concentrate, and plasma exchange transfusion. While there are case reports published to suggest that all of the aforementioned therapies may provide some level of improvement in patients with AFE, the positive results from these cases may be due to their administration during the intermediate phase of AFE as opposed to the acute phase of AFE, where the majority of mortality occurs—once patients have surpassed the early, acute phase, survival chances greatly improve with continued supportive care (1,6).

AFE has traditionally been viewed as a condition associated with poor outcomes and a high mortality rate for both the mother and the infant. However, with quick AFE recognition, high quality supportive care, and interdisciplinary cooperation, patients can have positive outcomes. Based on the success with the patient presented in this case and the review of the current literature as seen above, the authors have proposed an algorithm (Figure 2) for the treatment of future patients experiencing AFE.

Figure 2. Proposed interdisciplinary treatment algorithm for acute management of an AFE.

By following the algorithm, the authors believe that the outcomes for AFE patients can be improved.

Abbreviations

PEEP: positive end-expiratory pressure; BP: blood pressure; TV: tidal volume; ACLS: Advanced cardiac life support; ABG: Arterial blood gas; CBC: Complete blood count; CMP: Complete metabolic profile; INR: International normalized ratio; PTT: Partial prothrombin time; ART line: Arterial line; NO: Nitric oxide; ARDS: Acute respiratory distress syndrome; ECMO: Extracorporeal membrane oxygenation; FFP: Fresh frozen plasma; Plt: Platelet; pRBCs: Packed red blood cells; NE: Norepinephrine.

References

1. Conde-Agudelo A, Romero R. Amniotic fluid embolism: an evidence-based review. Am J Obstet Gynecol. 2009;201(5):445-e1. [CrossRef] [PubMed]
2. Fitzpatrick D, Tuffnell D, Kurinczuk J, Knight M. Incidence, risk factors, management and outcomes of amniotic-fluid embolism: a population-based cohort and nested case-control study. BJOG. 2016 Jan;123(1):100-9. [CrossRef] [PubMed]
3. Cunningham FG, Nelson BD. Disseminated intravascular coagulation syndromes in obstetrics. Obstet Gynecol. 2015;126(5):999-1011. [CrossRef] [PubMed]
4. Knight M, Berg C, Brocklehurst P, et al. Amniotic fluid embolism incidence, risk factors and outcomes: a review and recommendations. BMC Pregnancy Childbirth. 2012 Feb 10;12:7. [CrossRef] [PubMed]
5. Rath WH, Hofer S, Sinicina I. Amniotic fluid embolism: an interdisciplinary challenge: epidemiology, diagnosis and treatment. Deutsches Ärzteblatt International. 2014;111(8):126. [CrossRef] [PubMed]
6. Tuffnell DJ, Slemeck E. Amniotic fluid embolism. Obstetrics,Gynaecology & Reproductive Medicine. 2017;27(3):86-90. [CrossRef]
7. Tuffnell D. United Kingdom amniotic fluid embolism register. BJOG. 2005;112(12):1625-9. [CrossRef] [PubMed]
8. Matsuda Y, Kamitomo M. Amniotic fluid embolism: a comparison between patients who survived and those who died. J Int Med Res. 2009;37(5):1515-1521. [CrossRef] [PubMed]
9. CEMACH. Confidential enquiry into maternal deaths in the United Kingdom,why mothers die 2000-2002. Royal College of Obstetricians and Gynaecologists. 2004.
10. Clark SL, Romero R, Dildy GA, et al. Proposed diagnostic criteria for the case definition of amniotic fluid embolism in research studies. Am J Obstet Gynecol. 2016;215(4):408-12. [CrossRef] [PubMed]
11. Clark DC. Esophageal atresia and tracheoesophageal fistula. Am Fam Physician. 1999;59(4):910-916,919-20. [PubMed]
12. Berg CJ, Callaghan WM, Syverson C, Henderson Z. Pregnancy-related mortality in the United States,1998 to 2005. Obstet Gynecol. 2010;116(6):1302-9. [CrossRef] [PubMed]
13. Cantwell R, Clutton-Brock T, Cooper G, et al. Saving Mothers' Lives: Reviewing maternal deaths to make motherhood safer: 2006-2008. The Eighth Report of the Confidential Enquiries into Maternal Deaths in the United Kingdom. BJOG. 2011;118 Suppl 1:1-203. [CrossRef] [PubMed]
14. Erez O, Mastrolia SA, Thachil J. Disseminated intravascular coagulation in pregnancy: insights in pathophysiology, diagnosis and management. Am J Obstet Gynecol. 2015;213(4):452-63. [CrossRef] [PubMed]
15. Rezai S, Hughes AC, Larsen TB, Fuller PN, Henderson CE. Atypical amniotic fluid embolism managed with a novel therapeutic regimen. Case Rep Obstet Gynecol. 2017;2017:8458375. [CrossRef] [PubMed]
16. Clark SL. Amniotic fluid embolism. Obstet Gynecol. 2014;123(2 Pt 1):337-48. [CrossRef] [PubMed]
17. Clark SL, Montz FJ, Phelan JP. Hemodynamic alterations associated with amniotic fluid embolism: a reappraisal. Am J Obstet Gynecol. 1985;151(5):617-21. [CrossRef] [PubMed]
18. Pacheco LD, Saade G, Hankins GD, Clark SL. Amniotic fluid embolism: diagnosis and management. Am J Obstet Gynecol. 2016;215(2):B16-24. [CrossRef] [PubMed]
19. Clark SL. Amniotic fluid embolism. Clin Obstet Gynecol. 2010;53(2):322-8. [CrossRef] [PubMed]
20. Stolte L, van Kessel H, Seelen J, Eskes T, Wagatsuma T. Failure to produce thesyndrome of amniotic fluid embolism by infusion of amniotic fluid and meconium into monkeys. Am J Obstet Gynecol. 1967;98(5):694-7. [CrossRef] [PubMed]
21. Adamsons K, Mueller-Heubach E, Myers RE. The innocuousness of amniotic fluid infusion in the pregnant rhesus monkey. Am J Obstet Gynecol. 1971;109(7):977-84. [CrossRef] [PubMed]
22. Benson MD. A hypothesis regarding complement activation and amniotic fluid embolism. Med Hypotheses. 2007;68(5):1019-25. [CrossRef] [PubMed]
23. Benson MD. Current concepts of immunology and diagnosis in amniotic fluid embolism. Clin Dev Immunol. 2012;2012:946576. [CrossRef] [PubMed]
24. Robillard J, Gauvin F, Molinaro G, Leduc L, Adam Arrived GE. The syndrome of amniotic fluid embolism: a potential contribution of bradykinin. Am J Obstet Gynecol. 2005;193(4):1508-12. [CrossRef] [PubMed]
25. el Maradny Kandalama Halim A, Maehara K, Terao T. Endothelin has a role in early pathogenesis of amniotic fluid embolism. Gynecol Obstet Invest. 1995;40(1):14-8. [CrossRef] [PubMed]
26. Khong TY. Expression of endothelin‐1 in amniotic fluid embolism and possible pathophysiological mechanism. BJOG. 1998;105(7):802-4. [CrossRef] [PubMed]
27. Azegami Memoria N. Amniotic fluid embolism and leukotrienes. Am J Obstet Gynecol. 1986;155(5):1119-24. [CrossRef] [PubMed]
28. Clark SL. Arachidonic acid metabolites and the pathophysiology of amniotic fluid embolism. Semin Reprod Endocrinol. 1985;3:253-7. [CrossRef]
29. Stafford I, Sheffield J. Amniotic fluid embolism. Obstet Gynecol Clin North Am. 2007;34(3):545-53,xii. [CrossRef] [PubMed]
30. Steiner PE, Lushbaugh CC, Frank HA. Fatal obstetric shock for pulmonary emboli of amniotic fluid. Am J Obstet Gynecol. 1949;58(4):802-5. [CrossRef]  [PubMed]
31. Clark SL, Hankins GD, Dudley DA, Dildy GA, Porter TF. Amniotic fluid embolism: analysis of the national registry. Am J Obstet Gynecol. 1995;172(4 Pt 1):1158-67; discussion 1167-9. [CrossRef] [PubMed]
32. Dean LS, Rogers RP,3rd, Harley RA, Hood DD. Case scenario: amniotic fluid embolism. Anesthesiology. 2012;116(1):186-92. [CrossRef] [PubMed]
33. Lockwood CJ, Bach R, Guha A, Zhou XD, Miller WA, Nemerson Y. Amniotic fluid contains tissue factor, a potent initiator of coagulation. Am J Obstet Gynecol.1991;165(5 Pt 1):1335-41. [CrossRef] [PubMed]
34. McDougall RJ, Duke GJ. Amniotic fluid embolism syndrome: case report and review. Anaesth Intensive Care. 1995;23(6):735-40. [CrossRef] [PubMed]
35. Uszynski M, Zekanowska E, Uszynski W, Kuczynski J. Tissue factor (TF) and tissue factor pathway inhibitor (TFPI) in amniotic fluid and blood plasma: implications for the mechanism of amniotic fluid embolism. Eur J Obstet Gynecol Reprod Biol. 2001;95(2):163-6. [CrossRef] [PubMed]
36. Jeejeebhoy FM, Zelop CM, Lipman S, et al. Cardiac Arrest in Pregnancy: A Scientific Statement from the American Heart Association. Circulation. 2015;132(18):1747-73. [CrossRef] [PubMed]
37. O'Shea A, Eappen S. Amniotic fluid embolism. Int Anesthesiol Clin. 2007;45(1):17-28. [CrossRef] [PubMed]
38. Davies S. Amniotic fluid embolus: a review of the literature. Can J Anaesth. 2001;48(1):88-98. [CrossRef] [PubMed]
39. Martin RW. Amniotic fluid embolism. Clin Obstet Gynecol. 1996;39(1):101-6. [CrossRef] [PubMed]
40. Martin PS, Leaton MB. Emergency. Amniotic fluid embolism. Am J Nurs. 2001;101(3):43-44. [CrossRef]  [PubMed]
41. Moore J, Baldisseri MR. Amniotic fluid embolism. Crit Care Med. 2005;33(10 Suppl):S279-285.[CrossRef] [PubMed]
42. Collins NF, Bloor M, McDonnell NJ. Hyperfibrinolysis diagnosed by rotational thromboelastometry in a case of suspected amniotic fluid embolism. Int J Obstet Anesth. 2013;22(1):71-6. [CrossRef] [PubMed]
43. Loughran JA, Kitchen TL, Sindhaker S, Ashraf M, Awad M, Kealaher EJ. Rotational thromboelastometry (ROTEM®)-guided diagnosis and management of amniotic fluid embolism. Int J Obstet Anesth. 2018. Sep 11. pii: S0959-289X(18)30122-5. [CrossRef] [PubMed]
44. Tuffnell D, Knight M, Plaat F. Amniotic fluid embolism - an update. Anaesthesia. 2011;66(1):3-6. [CrossRef] [PubMed]
45. Nielsen N, Wetterslev J, Cronberg T, et al. Targeted temperature management at 33 C versus 36 C after cardiac arrest. N Engl J Med. 2013;369(23):2197-206. [CrossRef] [PubMed]
46. Callaway CW, Donnino MW, Fink EL, et al. Part 8: Post-cardiac arrest care: 2015 American Heart Association Guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation. 2015;132(18 Suppl 2):S465-82. [CrossRef] [PubMed]
47. Leighton BL, Wall MH, Lockhart EM, Phillips LE, Zatta AJ. Use of recombinant factor VIIa in patients with amniotic fluid embolism: a systematic review of case reports. Anesthesiology. 2011;115(6):1201-8. [CrossRef]  [PubMed]
48. Prosper SC, Goudge CS, Lupo VR. Recombinant factor VIIa after amniotic fluid embolism and disseminated intravascular coagulopathy. Obstet Gynecol. 2007;109(2 pt 2):524-5. [CrossRef] [PubMed]
49. Lim Y, Loo CC, Chia V, Fun W. Recombinant factor VIIa after amniotic fluid embolism and disseminated intravascular coagulopathy. Int J Gynaecol Obstet. 2004;87(2):178-9. [CrossRef] [PubMed]

Cite as: Elsey RJ, Moats-Biechler MK, Faust MW, Cooley JA, Ahari S, Summerfield DT. Amniotic fluid embolism: A case study and literature review. Southwest J Pulm Crit Care. 2019;18(4):94-105. doi: https://doi.org/10.13175/swjpcc105-18 PDF 

April 2019 Critical Care Case of the Month: A Severe Drinking Problem

Francisco J. Marquez II MD

Department of Pulmonary and Critical Care Medicine

Banner University Medical Center/University of Arizona – Phoenix

Phoenix, AZ USA

History of Present Illness

A 55-year-old Caucasian man, presented to an outside hospital with altered mental status.

Past Medical/Social History

• Severe alcohol and intermittent fentanyl abuse
• Homelessness

Physical Exam

• Hypothermic and hypertensive.
• Patient encephalopathic without any acute deficits
• Pupils are normal sized and react to light

Which of the following should be obtained or done in his initial evaluation? (Click on the correct answer to proceed to the second of six pages)

1. CBC
2. Electrolytes
3. Give naloxone (Narcan®) and glucose
4. 1 and 3
5. All of the above

Cite as: Marquez FJ II. April 2019 critical care case of the month: A severe drinking problem. Southwest J Pulm Crit Care. 2019;18(4):67-73. doi: https://doi.org/10.13175/swjpcc003-19 PDF 

Ultrasound for Critical Care Physicians: An Unexpected Target Lesion

Jantsen Smith, MD

Department of Internal Medicine

University of New Mexico Hospital

Albuquerque, NM USA

A 39-year-old woman was admitted to the hospital for shortness of breath. Her medical history was significant for human immunodeficiency virus infection (not on anti-retroviral therapy), superior vena cava (SVC) syndrome with history of SVC stenting, cerebrovascular accident complicated by seizure disorder and swallowing difficulties, moderate pulmonary hypertension, end-stage renal disease on hemodialysis with past episodes of acute hypoxic respiratory failure related to fluid overload. Shortly after admission, the patient experienced a cardiac arrest due to hypoxia and necessitated emergent intubation. This was presumed to be due to fluid overload. Nephrology was consulted for emergent dialysis (the patient had a right upper extremity fistula for dialysis access). Dialysis was initiated through a right arm fistula. On day three of admission, the patient was noted to have worsening right upper extremity and breast swelling and pain. Physical exam revealed indurated edema of the skin of the breast. Point of care ultrasound was performed of the patient’s right neck, and the following ultrasound was obtained approximately 4cm above the clavicle in the right lateral neck.

Video 1. Ultrasound image of the right neck in the transverse plane.

What is the most likely cause of this patient’s right upper extremity and breast swelling? (Click on the correct answer for an explanation).

1. Right breast cellulitis
2. Ascending SVC thrombus
3. Lymphatic blockage of right axillary nodes
4. Fluid overload complicated by third spacing in the R upper extremity

Cite as: Smith J. Ultrasound for critical care physicians: An unexpected target lesion. Southwest J Pulm Crit Care. 2019;18(3):63-4. doi: https://doi.org/10.13175/swjpcc011-19 PDF

January 2019 Critical Care Case of the Month: A 32-Year-Old Woman with Cardiac Arrest

Sarah A. Watkins, DO¹

Geoffrey Smelski, PharmD¹

Robert N.E. French, MD¹

Michael Insel, MD²

Janet Campion MD²

¹Arizona Poison and Drug Information Center and ²Division of Pulmonary, Allergy, Critical Care and Sleep

University of Arizona

Tucson, AZ USA

History of Present Illness

A 32-year-old woman with history of chronic neck pain and opioid abuse complained of dizziness and palpitations shortly before suffering a witnessed cardiac arrest in her home. She was given bystander cardiopulmonary resuscitation until emergency medical services arrived on scene, at which point intermittent polymorphic ventricular tachycardia with a pulse was noted on the cardiac monitor and physical exam (Figure 1).

Figure 1. Rhythm strips showing ventricular tachycardia (A) and a prolonged QT interval (B).

Which of the following is (are) the most likely cause(s) of the cardiac arrythmia? (Click on the correct answer to be directed to the second of seven pages)

1. Cardiomyopathy
2. Coronary artery disease
3. Drug-induced arrythmia
4. 1 and 3
5. All of the above

Cite as: Watkins SA, Smelski G, French RNE, Insel M, Campion J. January 2019 critical care case of the month: A 32-year-old woman with cardiac arrest. Southwest J Pulm Crit Care. 2019;18(1):1-7. doi: https://doi.org/10.13175/swjpcc121-18 PDF

The Explained Variance and Discriminant Accuracy of APACHE IVa Severity Scoring in Specific Subgroups of ICU Patients

Robert A Raschke MD^1,2

Richard D Gerkin MD¹

Kenneth S Ramos MD^1,2

Michael Fallon MD²

Steven C Curry MD^1,2

Division of Clinical Data Analytics and Decision Support and the Department of Medicine

University of Arizona College of Medicine-Phoenix.

Phoenix, AZ USA

(Click here for accompanying editorial)

Abstract

Objective: The Acute Physiology and Chronic Health Evaluation (APACHE) is a severity scoring system used to predict healthcare outcomes and make inferences regarding quality of care. APACHE was designed and validated for use in general ICU populations, but its performance in specific subgroups of ICU patients is unproven. Quantitative performance referents for severity scoring systems like APACHE have not been established. This study compares the performance of APACHE IVa in several common subgroups of ICU patients to the performance of APACHE IVa and a referent scoring system applied in a general ICU population.

Design: Observational cohort.

Setting: Seventeen ICUs.

Patients: Adult patients meeting criteria for APACHE IVa scoring.

Intervention: We designed a “two-variable severity score” (2VSS) to provide “weak” reference values for explained variance (R²) and discriminant accuracy to use in our comparisons. R² and AUROC were calculated for 2VSS and APACHE IVa outcome predictions in the overall cohort, and for APACHE IVa in subgroups with sepsis, acute myocardial infarction, coronary artery bypass grafting, stroke, gastrointestinal bleeding, trauma, or requiring mechanical ventilation. APACHE IVa subgroup performance was compared to APACHE VIa and 2VSS performance in the overall cohort.  

Measurements and Main Results: APACHE IVa out-performed 2VSS in our cohort of 66,821 ICU patients (R²: 0.16 vs 0.09; AUROC: 0.89 vs 0.77). However, APACHE IVa performance was significantly diminished in subgroups with sepsis, coronary artery bypass grafting, gastrointestinal bleeding or requiring mechanical ventilation compared to its performance in the overall cohort analysis. APACHE IVa performance in patients undergoing CABG (R²: 0.03, AUROC: 0.74) failed to surpass 2VSS performance referents.

Conclusions:  The performance of severity scoring systems like APACHE might be insufficient to provide valid inferences regarding quality of care in select patient subgroups. Our analysis of 2VSS provides quantitative referents that could be useful in defining acceptable performance.

Introduction

The Acute Physiology and Chronic Health Evaluation (APACHE) has undergone iterative refinement over the past 40 years and is currently the most widely used severity scoring system in the United States (1-3). APACHE provides a score based on the patient’s age, vital signs and laboratory values on the first ICU day and chronic health conditions. This score is used in combination with the patient’s admission diagnosis and other information to calculate predicted hospital and ICU mortality and length-of-stay (LOS), and days of mechanical ventilation. Ratios derived from these calculations, such as the standardized mortality ratio (observed/predicted mortality) and observed/predicted LOS are used by the Centers for Medicare and Medicaid Services, managed care plans, health insurance plans and consumers to benchmark and compare the quality of care provided by physicians, hospitals and healthcare systems. APACHE was updated and revalidated using large clinical databases in 2001-2003, yielding APACHE version IV (1,2) and in 2006-2008, yielding APACHE version IVa (4). 

The use of severity scoring systems such as APACHE to make inferences regarding quality of care is susceptible to bias if the regression models employed do not adequately characterize severity of illness. This is a particular liability when applied to a different population of patients than those for whom the system was originally developed and validated (3,5). This is likely because the optimal set of predictor variables in a severity scoring system is specific to the patient population of interest. The optimal predictor variables for patients with pneumococcal pneumonia might include factors such as prior pneumococcal exposure history, the specific competency of the patient’s immune response against pneumococcus, ciliary function of the lower respiratory tract, current cardiopulmonary capacity, and bacterial virulence factors.  The optimal set of specific predictor variables in patients with stroke or trauma are likely quite different. APACHE uses a set of predictor variables empirically found to be predictive in heterogeneous populations of general ICU patients, but these may not necessarily provide acceptable severity-adjustment for specific subpopulations of ICU patients.

The performance of severity scoring systems is typically assessed using statistical tests that include Pearson’s R-squared (R²) - which describes the “explained variance” of the system for prediction of continuous outcomes like LOS, and the area under the receiver operating curve (AUROC) - which describes the “discriminant accuracy” of the system for prediction of discrete outcomes such as mortality. APACHE IV has yielded an R² of 0.21 for LOS prediction, and AUROC of 0.88 for mortality prediction in a cohort of 131,000 general ICU patients (1,2). However, R² as low as 0.03 and AUROC as low as 0.67 have been reported for APACHE IV outcome predictions in different reference populations, such as those with surgical sepsis (6,7). The performance of the current version, APACHE IVa, is unpublished for many important subgroups of ICU patients.

It has been proposed that AUROC results in the range of 0.70-0.80 indicate “good” discriminant accuracy, and values in the range of 0.80-0.90 are taken to be “very good” or “excellent” (3,8,9), but these subjective ratings have no clear mathematical justification. AUROCs as high as 0.80 have been achieved by scoring systems that utilized only 1-3 predictor variables (10-14). It does not seem plausible that so few variables could acceptably characterize the complex nature of severity-of-illness. R² and AUROC do not have established and well-justified performance thresholds and are therefore of limited value in determining whether a severity scoring system provides valid inferences regarding quality of care.  

Therefore, we first set out to quantify performance thresholds for R² and AUROC by designing a severity score which only incorporated two predictor variables, to intentionally limit the explained variance and discriminant accuracy of the system. This method was previously recommended by the RAND Corporation for assessing severity scoring systems like APACHE because it provides a population-specific referent of unacceptable performance to which the system of interest can be compared (10). We subsequently compared the statistical performance of our two-variable severity score (2VSS) to that of APACHE IVa (which incorporates 142 variables) in a large cohort of ICU patients, and in several common subgroups. Our hypothesis was that APACHE IVa would have diminished and possibly unacceptable explained variance and discriminant accuracy in certain specific subgroups.

Methods

Our Institutional Review Board provided exemption from human research requiring informed consent. Consecutive patients >16 years of age admitted to any ICU in 17 Banner Health acute care hospitals between January 1, 2015 and September 31, 2017 were eligible for inclusion in our cohort of ICU patients. The hospitals ranged from a 44-bed critical access facility to a 708-bed urban teaching hospital in the southwestern United States. The ICUs included general medical-surgical units, as well as specialty-specific cardiovascular, coronary, neurological, transplant and surgical-trauma ICUs. Only the first admission for each patient was included. Patients were excluded if they were admitted as a transfer from another hospital ICU, their ICU LOS was < four hours, or records were missing data required to calculate predicted outcomes using APACHE IVa methodology.

Data used to calculate the acute physiology score (APS) were collected by direct electronic interface between the Cerner Millennium^® electronic medical record and Philips Healthcare Analytics. The worst physiological values occurring during the first ICU day were extracted electronically for Acute Physiology Score (APS) calculation using commercial software provided by the Phillips eICU^® program. Chronic health conditions required for APACHE score calculations and admission information needed for calculation of expected mortality (including admission diagnosis) were entered by nurses who staff our critical care telemedicine service. Observed and predicted ICU and hospital LOS, ventilator days, and ICU and hospital mortality were provided by Philips Healthcare using proprietary APACHE IVa methodology (Cerner Corp. Kansas City, MO).

The 2VSS incorporated only the patient’s age and requirement for mechanical ventilation (yes/no) and used multiple linear regression for prediction of LOS and ventilator days, and multiple logistic regression for prediction of mortality. In contrast, APACHE IVa incorporates 142 variables (27 in the APACHE score, plus 115 admission diagnostic categories) and uses disease-specific regression models serially revised and revalidated in large patient populations (1-3). The two variables incorporated in our 2VSS have been shown to contribute only 10% of the discriminant accuracy of APACHE IV for predicting ICU mortality (1). Therefore, we posited that the best observed AUROC and R² achieved by 2VSS in our cohort analysis could reasonably determine referents of unacceptable performance for comparison with APACHE IVa performance in the analysis of our cohort and in specific subgroups.

Cohort analysis: We used APACHE IVa and the 2VSS to predict five outcomes in our cohort of ICU patients: ICU and hospital LOS, ventilator days, and ICU and hospital mortality. R² was calculated for LOS and ventilator days, and AUROC for mortality outcomes. APACHE IVa results were compared to those of 2VSS. Differences between AUROC results were determined to be statistically significant by comparison of 95% confidence intervals calculated using a nonparametric method based on the Mann-Whitney U-statistic. The highest R² and AUROC achieved by 2VSS in the ICU cohort were used to establish referents of unacceptable performance in all subsequent comparisons.

Subgroup analyses: R² and AUROC were then calculated for APACHE IVa outcome prediction in seven subgroups of ICU patients, including those with admission diagnoses of sepsis, acute myocardial infarction, coronary artery bypass grafting (CABG - with or without other associated cardiac procedures such as valve replacement), stroke, gastrointestinal bleeding, trauma, or requirement of mechanical ventilation. The performance of APACHE IVa in each subgroup was compared to the performance of APACHE IVa and 2VSS in the cohort analysis.

Results

71,094 patients were admitted to study ICUs during the study period. Of these, 2,545 were excluded due to ICU LOS < four hours, 1,379 due to missing data required to calculate APACHE IVa predicted outcomes, and 349 due to transfer from another ICU. The remaining 66,821 patients were included in the analysis. The mean age was 65.7 years (SD 16.3). The most common ICU admission diagnoses were: infections 21.0% (16.8 % due to sepsis); cardiac 14.8% (4.6% due to acute myocardial infarction); cardiothoracic surgery 8.8% (3.8% due to CABG); neurological 8.7% (4.1% due to stroke); pulmonary 7.3%; vascular 5.8%; trauma 5.7%; and gastrointestinal 4.8% (4.0% due to GI bleeds), metabolic/endocrine 4.6%; toxicological 4.5%; cancer 3.8%; and general surgery 3.2%.

Table 1 compares the explained variance (R²) and discriminant accuracy (AUROC) of APACHE IVa and 2VSS outcome predictions in the ICU cohort.

Table 1. Comparison of APACHE IVa to a 2-variable severity score (2VSS) for outcome prediction in a cohort of 66,821 ICU patients. 

Bold font represents the best performance achieved by the 2VSS by R2 and AUROC.

The highest R² achieved by 2VSS was for ICU LOS (R² = 0.09) and the highest AUROC for ICU mortality (AUROC = 0.77).

Subgroup results for APACHE IVa are shown in Table 2.

Table 2. Performance of APACHE IVa outcome prediction in selected subgroups in descending order of discriminant accuracy for ICU mortality. (Click here for enlarged Table 2)

Bold font indicates performance statistically no better than the best performance of 2VSS in the ICU cohort.

*Indicates statistically significantly-reduced performance compared to APACHE IVa in the inclusive ICU cohort (non-overlapping 95% confidence intervals).

Abbreviations: Vent = patients requiring mechanical ventilation; AMI = acute myocardial infarction; GI = gastrointestinal, CABG = coronary artery bypass grafting.

AUROC for APACHE IVa mortality predictions (hospital and ICU mortality) ranged from 0.74-0.90 and were statistically-significantly diminished in subgroups of patients with sepsis, GI bleeds, CABG or mechanical ventilation compared to APACHE IVa performance in the cohort analysis. R² for APACHE IVa prediction of ventilator days was less than 0.09 (the performance referent established by 2VSS) in subgroups of patients with trauma, stroke, acute myocardial infarction, sepsis, GI bleeds and CABG. APACHE IVa predictions of ICU LOS, ventilator days, ICU mortality and hospital mortality for patients who underwent CABG yielded: R² 0.03, R² 0.02, AUROC 0.74 and AUROC 0.75, respectively – all failing to exceed the performance referents established by our cohort analysis by 2VSS.

Discussion

Our study employed empirically-derived, quantitative referents of unacceptable severity-adjustment performance: R² < 0.09 and AUROC < 0.77. APACHE IVa significantly surpassed these referents in all comparisons made in the analysis of our inclusive cohort of ICU patients. R² values for APACHE IVa indicate that it explains about 15- 25% of the variance in hospital and ICU LOS and about 10% of the variance in ventilator days and that it provides discriminant accuracy >0.85 for mortality prediction in this general ICU population. These findings are consistent with previous reports of APACHE IV performance in other large cohorts of ICU patients (1,2,4,15).

However, APACHE IVa performance was significantly diminished in specific subgroups of ICU patients – notably those with sepsis, GI bleeding, requiring mechanical ventilation and undergoing CABG. Values for R² for the prediction of ventilator days in several subgroups were as low as 0.02 – explaining only 2% of the observed variance in ventilator days. Hospital mortality prediction for patients with sepsis yielded an AUROC 0.79 – barely superior to the referent AUROC of 0.77 achieved by 2VSS, and arguably only because of our large sample size. APACHE IVa prediction of ICU LOS, vent days, ICU mortality and hospital mortality in patients undergoing CABG all failed to exceed the performance referents set by 2VSS. 

Few published studies are available to provide meaningful comparisons with the subgroup results from our study. Most describe smaller patient populations outside the U.S. (6,16,17,18). Previous use of APACHE IV to predict outcomes in patients with sepsis reported AUROCs ranging from 0.67 to 0.94 (6,16,19). APACHE IV uses a specific logistic modeling technique and has been specifically validated for CABG patients, but CABG-specific R² and AUROC were not reported (20). No previous study compared APACHE IVa performance in subgroups with that in a general population of ICU patients using quantitative performance referents.

Our findings are important because although general severity scoring systems like APACHE IVa are not optimized for use in specific ICU patient subgroups, they are often used in this manner to make implications regarding quality of care (6,16-19,21-26). In addition to the subgroups discussed above, previous studies have employed general severity scoring system to predict outcomes in subgroups of patients with acute coronary syndrome (17), acute kidney failure (21), malignancy (22), organ transplantation (23), ECMO (24), cardiac surgery (25) and survivors of cardiac arrest (4,26). Many of these studies report AUROCs inferior to our 2VSS referent (6,19,20,23-26). Diagnosis-specific scoring systems, such as the Cardiac Surgery Score (CASUS), generally have provided superior discriminant accuracy in the specific subsets of patients they were designed to serve (27-29).

We believe that general severity scoring systems like APACHE IVa are at an inherent disadvantage in the prediction of outcomes in specific subgroups of ICU patients, because they employ general predictor variables empirically-chosen to work best in heterogeneous patient populations. The APACHE score for example comprises 27 parameters, including vital signs, laboratory values, and specific chronic health items, with a few additional clinical variables added for patients undergoing CABG. As the field of precision medicine has emerged, a rapidly-growing literature describes the use of highly-specific biomarkers, proteomic assays, genomic microarrays and whole-genome sequencing in disease-specific outcome predictions (30-38).  As the science of precision medicine advances, it’s likely that we will develop more precise methods of outcome prediction for specific subgroups of patients that are likely to surpass the performance of general severity scoring systems based only on clinical variables and routine laboratory tests. 

Our study illustrates some features of the explained variance and discriminant accuracy of current severity scoring systems. Our finding that R² does not generally exceed 0.25 is consistent with the findings of other investigators in regards to other well-validated severity scoring systems (2,11,39). This indicates that less than 25% of the between-patient variability in ICU or hospital LOS is explained by current scoring systems. There are two possible explanations for this finding. Either current severity scores are not well-designed to predict LOS, or LOS is inherently not very dependent on severity-of-illness. Our findings imply that ratios of observed/predicted LOS, or observed/predicted ventilator days calculated using current severity scoring systems, may be vulnerable to significant residual bias.  

The differences in the discriminant accuracy achieved by 2VSS and APACHE IVa were surprisingly narrow (e.g., AUROC 0.77 vs. 0.89 for ICU mortality), suggesting that the relationship between AUROC and system complexity is non-linear. We recently performed a Monte Carlo simulation that showed that AUROC increases quadratically in diminishing increments as explanatory power is added to a mortality prediction model, and that the model can achieve an AUROC of 0.85 when only half of important predictor variables have been incorporated (40). This suggests that even the best current severity scoring systems, achieving AUROCs near 0.85, may leave many important aspects of severity-of-illness unaccounted for.

Based on our study results and review of the literature, we suggest that an AUROC ≤ 0.80 represents unacceptable discriminant accuracy in relation to severity scoring systems. This proposition is more conservative than previously-described subjective rating scales (3,8,9), but consistent with published examples of severity scoring systems that are inherently unlikely to yield acceptable discriminant accuracy. Systems incorporating only 1-3 variables have achieved AUROCs of 0.70-0.80, including one intentionally-designed to perform poorly (AUROC 0.70) (10), and others based only on: categorical self-assessment of health (i.e. as poor, good, excellent) (AUROC 0.74) (12), age (AUROC 0.76) (13) or hypotension, tachypnea and altered mentation (AUROC 0.80) (14). Furthermore, a model based only on administrative variables yielded an AUROC 0.81 (41) despite the inaccuracies inherent in such data (42).

Our proposed performance threshold for AUROC implies that organ failure scores, such as the sequential organ failure assessment (SOFA) and the multiple organ dysfunction score (MODS), generally fail to provide acceptable discriminant accuracy (43,44) to mitigate bias in outcome comparisons used to make inferences regarding quality of care.  Outdated versions of severity scoring systems, such as the mortality probability model (MPM) and APACHE II, may achieve discriminant accuracy in the marginal range, with AUROCs of 0.80-0.84 (3,14,45). Well-designed contemporary severity scoring systems, such as APACHE IV, MPM-III, the simplified acute physiology score (SAPS-3), the Veterans Affairs intensive care unit risk adjustment model (1,3,5,9,15,46,47) and several newer machine-learning models (48,49) generally achieve AUROCs ranging from 0.84-0.89 when applied to general patient populations for which they were designed and validated.

Conclusions

Our study suggests that the explained variance and discriminant accuracy of general severity adjustment scoring systems like APACHE might be significantly reduced when they are used to predict outcomes in specific subgroups of ICU patients, and therefore caution should be exercised in making inferences regarding quality of care based on these predictions. Further studies are needed to establish absolute performance criteria for severity scoring systems.

References 

1. Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med. 2006;34:1297-1310. [CrossRef] [PubMed]
2. Zimmerman JE, Kramer AA, McNair DS, Malila FM, Shaffer VL. Intensive care unit length of stay: Benchmarking based on Acute Physiology and Chronic Health Evaluation (APACHE) IV. Crit Care Med. 2006;34:2517-29. [CrossRef] [PubMed]
3. Breslow MJ, Badawi O. Severity scoring in the critically ill: Part 1 – Interpretation and accuracy of outcome prediction scoring systems. Chest. 2012;141:245-52. [CrossRef] [PubMed]
4. Kramer A, Zimmerman J, Yi J. Performance of MPMo-III and APACHE IV mortality predictions in a multi-institutional ICU database – implications for national benchmarking. Crit Care Med. 2012;40:1–328 [Abstract]. [CrossRef]
5. Vincent J-L, Moreno R. Clinical review: Scoring systems in the critically ill. Crit Care. 2010;14:207-16. [CrossRef] [PubMed]
6. Chan T, Bleszynski MS, Buczkowski AK. Evaluation of APACHE IV predictive scoring in surgical abdominal sepsis: A retrospective cohort study. J Clin Diag Res. 2016;10:16-8. [CrossRef] [PubMed]
7. Ghorbani M, Ghaem H, Rezaianzadeh A, et al. A study on the efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran. F1000 research. 2017;6:2032. [CrossRef] [PubMed]
8. Afessa B, Gajic O, Keegan MT. Severity of illness and organ failure assessment in adult intensive care units. Crit Care Clinics. 2007;23:639-58. [CrossRef] [PubMed]
9. Keegan MT, Gajic O, Afess B. Comparison of APACHE III, APACHE IV, SAPS 3 and MPMoIII and influence of resuscitation status on model performance. Chest. 2012;142:851-8. [CrossRef] [PubMed]
10. Hadorn DC, Keeler EB, Rogers WH, Brook RH. Assessing the performance of mortality prediction models. RAND/UCLA/Harvard Center for Healthcare Financing Policy Research. 1993. ISBN: 0-8330-1335-1.
11. Verburg IWM, de Keizer NF, de Jonge E, Peek N. Comparison of regression methods for modeling intensive care length of stay. PLOS One. 2014;9:1-11. [CrossRef] [PubMed]
12. DeSalvo KB, Fan VS, McDonell MB, Fihn SD. Predicting mortality and healthcare utilization with a single question. HRS: Health Services Research. 2005;40:1234-46. [CrossRef]
13. Austin PC, Styerberg EW. Interpreting the concordance statistic of a logistic regression model: Relation to the variance and odds ratio of a continuous explanatory variable. Med Care. 2013;51:275-84. [CrossRef] [PubMed]
14. Freund Y, Lemachatti N, Krastinova E, Van Laer M, et al. Prognostic accuracy of Sepsis-3 criteria for in-hospital mortality among patients with suspected infection presenting to the emergency department. JAMA. 2017;317:301-8. [CrossRef] [PubMed]
15. Kuzniewicz MW, Vasilevskis EE, Lane R, et al. Variation in ICU risk-adjusted mortality: Impact of methods of assessment and potential confounders. Chest. 2008;133:1319-27. [CrossRef] [PubMed]
16. Khwannimit B. Bhurayanontachai R, Vattanavanit V. Validation of the sepsis severity score compared with updated severity scores in predicting hospital mortality in sepsis patients. Shock. 2017;47:720-5. [CrossRef] [PubMed]
17. Nassar AP, Mocelin AO, Andrade FM, Brauer L. SAPS 3, APACHE IV or GRACE which score to choose for acute coronary syndrome patients in intensive care units? Sao Paulo Med. J. 2013;131:173-8. [CrossRef]
18. Ayazoglu TA. A comparison of APACHE II and APACHE IV scoring systems in predicting outcomes in patients admitted with stroke to an intensive care unit. Anaesth Pain and Intens Care. 2011;15:7-12. Available at: http://www.apicareonline.com/original-article-a-comparison-of-apache-ii-and-apache-iv-scoring-systems-in-predicting-outcome-in-patients-admitted-with-stroke-to-an-intensive-care-unit/ (accessed 12/26/18).
19. Bauer PR, Kashyap R, League SC, Park JG, et al. Diagnostic accuracy and clinical relevance of an inflammatory biomarker panel for sepsis in adult critically ill patients. Diagn Microbiol Infect Dis. 2016;84:175–80. [CrossRef] [PubMed]
20. Kramer AA, Zimmerman JE. Predicting outcomes for cardiac surgery patients after intensive care unit admission. Semin Cardiothorac Vasc Anesth. 2008;12:175-83. [CrossRef] [PubMed]
21. Costa e Silva VT, de Castro I, Liano F, et al. Performance of the third-generation models of severity scoring systems (APACHE IV, SAPS 3 and MPM-III) in acute kidney injury critically ill patients. Nephrol Dial Transplant. 2011;26:3894-901. [CrossRef] [PubMed]
22. Soares M, Salluh JI. Validation of the SAPS 3 admission prognostic model in patients with cancer in need of intensive care. Intensive Care Med. 2006;32:1839-44. [CrossRef] [PubMed]
23. Oliveira VM, Brauner JS, Rodrigues Filho EIs SAPS 3 better than APACHE II at predicting mortality in critically ill transplant patients? Clinics (Sao Paulo). 2013;68:153-8. [CrossRef]
24. Tsai CW, Lin YF, Wu VC, et al. SAPS 3 at dialysis commencement is predictive of hospital mortality in patients supported by extracorporeal membrane oxygenation and acute dialysis. Eur J Cardiothorac Surg. 2008;34:1158-64. [CrossRef] [PubMed]
25. Doerr F, Badreldin AM, Can F, et al. SAPS 3 is not superior to SAPS 2 in cardiac surgery patients. Scand Cardiovasc J. 2014;48:111-9. [CrossRef] [PubMed]
26. Salciccioli JD, Cristia C, Chase M, et al. Performance of SAPS II and SAPS III scores in post-cardiac arrest. Minerva Anestesiol. 2012;78:1341-7. [PubMed]
27. Badreldin AM, Kroener A, Heldwein MB, et al. Prognostic value of daily cardiac surgery score (CASUS) and its derivatives in cardiac surgery patients. Thorac Cardiovasc Surg. 2010;58:392-7. [CrossRef] [PubMed]
28. Doerr F, Badreldin AMA, Heldwein MC, et al. A comparative study of four intensive care outcome prediction models in cardiac surgery patients. J Cardiothorac Surg. 2011;6:21. [CrossRef] [PubMed]
29. Hekmat K. Prediction of mortality in intensive care unit cardiac surgical patients, European J Cardiothorac Surg. 2010;38:104–9. [CrossRef] [PubMed]
30. Finnerty CC, Ju H, Spratt H, et al. Proteomics improves the prediction of burns mortality: results from regression spline modeling. Clin Transl Sci. 2012;5:243-9. [CrossRef] [PubMed]
31. Mikacenic C, Price BL, Harju-Baker S, et al. Two-Biomarker Model Predicts Mortality in the Critically Ill with Sepsis. Am J Respir Crit Care Med. 2017;196:1004-11. [CrossRef] [PubMed]
32. Cuenca AG, Gentile LF, Lopez MC, et al. Development of a genomic metric that can be rapidly used to predict clinical outcome in severely injured trauma patients. Crit Care Med. 2013;41:1175-85. [CrossRef] [PubMed]
33. Stortz JA, Mira JC, Raymond SL, et al. Benchmarking clinical outcomes and the immunocatabolic phenotype of chronic critical illness after sepsis in surgical intensive care unit patients. J Trauma Acute Care Surg. 2018;84:342-9. [CrossRef] [PubMed]
34. Muehlschlegel JD, Liu KY, Perry TE, et al. Chromosome 9p21 variant predicts mortality after coronary artery bypass graft surgery. Circulation. 2010;122:S60-5. [CrossRef] [PubMed]
35. Perry TE, Muehlschlegel JD, Liu KY, et al. Preoperative C-reactive protein predicts long-term mortality and hospital length of stay after primary, non-emergent coronary artery bypass grafting. Anesthesiology. 2010;112:607-13. [CrossRef] [PubMed]
36. Scicluna BP, van Vught LA, Zwinderman AH, et al. Classification of patients with sepsis according to blood genomic endotype: a prospective cohort study. Lancet Respir Med. 2017;5:816-26. [CrossRef] [PubMed]
37. Sharma NK, Salomao R. Sepsis through the eyes of proteomics: The progress in the last decade. Shock. 2017;47:17-25. [CrossRef] [PubMed]
38. Miajlovic H, Aogáin MM, Collins CJ, et al. Characterization of Escherichia coli bloodstream isolates associated with mortality. J Med Microbiol. 2016;65:71-9. [CrossRef] [PubMed]
39. Vasilevskis EE, Kuzniewicz MW, Brian A. Cason BA. Mortality Probability Model III and Simplified Acute Physiology Score II: Assessing their value in predicting length of stay and comparison to APACHE IV. Chest. 2009;136:89–101. [CrossRef] [PubMed]
40. Submitted Am J Epidemiol.
41. Male, E, Ghamande S, White HD. Superiority of APACHE IV vs. APR-DRG in evaluating ICU mortality based on discharge billing codes. Am J Resp Crit Care Med. 2017;195:A2772 [Abstract].
42. van Walraven C, Bennett C, Forster AJ. Administrative database research infrequently used validated diagnostic or procedural codes. J Clin Epidemiol. 2011;64:1054-9. [CrossRef] [PubMed]
43. Bota, P, Melot C, Ferreira L, et al. The Multiple Organ Dysfunction Score (MODS) versus the Sequential Organ Failure Assessment (SOFA) score in outcome prediction. Inten Care Med. 2002;28:1619-24. [CrossRef] [PubMed]
44. Zygun DA, Laupland KB, Fick GH, et al. Limited ability of SOFA and MOD scores to discriminate outcome: a prospective evaluation in 1,436 patients. Can J Anaesth. 2005;52:302-8. [CrossRef] [PubMed]
45. Ho KM, Williams TA, Harahsheh Y, Higgins TL. Using patient admission characteristics alone to predict mortality of critically ill patients: A comparison of 3 prognostic score. J Crit Care. 2016;31:21-5. [CrossRef] [PubMed]
46. Render ML, Deddens J, Freyberg R, Almenoff P, et al. Veterans Affairs intensive care unit risk adjustment model: validation, updating, recalibration. Crit Care Med. 2008;36:1031-42. [CrossRef] [PubMed]
47. Lipshutz AKM, Feiner JR, Grimes B, Gropper MA. Predicting mortality in the intensive care unit: a comparison of the University Health Consortium expected probability of mortality and the Mortality Prediction Model III. Intensive Care. 2016;4:35. [CrossRef] [PubMed]
48. Kim S, Kim W, Park RW. A comparison of intensive care unit mortality prediction models through the use of data mining techniques. Healthcare Inform Res. 2011;17:232-43. [CrossRef] [PubMed]
49. Pirracchio R, Petersen ML, Carone M, Rigon MR, Chevret S, van der Laan MJ. Mortality prediction in intensive care units with the Super ICU Learner Algorithm (SICULA): a population-based study. Lancet Respir Med. 2015;3:42–52. [CrossRef] [PubMed]

Acknowledgements

We would like to acknowledge the work of Maria Calleja and Banner Health Clinical Performance Analytics in providing the data used in our analysis.

Author’s contributions

Conception and design: RAR, RDG, KSR, MF, SCC

Data collection: RAR

Statistical analysis: RDG, RAR

Interpretation: RAR, RDG, KSR, MF, SCC

Writing the manuscript: RAR, RDG, KSR, MF, SCC

Guarantor taking full responsibility for integrity of the study: RAR

The authors have no conflicts of interest to report and there was no direct funding for this project.

Abbreviation List

• 2VSS: two-variable scoring system
• APACHE: Acute Physiology and Chronic Health Evaluation
• APS: acute physiology score
• AMI: acute myocardial infarction
• AUROC: area under the receiver operating curve
• CABG: coronary artery bypass grafting
• CASUS: cardiac surgery score
• GI: gastrointestinal
• ICU: intensive care unit
• LOS: length of stay
• MODS: multiple organ dysfunction score
• MPM: mortality probability model
• RAND (corporation): research and development
• R²: Pearson’s coefficient of determination
• SAPS: simplified acute physiology score
• SOFA: sequential organ failure assessment

Cite as: Raschke RA, Gerkin RD, Ramos KS, Fallon M, Curry SC. The explained variance and discriminant accuracy of APACHE IVa severity scoring in specific subgroups of ICU patients. Southwest J Pulm Crit Care. 2018;17:153-64. doi: https://doi.org/10.13175/swjpcc108-18 PDF