When would I be surprised? Variability in predicted probability of survival for being “surprised” and “not surprised” to the surprise question

Purpose: The surprise question is commonly used in clinical practice; however, the variability in how “surprise” is defined is unclear. We examined the variability in the predicted probability of survival for being “surprised” and “not surprised” among palliative care physicians across seven timeframes. Methods: Palliative care specialists completed the surprise question for 3 days, 1 week, 2 weeks, 1 month, 2 months, 3 months, and 6 months for patients with advanced cancer at an outpatient clinic and concurrently predicted the probability of survival (0–100%). The primary outcome, coefficient of variation (CV) for the predicted probability of survival, was computed for “surprised” and “not surprised” answers for each physician and each timeframe. A mixed-effect logistic regression assessed the probability threshold for being “surprised.” Results: Twenty physicians provided 3024 survival estimates for 216 patients (mean age 61, 50% female). When “surprised,” physicians consistently predicted a probability of survival > 50%, with low variation among physicians (mean CV 6–23%) and across timeframes (mean CV 5–20%). In contrast, physicians who answered “not surprised” predicted a probability of survival from 2 to 100%, with high variation among physicians (mean CV 15–83%) and across timeframes (mean CV 22–69%). Variability increased with longer timeframes. The probability of survival thresholds for being “surprised” were ≥ 74%, ≥ 62%, ≥ 68%, ≥ 83%, and ≥ 84% for 1 week, 2 weeks, 1 month, 2 months, and 3 months, respectively. Conclusion: We found low variability for predicted probability of survival when clinicians were “surprised” but high variability when they were “not surprised.”