External validation and refinement of the Psychosis Polyrisk Score to detect individuals at clinical high risk for psychosis
Riccardo Stefanelli, Dominic Oliver, Stefano Damiani, Marika Orlandi, Umberto Provenzani, Elisa Bortolin, Margherita Rovida, Jean Addington, Luis Alameda, Celso Arango, Nicholas J Breitborde, Matthew Broome, Kristin S Cadenhead, Monica E Calkins, Ricardo E Carrion, Rolando I Castillo-Passi, Yu Hai Eric Chen, Jimmy Choi, Philippe Conus, Barbara A Cornblatt, Covadonga M Díaz-Caneja, Lauren M Ellman, Pablo A Gaspar, Carla Gerber, Louise Birkedal Glenthøj, Leslie E Horton, Christy LM Hui, Joseph Kambeitz, Lana Kambeitz-Ilankovic, Matcheri S Keshavan, Minah Kim, Sung-Wan Kim, Nikolaos Koutsouleris, Jun Soo Kwon, Kerstin Langbein, Daniel Mamah, Daniel H Mathalon, Vijay A Mittal, Meredete Nordentoft, Godfrey D Pearlson, Nora Penzel, Jesus Perez, Diana O Perkins, Albert R Powers, Jack R Rogers, Fred W Sabb, Jason Schiffman, Jai Shah, Steven M Silverstein, Stefan Smesny, William S Stone, Gregory P Strauss, Judy L Thompson, Rachel Upthegrove, Swapna Verma, Jijun Wang, Daniel H Wolf, TianHong Zhang, Sylvain Bouix, Cheryl M Corcoran, Tina Kapur, Ofer Pasternak, Carrie Bearden, Rene S Kahn, John M Kane, Patrick D McGorry, Barnaby Nelson, Martha E Shenton, Scott W Woods, , Paolo Fusar-PoliBackground
The Psychosis Polyrisk Score (PPS) was developed to characterise exposure to environmental and developmental risk factors for psychosis. In previous studies, the PPS showed promise particularly in discriminating between individuals at clinical high risk for psychosis (CHR-P) and community controls (CC).
Objectives
This study aimed to (1) perform an external validation of the PPS model in a large independent sample and (2) refine the model using expanded predictor data.
Methods
Participants were recruited through the AMP SCZ program: n=1642 CHR-P, n=519 CC. We performed a formal external validation of the PPS original model. The model was then refined incorporating expanded predictor data (perceived loneliness, neighbourhood quality) and increasing the granularity of existing predictors (ethnicity, childhood trauma, parental severe mental illness). Next, we evaluated the refined model through repeated nested cross-validation and compared it to the original model retrained in the new dataset. Discrimination (Harrell’s C-index) and calibration (intercept and slope) were used to assess model performance.
Findings
The externally validated PPS original model showed good discrimination (C=0.777, 95% CI 0.756 to 0.797) between CHR-P and CC and good calibration (intercept=0; slope=1.003) after recalibration. The refined PPS model exhibited higher internally cross-validated discrimination than the retrained original model (C=0.876, 95% CI 0.860 to 0.891 vs C=0.796, 95% CI 0.775 to 0.816), as well as good calibration performance (intercept=−0.008; slope=1.031).
Conclusions
The PPS showed good generalisability to a large, independent sample, and performance improved after model refinement.
Clinical implications
The PPS represents a potential scalable tool within stepped assessment frameworks for identifying individuals who warrant further CHR-P assessment.
Trial registration number