Validity and Reliability of ICD Symptom Coding and Its Implications for Symptom Assessment and Research Using Electronic Health Records

Symptom data is critical to the diagnostic process, but little is known about how well symptoms are coded in electronic health records. Our study aimed to quantify the accuracy and completeness of symptom coding by ICD-9 and ICD-10. We randomly selected 150 unique patient-admission records from MIMIC-IV version 2.2. Four clinical experts reviewed the clinical notes in those records to extract and map the noted symptoms to 65 predefined symptom categories. In addition, the ICD diagnosis codes in those records were automatically extracted and mapped to 1888 previously identified ICD symptom codes with their predefined categories as applicable. We then computed the performance measures of ICD symptom coding compared with the clinical expert-confirmed symptoms. The obtained sensitivity, specificity, positive predictive value, and negative predictive value with the 95% confidence intervals were, respectively, 0.134 [0.117, 0.153], 0.998 [0.996, 0.999], 0.909 [0.862, 0.941], and 0.872 [0.865, 0.879]. To our knowledge, this study is the first attempt to quantitatively validate ICD symptom coding that is not limited to any specific diseases. The low sensitivity highlights the need to improve symptom coding. In addition, clinical notes remain essential for symptom extraction in automated symptom assessment and secondary data analyses using electronic health records.