EvalDepApps: Co-creation of an evidence-based tool to assess health apps to manage Depression

Background

Mental health apps represent a digital revolution, providing 24/7 accessible tools that empower users while reducing stigmatization. However, no standardized method exists to assess their effectiveness and safety. The EvalDepApps project aims to develop an evidence-based tool to evaluate apps focused on depression management.

Methods

The project followed six stages: 1) systematic mapping of marketplaces; 2) meta-analysis of clinical effectiveness; 3) Delphi study for criteria consensus; 4) stakeholder co-creation workshops; 5) tool development and piloting (n = 13); and 6) final validation with healthcare professionals (n = 71) and individuals affected by depression (n = 59).

Results

Mapping identified 30 apps, and 29 RCTs were included in the meta-analysis. Digital interventions, particularly psychoeducation, goal setting, and gamification, showed a significant effect in reducing depressive symptoms (95% CI: -0.87, -0.37; I2=87%). Hybrid interventions were the most effective. The Delphi study reached consensus on 28 criteria. During workshops, the tool’s structure was created and subsequently refined during piloting to ensure usability. The final validation demonstrated high internal consistency (KR-20=0.85). Sampling adequacy was confirmed by KMO=0.756 and a significant Bartlett’s test (p < 0.001). CFA indicated an acceptable structural fit (RMSEA=0.077). A high correlation between item loadings and communalities (r = 0.98) further confirmed internal consistency. Finally, users and professionals rated EvalDepApps as a safe, functional, and user-friendly tool, highly valued for its capacity to assess app quality.

Conclusions

mHealth interventions are effective, but standardized evaluation is essential. By integrating clinical evidence with a human-centered co-creation approach, EvalDepApps provides a robust framework for assessing mental health apps, with high potential for adaptation to other health conditions.

Key messages

• mHealth interventions effectively reduce depressive symptoms, but standardized evaluation is vital to ensure clinical safety and reliability.

• Combining human-centered co-creation with rigorous statistical validation provides a reliable and usable tool to assess the quality of digital interventions.