A Scoping Review of Composite Evaluation Approaches for Software Architectures on Resource-Constrained Hardware

Background: Hardware-software co-design for resource-constrained systems requires evaluation methods that integrate multiple quality attributes rather than isolated performance indicators. Objectives: This scoping review maps composite evaluation approaches used to assess architecture-relevant software or system artifacts, including software architectures where applicable, on resource-constrained hardware, while distinguishing explicitly aggregated quantitative composite metrics from qualitative and user-centric forms of evaluation. Methods: Using a PRISMA-ScR-informed reporting structure, the review analyzes a curated corpus of publications to identify foundational quality attributes, evaluation mechanisms, target hardware platforms, and architectural trade-off methods. Title and abstract screening, full-text eligibility assessment, and data charting were conducted by two investigators, with a third investigator available to resolve discrepancies when necessary. Results: Within the reviewed corpus, a descriptive shift is observed from single-attribute optimization toward broader multi-attribute evaluation, including explicit objective functions, benchmark-guided system-level modeling, structured qualitative trade-off assessment, and Quality-of-Experience (QoE)-oriented evaluation. These mechanisms are reported across FPGA-based accelerators, ARM-based edge devices, low-end IoT frameworks, heterogeneous intelligent devices, and XR platforms. However, explicitly aggregated architectural-level composite metrics remain rare, and representative studies such as FLASH, HW-FlowQ, Monarch, and ILLIXR illustrate related but distinct evaluation forms, rather than a single unified metric family. Conclusions: The reviewed corpus indicates that composite evaluation approaches support architecture-relevant reasoning under hardware constraints, although further work is needed to improve reproducibility, operational consistency, and cross-domain validation.