NLU Difficulty-Aware Routing for Conversational Expert Systems, Avoiding Post-Hoc Cascading Through Pre-Inference Decision Making

Hybrid approaches for conversational NLU commonly rely on cascading strategies, where model selection is performed after inference based on output confidence or uncertainty. Although effective, such post-hoc decision processes introduce redundant computation and accumulated latency due to sequential model execution. In this paper, we propose a pre-inference routing framework that selects an appropriate model before executing any candidate NLU model and without relying on candidate-model confidence or uncertainty. Instead of relying on output-dependent signals, we model NLU difficulty as a multi-dimensional representation of the joint intent recognition and slot extraction task, thereby distinguishing the proposed approach from prior post-hoc routing methods that rely primarily on confidence or uncertainty. To further reflect real-world dialogue characteristics, we incorporate temporal conversational factors, including contextual dependency and topic transitions. Based on these features, a lightweight routing mechanism efficiently assigns each utterance to an appropriate model from a pool of domain-specialized and general-purpose models. Experimental results on single-domain turns show that the proposed method maintains competitive Semantic Accuracy while reducing high-capacity model invocation, cost, and latency, thereby improving inference efficiency. These results demonstrate that input-observable difficulty features can provide a practical basis for pre-inference model selection without relying on post-hoc cascading.