A Sparse Model for Ordinal Regression

In ordinal regression, where output classes exhibit a natural ordering (e.g., age groups or disease severity), two structural properties are highly desirable: unimodality—the predicted probability distribution should be single-peaked around the true class—and sparsity, meaning the probability mass should concentrate on a small subset of classes. However, existing approaches either ignore ordinality entirely or enforce unimodality while ignoring sparsity. We propose unisparse, a differentiable output layer that first uses the Pool-Adjacent-Violators Algorithm (PAVA) to obtain the exact Euclidean projection onto the unimodal cone and then applies sparsemax to project the result onto the probability simplex. The second projection is order-preserving, and thus the composition guarantees a unimodal probability vector while inducing exact zeros. Unlike parametric approaches (e.g., binomial unimodal), the method does not impose a prescribed distributional family. The composition is piecewise differentiable and supports end-to-end training. Experiments on seven benchmarks (five tabular and two vision) show that unisparse is competitive with six ordinal and multiclass baselines (Softmax, Sparsemax, UnimodalNet, CORAL, CORN, ORD-ACL, and VS-SL) in terms of accuracy and MAE while consistently producing much smaller supports. Our layer is architecture-agnostic and easy to integrate into modern deep network architectures.