Information-Geometric Selection for Validation-Guided Ensembles of Extreme Learning Machines: The LEA-EELM Framework

Extreme learning machines (ELMs) solve output weights in closed form, so an optimized population of ELMs leaves open which members should form the ensemble. We report a controlled comparison of five rules for this selection step inLEA-EELM, where a Lotus Effect Algorithm (LEA) variant generates fuzzy-activation ELM candidates and M=10 members are chosen from a shared top-2M validation shortlist. Two rules are derived from the Fisher information I(β)=H⊤H/σ2 of a candidate’s linear-Gaussian output stage. Because H⊤H is only positive semidefinite and candidates have different hidden layers, F-complementary compares candidates by the Kullback–Leibler divergence between their predictive distributions, not by a Fisher–Rao distance. For F-norm, we prove that, with least-squares output weights, ∥Hβ∥2=∥T∥2−∥T−Hβ∥2, so the rule selects the worst training fits; it is significantly worse than β-norm on three of four regression benchmarks. F-complementary performs like greedy validation selection and is never significantly better. The information-geometric rules therefore add no measured benefit; the gains come from validation-guided greedy selection, which lowers test RMSE on three of four regression datasets and raises accuracy only on Satellite and Letter, but not on Letter after duplicate rows are removed. All findings are conditional on the candidate pool of a non-canonical (subtractive-sign) LEA variant.