A Novel Flexible Rayleigh–Exponential Mixture Detection Model for Line Transect Sampling

This study presents a novel flexible Rayleigh–exponential mixture detection model (REMDM) for estimating population abundance under line transect sampling. The proposed detection function combines a Rayleigh-type component with an exponential component to provide greater flexibility in modelling perpendicular distance data and capturing the complex detection patterns commonly observed in ecological surveys. The model exhibited smooth behaviour near the transect line and flexible tail decay, making it suitable for heterogeneous detection structures. Several statistical properties of the proposed REMDM were derived, including the probability density function, cumulative distribution function, moments, and hazard rate function. Parameters were estimated by using the maximum likelihood estimation method. The performance of the estimators is evaluated through extensive Monte Carlo simulation studies under various sample sizes and parameter settings. The simulation results indicate that the proposed estimators are consistent and efficient in terms of bias and mean squared error, with improved performance as the sample size increases. The applicability of the proposed model is demonstrated using a real perpendicular distance dataset and model performance is assessed using several goodness-of-fit measures, including the Akaike Information Criterion, Bayesian Information Criterion, Kolmogorov–Smirnov statistic, Anderson–Darling statistic, and Cramér–von Mises statistic. The results show that the REMDM provides a superior fit to several existing detection functions. In general, the proposed model offers a flexible and effective alternative for modelling detection probability and improving population abundance estimates in ecological distance sampling.