A Novel Framework for the Robust Implementation of the 3D Hoek–Brown Criterion in Shear Strength Analysis

Abstract

The shear strength of rock masses is a critical parameter in assessing the stability of rock engineering structures. Although the three-dimensional (3D) generalized Hoek–Brown (H-B) criterion captures the nonlinear strength of rock masses, including the influence of the intermediate principal stress, its adoption remains limited compared with the Mohr–Coulomb (M-C) criterion because it does not provide an explicit relationship between normal and shear stresses. To address this limitation, this study introduces an intermediate principal-stress coefficient ( λ ) and recasts the 3D criterion into a quasi-two-dimensional representation. Building on this reformulation and leveraging the geometric relationship between the M-C strength envelope and the Mohr circle, together with an equivalent-envelope transformation, a general framework is developed to compute rock-mass shear strength and the corresponding instantaneous shear parameters (internal friction angle and cohesion). To solve the resulting highly nonlinear implicit equations, a dual-layer iterative algorithm is proposed that couples an outer bisection loop with an inner fixed-point iteration. This scheme mitigates convergence difficulties and enables stable and efficient evaluation of rock-mass shear strength under the 3D generalized H-B criterion. Validation on four representative rock-mass cases demonstrates practical applicability, and sensitivity analyses indicate low sensitivity to rock-mass quality and disturbance factor. Moreover, three characteristic λ -dependent patterns are identified—symmetric, eccentric, and unidirectional distributions—clarifying the mechanistic basis of the intermediate principal-stress influence. Overall, the results provide theoretical support and practical tools for applying the 3D generalized H-B criterion in engineering practice.