Empirical 5/3 Spectral Scaling in Contextual Representations of Language

Natural language is a complex system that exhibits robust statistical regularities. Here, we represent text as a trajectory in a high-dimensional embedding space generated by transformer-based language models, and quantify scale-dependent fluctuations along the token sequence using an embedding-step signal. Across multiple languages and corpora, the resulting power spectrum exhibits a robust power law with an exponent close to 5/3 over an extended frequency range. This scaling is observed consistently in contextual embeddings from both human-written and AI-generated text, but is absent in static word embeddings and is disrupted by randomization of token representation order. These results show that the observed scaling reflects multiscale, context-dependent organization rather than lexical statistics alone. These results identify an empirical spectral regularity in contextual language representations and provide a quantitative, model-agnostic benchmark for studying their multiscale structure.