Protecting User Privacy in LLM Reasoning: A Textual Differential Privacy Framework for Context-Aware Applications

The growing adoption of context-aware reasoning in Large Language Model (LLM)-based applications, particularly through retrieval-augmented and prompt-enriched architectures, has substantially broadened the utility of LLMs in specialized, real-world scenarios. Yet this progress comes with an underexplored vulnerability: users must supply sensitive contextual information as part of the reasoning process, exposing private data to external LLM service providers and raising serious concerns about informational self-determination. Existing privacy-preserving approaches in NLP are largely ill-suited to this setting, as they either degrade reasoning quality unacceptably or fail to provide formal privacy guarantees. This paper addresses this gap by introducing Textual Differential Privacy (TDP), a principled framework that embeds differential privacy guarantees directly into the textual context supplied to LLMs, enabling privacy-preserving reasoning without sacrificing the semantic coherence that models depend upon. Central to the framework is a Differential Embedding Substitution mechanism, which perturbs sensitive entities in the input space while preserving the structural and semantic properties necessary for downstream reasoning tasks. The framework further introduces a formal privacy loss quantification scheme, providing practitioners with interpretable metrics to navigate the privacy–utility trade-off in deployment. We prove that the mechanism, instantiated as an exponential mechanism over a semantically constrained candidate set, satisfies metric differential privacy with a privacy budget that scales with the cosine distance between entities, and empirical evaluation across context-aware reasoning benchmarks validates its practical effectiveness. By treating privacy as a first-class design principle in LLM-based architectures, this work contributes toward a more trustworthy and broadly deployable foundation for intelligent, context-sensitive AI systems.