Evaluating LLM Tokenization Efficiency on Kuwaiti Dialect, MSA and Other Arabic Dialects

Tokenization affects the efficiency and cost of large language models (LLMs), especially for dialectal Arabic. To the best of our knowledge, this is the first study to evaluate LLM tokenization for the Kuwaiti Arabic dialect. We evaluated twelve LLM tokenizers using the MADAR Multi-dialect Arabic Parallel Dataset, along with a constructed KA dataset validated by native speakers, resulting in 57,805 semantically aligned sentences across Modern Standard Arabic (MSA) and four dialects: Kuwaiti (KA), Qatari (QA), Egyptian (EA), and Lebanese (LA). We used four main efficiency and degradation metrics: compression rate (CR), fertility rate (FR), dialect degradation delta, and token overhead (TO%). We also report the out-of-vocabulary rate (OOV) as an additional diagnostic indicator. MSA was tokenized most efficiently (CR = 2.71, FR = 1.98), while all dialects were less efficient. KA showed moderate degradation (CR = 2.55, FR = 2.12), whereas Lebanese degraded the most. KA required about 6.7% more tokens than MSA on average, reaching 8.6% for GPT-4o but only 2.7% for ALLaM-7B-Instruct-preview, raising inference cost under token-based pricing. Across the eight fine-tuned open-weight LLMs, tokenization efficiency showed no clear or consistent association with macro-F1 on the two KA short-text classification tasks. Overall, the results suggest that tokenization efficiency is associated with tokenizer design, vocabulary adaptation, and segmentation behavior, while vocabulary coverage alone does not explain the observed differences in CR and FR.