Arabic NLP in Production: Why Standard LLMs Struggle and What to Do About It

Most engineers building LLM-powered products have worked almost entirely in English. That's not a criticism - it's just true. The major model APIs, the benchmark datasets, the blog posts, the Stack Overflow answers - all of it skews heavily toward English. When these teams get handed a project with Arabic-speaking users, they typically do one of two things: they assume a capable multilingual model handles it well enough, or they discover the failure modes the hard way after deployment.

Both paths lead to the same conversation eventually. Arabic NLP in production has specific, well-understood failure modes. Understanding them before you build saves significant pain later.

The Tokenization Problem Is Worse Than You Think

Here's where most teams get surprised first.

Large language models don't process text as characters or words - they process tokens, and the tokenizer determines how text gets split into those tokens. GPT-4's tokenizer, like most BPE (Byte Pair Encoding) tokenizers trained on English-dominated corpora, was optimised for Latin-script languages. The consequence for Arabic is brutal: Arabic text tokenizes at roughly three to four times the token count of equivalent English content.

That matters for three reasons. Cost, since you're paying per token on API-based models. Context window consumption, since your 8,000-token window fits far less Arabic content than the benchmark suggested. And model performance, because the model has seen far less Arabic token sequence data during training, meaning its representations of Arabic subword units are weaker than its English equivalents.

The deeper issue is how Arabic morphology interacts with BPE tokenization. Arabic is an agglutinative language - prefixes and suffixes attach directly to the root word as single written strings. The word وَسَيَكْتُبُونَهَا (roughly: "and they will write it") is one token in human reading, but it carries a conjunction, a future marker, a verb root, a plural subject agreement, and a pronominal object. A tokenizer optimised for English will shred this into fragments that carry no coherent linguistic meaning at the token boundary level. The model is then trying to learn relationships between chunks that don't correspond to anything semantically real in Arabic.

AraBERT, CAMeL-BERT, and JAIS - Arabic and Arabic-centric multilingual models - were trained with tokenizers aware of this morphological structure. That's not a minor advantage. It's foundational to how well the model internalises Arabic text.

Morphological Complexity Is the Other Layer

Arabic's root-pattern morphology is genuinely unusual from an NLP standpoint. Most words derive from a trilateral root - three consonants - and meaning shifts based on the pattern applied to that root. The root ك-ت-ب (k-t-b) produces كَتَبَ (he wrote), كِتَاب (book), كَاتِب (writer), مَكْتَبَة (library), مَكْتُوب (written/letter), and dozens more. A model that hasn't been trained on substantial Arabic data will not reliably capture these relationships - it treats them as unrelated vocabulary items rather than morphological variants of shared meaning.

This shows up in practical systems as inconsistent entity recognition, poor generalisation on inflected forms you didn't see in your test set, and classification errors when the same root appears in different morphological patterns. You test on "كاتب" and it works. Then it fails on "كُتَّاب" (plural) because the model didn't learn the morphological relationship - it memorised surface forms.

For NER, intent classification, and any task requiring generalisation across inflected variants, this is a real production problem, not a theoretical one.

Dialectal Variation Is Not a Minor Issue

Modern Standard Arabic (MSA) is what's written in newspapers, formal documents, and most training corpora. It's nobody's native spoken dialect. What your users actually type - especially in customer-facing applications - is a mix of Gulf Arabic, Egyptian Arabic, Levantine Arabic, Moroccan Darija, and increasingly a code-switched blend of Arabic and English that linguists call "Arabizi" when written in Latin script.

For a KSA deployment, you're primarily dealing with Najdi and Hejazi dialect variation, but urban users in Riyadh also write a heavily code-switched hybrid that's neither clean MSA nor clean dialect. A customer service chatbot trained on MSA will handle formal queries acceptably and fall apart the moment someone types كيف احصل على ريجيكت؟ - a perfectly natural Gulf Arabic sentence mixing Arabic grammar with the English loanword "reject."

Here's what this means practically: if you're evaluating a multilingual model's Arabic capability using MSA benchmarks, you're not measuring what your users will actually send it. JAIS is trained on a mix of MSA and dialectal data and handles Gulf Arabic better than most. CAMeL Tools includes dialect identification as a preprocessing step, which lets you route inputs to dialect-appropriate processing. Neither solves the problem completely - dialect-specific fine-tuning on real user data from your application context is the honest answer for high-stakes customer-facing use cases.

RTL Rendering Is a Different Class of Problem

Tokenization and morphology are model problems. Right-to-left rendering is a frontend problem, and it's worth separating the two because they have entirely different solutions.

Arabic text in a UI context requires RTL layout direction. That sounds simple. In practice, it produces a category of bugs that English-trained frontend engineers consistently underestimate until they've hit them. Mixed-direction content - an Arabic sentence containing an English brand name, a URL, a number, or a code snippet - triggers the Unicode bidirectional algorithm, which determines display order based on character properties. When that algorithm doesn't have explicit direction markers to work with, it guesses. It guesses wrong often enough to cause real rendering issues: parentheses that appear on the wrong side, numbers that read in the wrong order, punctuation that jumps to unexpected positions.

The fix is explicit direction control. In React, dir="rtl" on the container is table stakes, but it doesn't handle mixed-content cases. For inline mixed-direction text, you need Unicode control characters - specifically LRE/RLE markers or the <bdi> HTML element - wrapping the directional segments. Styling that uses text-align: right instead of text-align: start will behave incorrectly when you switch direction. Every component in the library needs to have been tested against RTL layout, not just the top-level container.

We caught a particularly painful version of this in a voice agent transcription display: the Arabic transcript rendered correctly, but timestamps and speaker labels - left-to-right content - were appearing at the wrong end of the line because the bidi algorithm was treating the entire line as RTL. The fix took an hour. Finding it took three days.

When to Use a Multilingual Model vs. Fine-Tuning on Arabic Corpora

The decision tree here is similar to the general fine-tuning question, but Arabic-specific factors shift the thresholds.

Start with a strong Arabic-aware model when your task is general, your volume is moderate, and your users write in MSA or near-MSA. JAIS-13B handles Arabic comprehension and generation well enough for summarisation, Q&A over Arabic documents, and classification tasks with clearly defined categories. For API-based deployments, GPT-4o has meaningfully better Arabic performance than GPT-3.5 - the gap is large enough to matter for production quality.

Reach for fine-tuning earlier than you would for English tasks, for two reasons. First, because the general-purpose Arabic capability of even strong multilingual models is lower than their English capability, meaning the improvement from task-specific fine-tuning is proportionally larger. Second, because dialectal and domain-specific vocabulary - the terminology of your industry, your users' actual dialect, your brand voice - is underrepresented in pretraining data in a way that English domain vocabulary typically isn't.

For a KSA logistics client we built a document classification system for, GPT-4 with a well-engineered Arabic prompt achieved around 78% accuracy on their internal shipment document categories. A fine-tuned AraBERT model on 4,000 labelled examples from their own document archive hit 94%. The gap was almost entirely explained by domain terminology and document formatting conventions that exist nowhere in GPT-4's pretraining data.

The practical threshold: if you have 2,000 or more labelled examples in the dialect and domain your users actually operate in, fine-tuning on an Arabic-aware base model will outperform a prompted frontier model on task-specific metrics. If you don't have that data yet, start with a capable multilingual model while you collect it.

What We've Actually Learned Building for Arabic-Speaking Users in KSA

A few things that didn't come from documentation:

Preprocessing matters more than it does in English. Arabic text in the wild contains inconsistent use of diacritics, variation between Arabic-Indic numerals (٣) and Western numerals (3), inconsistent hamza forms, and whitespace patterns that don't follow MSA conventions. A normalisation step before any model inference - standardising numeral forms, removing diacritics unless your task requires them, normalising hamza - reduces noise that would otherwise degrade model performance. This is unglamorous work that makes a real difference.

Your test set needs to reflect real user inputs. This sounds obvious and gets ignored constantly. Building a test set from MSA news articles and evaluating on customer service chat from Gulf users are not the same thing. We've seen systems that looked strong on internal benchmarks and degraded significantly when the pilot launched with real users. Build your evaluation set from real inputs as early as you can.

Latency is a user experience problem that Arabic tokenization makes worse. If you're building a real-time Arabic application on a frontier model API, model your token counts using real Arabic input, not English-equivalent length. The token inflation means responses take longer and cost more than your English-based estimates suggested. For latency-sensitive applications - voice agents, inline suggestions - this is sometimes the deciding factor in moving to a self-hosted Arabic model.

Code-switching needs an explicit handling strategy. Decide early what you do with inputs that mix Arabic and English. Do you detect language and route? Do you pass it through to the model as-is? Do you normalise? There's no universally right answer - it depends on your use case - but not deciding results in inconsistent behavior that's hard to debug and harder to explain to stakeholders.

Arabic NLP is a solvable problem. It just requires treating Arabic as a first-class engineering concern from the start, not an afterthought you handle by pointing a general-purpose model at it and hoping.

If you're building a system that needs to serve Arabic-speaking users and you want to get the architecture right before you're debugging in production, talk to us. This is work we do specifically, not a capability we bolt on.