Arabic evaluation suites
Task-level benchmarks in MSA and relevant dialects, run per release — the number that keeps vendors and models honest.
Services · 04
Your customers live in Arabic. Your AI should too.
A model that drops twenty points in Arabic isn’t twenty percent worse here — it’s unusable. We engineer systems that perform across fusha, Gulf dialects, and mixed Arabic-English reality, and we prove it with evals.
The situation
Most enterprise AI in the region is English-first with Arabic as a translation afterthought — and it shows the moment a customer writes in dialect, a contract arrives in fusha, or an interface renders RTL as an afterthought mirror. The performance gap is invisible in the vendor demo and everywhere in production.
We treat Arabic as an engineering discipline: benchmark suites across MSA and Gulf dialects, model selection driven by measured Arabic performance, tokenization and cost analysis for Arabic text, and interfaces designed right-to-left from the first wireframe. For government-facing systems, Arabic isn’t a feature — it’s the requirement.
What we deliver
Dialect-aware assistants
Customer- and citizen-facing systems that handle Gulf dialects, code-switching, and Arabizi without falling back to English.
Arabic document pipelines
Extraction and processing for contracts, correspondence, and identity documents — including handwriting and mixed-language layouts.
RTL-native interfaces
Products designed for Arabic first — typography, direction, and interaction patterns — with English as the second layer, not the template.
Model strategy for Arabic
Selection and fine-tuning guidance grounded in measured performance and token economics, not marketing benchmarks.
How the engagement runs
Weeks 1–3
Benchmark
Build the Arabic eval suite for your actual tasks and score candidate models on it. The gap analysis usually surprises.
Weeks 4–10
Engineer
Build or retrofit the system against the suite — prompts, retrieval, tuning, and interface — with native-speaker review in the loop.
Weeks 11–12
Certify
Release-gate evals, dialect coverage report, and a maintenance plan so Arabic quality survives model upgrades.
How success is measured
- Arabic task performance within an agreed band of English
- Dialect coverage measured, not assumed
- Zero Arabic-blocking launch defects post-certification
- Token cost per Arabic interaction modeled and controlled
Straight answers
It changes quarterly, and it depends on the task — which is exactly why we sell you an eval suite and a method, not a model opinion.
Usually yes: benchmark first, then fix in order of impact — retrieval and prompts before tuning, tuning before replacement.
Our depth is MSA and Gulf dialects; for Levantine, Egyptian, and North African coverage we extend the same eval-first method with region-specific reviewers.
Scope a arabic-first ai engagement.
One paragraph on where you are, and we’ll come back with the shape of the engagement we’d run — scope, duration, and the number it should move.