From virtual assistants, to real-time translation, to sentiment analysis, Natural Language Processing (NLP) is transforming how we manage and interact with language online and in the field of data science specifically.
While English and to a lesser extent Mandarin dominate NLP in its current state, there is a whole new level of complexity and lucrative opportunity for NLP presented by the Arabic language, spoken by upwards of half a billion people.
Here, we’ll explore the unique challenges machines face when learning Arabic, as well as what models and tools arose to close this gap, and where the field is heading as it stands.
Why Arabic is challenging for NLP
It’s a mistake to consider Arabic as a unified language in the same sense that we could consider English to be a unified language.
While there is Modern Standard Arabic (MSA), which is utilized in a majority of official capacities such as governmental, news, or academia, Arabic in general is not just a single language, but rather a spectrum of diverse dialects built upon a shared classical root (meaning, MSA), and the vast majority of the Arabic-speaking world converses in local dialects and not the classical root.
This creates several major hurdles for NLP systems:
Morphological richness: A single word can carry verb tense, number, gender, if not more. Take for example “كتبوا” [katabu], which is a single word of five letters that means “they wrote.”
Dialect diversity: While MSA is used in most official capacities, most conversations across the Arab world occur using local dialects that vary dramatically. In fact, the Syrian, Saudi, Egyptian, Iraqi, and Moroccan dialects, for example, are so different from each other it is not uncommon for speakers of these dialects to have trouble even understanding one another.
Another important aspect of dialect diversity is how different cultural or dialect contexts can drastically change the connotations of certain words: the word “عافية” [aafiya] is used to convey thanks in Levantine Arabic, while in the Moroccan it means Hellfire.
Missing diacritics: Arabic text is often written without the crucial vowel markers, and most readers simply infer those vowel markers using context clues. This creates ambiguity for machine learning, however: for example, word “علم” [‘alam] can mean “flag,” “science,” or “he knew”, depending not the implied vowel markers that would not be readily apparent to someone not intimately familiar with the Arabic language.
Code-switching: Western vocabulary, such as English and French, is often colloquially interspersed among Arabic speakers, especially online and in casual conversations, presenting problems for language models that are not adequately trained to handle mixed-language input.
Right-to-left script and other textual challenges: Arabic text is written from right-to-left, and letters within a word are connected together in particular ways, requiring extra care in design and rendering (we’ve all seen Google translate mishaps, or Arabic tattoos with unconnected letters or written).
Word embeddings: The first wave
Prior to transformers taking over NLP, Arabic researching relied heavily on word embeddings, or vectors that represent words in multi-dimensional space.
Some of the notable word embeddings include:
- AraVec, which is trained on Arabic news and Arabic Wikipedia.
- FastText Arabic, which is a Facebook AI project that is useful for morphologically rich languages as it includes subword information.
Despite the efficiency of these models, they were severely limited by the lack of context provided in their processing. For the example we mentioned above, the word “علم” would receive the same vector in all its uses, regardless of the fact that context would dramatically alter its intended meaning and use.
Transformer models: The game changer
That limitation was addressed by the introduction of transformers, which are deep learning models more adept at understanding language through context.
A number of transformer-based models have emerged for Arabic:
- AraBERT, which has become the go-to for Modern Standard Arabic modeling.
- MARBERT, which excels in dialect-heavy and informal text due to being trained on Arabic tweets.
- ARBERT & Arabic ALBERT, which were designed for more efficiency with a solid general understanding of Arabic.
These models are greatly improved performance on tasks such as sentiment analysis, named entity recognition (NER), and answering questions, and are readily available on Hugging Face for fine-tuning.
What gaps still exist for Arabic NLP
Despite the strides made, there is still much development required to fully realize the potential of Arabic NLP:
More dialect coverage, as most models still perform best on MSA and some dialects such as Sudanese or Moroccan are as yet underrepresented.
Larger and more diverse datasets, particularly for use in processing conversational and informal text that is used by the vast majority of the Arabic-speaking world.
Bias detection and mitigation, as cultural and religious biases still do creep into model performances.
More sophisticated tools, as core NLP libraries such as spaCy still lag greatly behind in Arabic support compared to English.
However, as more researchers, scientists, developers, and communities get involved, Arabic NLP is slowly filling those gaps, becoming more accurate, inclusive, efficient, and capable of real-world applications across the Arab world.
We are currently seeing developments made in matters such as:
- Dialect-specific models (like GigaBERT or MARBERT).
- Open-source projects like CAMeL tools or QALB.
- Arabic LLMs trained on billions of words (like Noor and Jais).
- Research in hybrid MSA-dialect training.
Final thoughts
While major strides have been made in Arabic NLP, there is still a long way to go to fully capture the potential of this rising field.
With more than 400 million Arabic speakers across two continents with dramatically varying dialects and cultural specificities, it’s clear there is much to be gained in developing more competent and nuanced NLP models for Arabic, be it to locate business, investment, and marketing opportunities, or political and social research.
Appendix
- Try AraBERT or MARBERT on Hugging Face
- Explore dialect data at CAMeL Lab
- Follow projects like QALB for annotated corpora
Mohammad Abou-Ghazala 