IslamicMMLU: A Benchmark for Evaluating LLMs on Islamic Knowledge
Large language models are increasingly consulted for Islamic knowledge, yet no comprehensive benchmark evaluates their performance across core Islamic disciplines. We introduce IslamicMMLU, a benchmark of 10,013 multiple-choice questions spanning three tracks: Quran (2,013 questions), Hadith (4,000 questions), and Fiqh (jurisprudence, 4,000 questions). Each track is formed of multiple types of questions to examine LLMs capabilities handling different aspects of Islamic knowledge. The benchmark is used to create the IslamicMMLU public leaderboard for evaluating LLMs, and we initially evaluate 26 LLMs, where their averaged accuracy across the three tracks varied between 39.8% to 93.8% (by G...
| Authors | Ali Abdelaal, Mohammed Nader Al Haffar, Mahmoud Fawzi et al. |
|---|---|
| Primary category | cs.CL |
| Published | 24 March 2026 |
| Last updated at source | 11 July 2026 |
Connected in the knowledge graph
- Fatan-Cultural-Eval/fatanDataset · cited by
Models and datasets referencing this paper
- Fatan-Cultural-Eval/fatanDataset
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 17 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.