Research · Open Arabic medical encoder
vioBERT-v3: the first open Arabic medical BERT, built in Cairo
General Arabic encoders only see news and social text; English biomedical models never see Arabic. We domain-adapted MARBERTv2 on 1.12 million Arabic medical documents, cutting clinical perplexity by 82.7% and lifting fill-mask accuracy by 15.6 points, then published the encoder and the recipe.
- Base model
- MARBERTv2
- Domain adaptation
- 1.12 million Arabic medical documents
- Training
- Whole-word masking, 22K steps
- Masked-LM perplexity
- 24.34 → 4.21 (−82.7%)
- Downstream classification
- +1.62 points macro-F1, 95% CI above zero
- Licence
- Apache 2.0
MARBERTv2 (base)70.4%
vioBERT-v278.9%
vioBERT-v385.9%
199 held-out masked items · DataCollator MLM, 15% masking, 2026.
Limits
This is a research model, published openly with its method. It is not for diagnosis, triage, dosing or treatment decisions, and its output needs verification by a qualified clinician. We keep research models separate from the production systems hospitals run.
Updated