Bambara ASR Leaderboard
Powered by MALIBA-AI • "No Malian Language Left Behind"
🏆 Current Champion: MALIBA-AI/bambara-whisper-small
Main Leaderboard
Choose how to rank the models
📋 Leaderboard Rankings - Lower scores indicate better performance
1 | bambara-whisper-small | 22.64 | 10.94 | 🥉 Good | Whisper-based | 2025-03-15 |
📊 Visual Performance Comparison
🎯 Automatic Speech Recognition Evaluation Metrics
Word Error Rate (WER)
WER measures transcription accuracy at the word level:
- Formula:
(Substitutions + Insertions + Deletions) / Total Reference Words - Range: 0% (perfect) to 100%+ (very poor)
- Interpretation:
- 0-5%: 🏆 Excellent performance
- 5-15%: 🥉 Good performance
- 15-30%: 📈 Fair performance
- 30%+: Poor performance
Character Error Rate (CER)
CER measures transcription accuracy at the character level:
- Advantage: More granular than WER, captures partial matches
- Benefit for Bambara: Particularly valuable for agglutinative languages
- Typical Range: Usually lower than WER values
Combined Score (Primary Ranking Metric)
Formula: Combined Score = 0.7 × WER + 0.3 × CER
- Rationale: Balanced evaluation emphasizing word-level accuracy
- Usage: Primary metric for model ranking
🎯 Performance Categories
- 🏆 Excellent: < 15% Combined Score
- 🥉 Good: 15-30% Combined Score
- 📈 Fair: > 30% Combined Score
Submit Your Bambara ASR Model
🚀 Ready to benchmark your model? Submit your results and join the leaderboard!
Follow these steps to submit your Bambara ASR model for evaluation.
Use a descriptive name (organization/model format preferred)
Select the type/architecture of your model
Country or region of the developing institution
📋 Submission Requirements
CSV Format:
- Columns:
id,text - Match all reference dataset IDs
- No duplicate IDs
- Text transcriptions in Bambara
Data Quality:
- Clean, normalized text
- Consistent formatting
- Complete coverage of test set
Upload your model's transcriptions in the required CSV format
📊 Updated Leaderboard
1 | bambara-whisper-small | 22.64 | 10.94 | 🥉 Good | Whisper-based | 2025-03-15 |
Compare Two Models
Select two models to compare their performance side-by-side
Select the first model for comparison
Select the second model for comparison
Note on Comparison Results:
- Positive difference values (🟢) indicate Model 1 performed better
- Negative difference values (🔴) indicate Model 2 performed better
- Lower error rates indicate better performance
📊 Model Comparison Results
Select two models and click Compare to see the results. |
Dataset & Methodology
🎯 About the Bambara Speech Recognition Benchmark
📈 Dataset Overview
Our benchmark is built on the sudoping01/bambara-speech-recognition-benchmark dataset, featuring:
- 🎙️ Diverse Audio Samples: Various speakers, dialects, and recording conditions
- 🗣️ Speaker Variety: Multiple native Bambara speakers from different regions
- 🎵 Acoustic Diversity: Different recording environments and quality levels
- ✅ Quality Assurance: Manually validated transcriptions
- 📚 Content Variety: Multiple domains and speaking styles
🔬 Evaluation Methodology
Text Normalization Process
- Lowercase conversion for consistency
- Punctuation removal to focus on linguistic content
- Whitespace normalization for standardized formatting
- Unicode normalization for proper character handling
Quality Controls
- Outlier Detection: Extreme error rates are capped to prevent skewing
- Data Validation: Comprehensive format and completeness checks
- Duplicate Prevention: Automatic detection of duplicate submissions
- Missing Data Handling: Identification of incomplete submissions
🚀 How to Participate
Step 1: Access the Dataset
from datasets import load_dataset
dataset = load_dataset("sudoping01/bambara-speech-recognition-benchmark")
Step 2: Generate Predictions
- Process the audio files with your ASR model
- Generate transcriptions for each audio sample
- Ensure your model outputs text in Bambara language
Step 3: Format Results
Create a CSV file with exactly these columns:
id: Sample identifier (must match dataset IDs)text: Your model's transcription
Step 4: Submit & Evaluate
- Upload your CSV using the submission form
- Your model will be automatically evaluated
- Results appear on the leaderboard immediately
🏆 Recognition & Impact
Top-performing models will be:
- Featured prominently on our leaderboard
- Highlighted in MALIBA-AI communications
- Considered for inclusion in production systems
- Invited to present at community events
🤝 Community Guidelines
- Reproducibility: Please provide model details and methodology
- Fair Play: No data leakage or unfair advantages
- Collaboration: Share insights and learnings with the community
- Attribution: Properly cite the benchmark in publications
📚 Technical Specifications
| Aspect | Details |
|---|---|
| Audio Format | WAV, various sample rates |
| Language | Bambara (bam) |
| Evaluation Metrics | WER, CER, Combined Score |
| Text Encoding | UTF-8 |
| Submission Format | CSV with id, text columns |
📚 Citation
If you use the Bambara ASR Leaderboard for your scientific publication, or if you find the resources useful, please cite our work:
@misc{bambara_asr_leaderboard_2025,
title={Bambara Speech Recognition Leaderboard},
author={MALIBA-AI Team},
year={2025},
url={https://huggingface.co/spaces/MALIBA-AI/bambara-asr-leaderboard},
note={A community initiative for advancing Bambara speech recognition technology}
}
About MALIBA-AI
MALIBA-AI: Empowering Mali's Future Through Community-Driven AI Innovation
"No Malian Language Left Behind"
This leaderboard is maintained by the MALIBA-AI initiative to track progress in Bambara speech recognition technology. For more information, visit MALIBA-AI or our Hugging Face page.