NLP-progress
A community-driven repository tracking datasets and state-of-the-art results for common NLP tasks across multiple languages.
What is NLP-progress?
NLP-progress is a community-maintained repository that systematically tracks the state-of-the-art in Natural Language Processing. It aggregates benchmark datasets and model performance across dozens of NLP tasks, providing a quick reference for researchers to understand current capabilities and gaps. The project serves as a living document of the field's evolution.
NLP researchers, machine learning engineers, and students who need to benchmark models, identify research opportunities, or survey current SOTA results for specific tasks.
It offers a uniquely comprehensive and structured overview of NLP progress across many tasks and languages, all in one open, community-updated location. Unlike scattered papers or leaderboards, it provides a centralized, easily navigable resource.
Overview
Repository to track the progress in Natural Language Processing (NLP), including the datasets and the current state-of-the-art for the most common NLP tasks.
Use Cases
Best For
- Finding benchmark datasets for a specific NLP task
- Checking the current state-of-the-art model for sentiment analysis or machine translation
- Discovering NLP tasks and datasets in non-English languages
- Identifying open research problems by seeing which tasks have lower performance ceilings
- Quickly surveying the NLP landscape before starting a new research project
- Contributing new SOTA results to a community-recognized resource
Not Ideal For
- Teams needing real-time, interactive leaderboards for live model comparison and dynamic updates
- Developers seeking hands-on tutorials, code implementations, or model deployment guidance
- Organizations requiring integrated, commercial-grade benchmarking tools with automated validation and support
- Projects dependent on absolutely current data for time-sensitive decisions or regulatory compliance
Pros & Cons
Pros
Tracks over 40 core NLP tasks from machine translation to sentiment analysis, providing a one-stop reference for diverse research needs, as listed in the detailed table of contents.
Includes progress for languages beyond English like Chinese, French, and Vietnamese, aiding global NLP initiatives and reducing English-centric bias in benchmarking.
Built on GitHub with clear contribution guidelines, allowing the community to submit updates via pull requests, ensuring the resource evolves with the field.
Provides machine-readable JSON exports of all tasks and SOTA tables, enabling easy data extraction for meta-analyses or integration into research workflows.
Prefers results from published papers with exceptions for influential preprints, enhancing credibility and making it a reliable academic reference.
Cons
Relies on manual pull requests for updates, causing delays in reflecting the latest SOTA or fast-moving research breakthroughs, unlike automated leaderboards.
Coverage varies by language and task; for example, English has extensive entries while some languages like Nepali have only one task, limiting utility for low-resource NLP.
No built-in verification process for submitted results, relying on community oversight which can lead to errors or biased data persisting without correction.
Markdown-based tables lack interactive features like filtering or visualization, making it less engaging and harder to navigate than dynamic platforms like Papers with Code.
Frequently Asked Questions
Related Projects
Awesome NLP:book: A curated list of resources dedicated to Natural Language Processing (NLP)
awesome-chinese-nlpA curated list of resources for Chinese NLP 中文自然语言处理相关资料
nlp-datasetsAlphabetical list of free/public domain datasets with text data for use in Natural Language Processing (NLP)
awesome Information RetrievalA curated list of awesome information retrieval resources
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.