Tokenization
Showing 9 of 9 projects
A Python library for Chinese text segmentation, offering multiple modes, custom dictionaries, and keyword extraction.
A comprehensive Python library for natural language processing, providing modules, datasets, and tutorials for NLP research and development.
A lightweight JavaScript library for natural language processing that transforms text into structured data with a modest, pragmatic approach.
A comprehensive Node.js library offering a wide range of natural language processing facilities.
Fast, state-of-the-art tokenizers for training and tokenization, optimized for both research and production.
A Python NLP library from Stanford for tokenization, sentence segmentation, NER, and dependency parsing across 60+ languages.
A Go library for efficient multilingual text segmentation and NLP, supporting English, Chinese, Japanese, and more.
A spaCy pipeline and models specifically designed for processing scientific and biomedical documents.
A fast, straightforward, reliable tool for performing massive, automated code refactoring using custom Python patterns.
Related Tags
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.