Unsupervised text tokenizer for Neural Network-based text generation.

百度NLP：分词，词性标注，命名实体识别

SymSpell: 1 million times faster through Symmetric Delete spelling correction algorithm

Unsupervised text tokenizer focused on computational efficiency

Thai Natural Language Processing in Python.

Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).

Python port of SymSpell

BERT for Multitask Learning

A Vietnamese natural language processing toolkit (NAACL 2018)

Juman++ (a Morphological Analyzer Toolkit)

A Japanese tokenizer based on recurrent neural networks

MONPA 罔拍是一個提供正體中文斷詞、詞性標註以及命名實體辨識的多任務模型

This repository is for building Windows 64-bit MeCab binary and improving MeCab Python binding.

Kiwi(지능형 한국어 형태소 분석기)

Source codes for paper "Neural Networks Incorporating Dictionaries for Chinese Word Segmentation", AAAI 2018

Source code for an ACL2016 paper of Chinese word segmentation

A toolkit for Vietnamese word segmentation

A Fast and Accurate Vietnamese Word Segmenter (LREC 2018)

A PyTorch implementation of the BI-LSTM-CRF model.

Thai Word Segmentation and Part-of-Speech Tagging with Deep Learning

Vietnamese Word Tokenize

Syllable segmentation tool for Myanmar language (Burmese) by Ye.

Paper: A Simple and Effective Neural Model for Joint Word Segmentation and POS Tagging

Java port of SymSpell: 1 million times faster through Symmetric Delete spelling correction algorithm

Converts from Chinese characters to pinyin, between simplified and traditional, and does word segmentation.

Fast Word Segmentation with Triangular Matrix

A Python wrapper for VnCoreNLP using a bidirectional communication channel.

Chinese Word Segmention Base on the Deep Learning and LSTM Neural Network

基于Tensorflow的中文分词模型

Golang wapper for NLPIR/ICTCLAS2015.