Skip to content
#

ngram

Here are 111 public repositories matching this topic...

yt605155624
yt605155624 commented Jan 6, 2022

目前的多音字使用 pypinyin 或者 g2pM,精度有限,想做一个基于 BERT (或者 ERNIE) 多音字预测模型,简单来说就是假设某语言有 100 个多音字,每个多音字最多有 3 个发音,那么可以在 BERT 后面接 100 个 3 分类器(简单的 fc 层即可),在预测时,找到对应的分类器进行分类即可。
参考论文:
tencent_polyphone.pdf

数据可以用 https://github.com/kakaobrain/g2pM 提供的数据

进阶:多任务的 BERT
![image](https://user-images.githubusercontent.com/24568452

Colibri core is an NLP tool as well as a C++ and Python library for working with basic linguistic constructions such as n-grams and skipgrams (i.e patterns with one or more gaps, either of fixed or dynamic size) in a quick and memory-efficient way. At the core is the tool ``colibri-patternmodeller`` whi ch allows you to build, view, manipulate and query pattern models.

  • Updated May 6, 2020
  • C++

A fuzzy matching string distance library for Scala and Java that includes Levenshtein distance, Jaro distance, Jaro-Winkler distance, Dice coefficient, N-Gram similarity, Cosine similarity, Jaccard similarity, Longest common subsequence, Hamming distance, and more..

  • Updated Jan 7, 2022
  • Scala

Improve this page

Add a description, image, and links to the ngram topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the ngram topic, visit your repo's landing page and select "manage topics."

Learn more