corpora

A collection of corpora for named entity recognition (NER) and entity recognition tasks. These annotated datasets cover a variety of languages, domains and entity types.

NLTK Data

Data repository for pretrained NLP models and NLP corpora.

微信公众号语料库

Fuzzing resources for feeding various fuzzers with input. 🔧

A collaborative catalog of NLP resources for Indic languages

A web-based engine for creating and annotating textual corpora

Links to Russian corpora + Python functions for loading and parsing

Official source for spanish Language Models and resources made @ BSC-TEMU within the "Plan de las Tecnologías del Lenguaje" (Plan-TL).

The Self-dialogue Corpus - a collection of self-dialogues across music, movies and sports

Open Korean NLP Dataset Curation for the Users All Around the Globe

CrossNER: Evaluating Cross-Domain Named Entity Recognition (AAAI-2021)

Automatic categorization of documents, consists in assigning a category to a text based on the information it contains. We'll follow different approach of Supervised Machine Learning.

Unannotated Spanish 3 Billion Words Corpora

An R package for dynamic exploration of text collections

An advanced, extensible web front-end for the Manatee-open corpus search engine

Named Entity Recognition for biomedical entities

A curated list of resources dedicated to Natural Language Processing (NLP) of Cantonese | 粵語 NLP

Tools for filtering and cleaning parallel and monolingual corpora for machine translation and other natural language processing tasks.

A variety of loaders for various NLP corpora.

Reading the data from OPIEC - an Open Information Extraction corpus

The Data Format for Digital Linguistics (DaFoDiL)

Extract corpora from Wikipedia dumps

Table compiling the list of biomedically-related corpora available for named entity recognition (and some also suitable for association detection). First version has was published as part of the paper: Dieter Galea, Ivan Laponogov, Kirill Veselkov; Exploiting and assessing multi-source data for supervised biomedical named entity recognition, Bioinformatics, bty152, https://doi.org/10.1093/bioinformatics/bty152 . If you would like to add other (or your) corpora, please submit a pull request and I'll happily approve it.

An unofficial Python API that allows users to create a corpus of lyrical text from their favorite artists and billboard charts

repo for Tibetan corpora

The Potsdam Twitter Sentiment Corpus

Dataset containing Semantic Relations and Metadata, for Training and Evaluating Distributional Semantic Models in English and Mandarin Chinese

Textstelle is a collection of corpora for the creation of bots and other things that generate text 🤖

Korpuslinguistik war noch nie so einfach...