pre-trained-language-models

Must-read papers on prompt-based tuning for pre-trained language models.

An Open-Source Framework for Prompt-Learning.

Top2Vec learns jointly embedded topic, document and word vectors.

RoBERTa中文预训练模型: RoBERTa for Chinese

A curated list of NLP resources focused on Transformer networks, attention mechanism, GPT, BERT, ChatGPT, LLMs, and transfer learning.

A novel method to tune language models. Codes and datasets for paper ``GPT understands, too''.

Awesome papers on Language-Model-as-a-Service (LMaaS)

Keyphrase or Keyword Extraction 基于预训练模型的中文关键词抽取方法（论文SIFRank: A New Baseline for Unsupervised Keyphrase Extraction Based on Pre-trained Language Model 的中文版代码）

A PyTorch-based model pruning toolkit for pre-trained language models

The code of our paper "SIFRank: A New Baseline for Unsupervised Keyphrase Extraction Based on Pre-trained Language Model"

Code for the ICLR2022 paper "Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners"

Must-read papers on improving efficiency for pre-trained language models.

We start a company-name recognition task with a small scale and low quality training data, then using skills to enhanced model training speed and predicting performance with least artificial participation. The methods we use involve lite pre-training models such as Albert-small or Electra-small with financial corpus, knowledge of distillation an…

HugNLP is a unified and comprehensive NLP library based on HuggingFace Transformer. Please hugging for NLP now!😊

VaLM: Visually-augmented Language Modeling. ICLR 2023.

Code and datasets for the ICLR2023 paper "Multimodal Analogical Reasoning over Knowledge Graphs. "

Code for EMNLP 2021 main conference paper "Dynamic Knowledge Distillation for Pre-trained Language Models"

Zero-shot Transfer Learning from English to Arabic

Code and pre-trained model for the paper "Molecular Language Model as Multi-task Generator."

SIGIR'22 paper: Axiomatically Regularized Pre-training for Ad hoc Search