language-model

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

Code and documentation to train Stanford's Alpaca models, and generate the data.

🔍 Haystack is an open source NLP framework to interact with your data using Transformer models and LLMs (GPT-3 and alike). Haystack offers production-ready tools to quickly build ChatGPT-like question answering, semantic search, text generation, and more.

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

NeMo: a toolkit for conversational AI

A PyTorch-based Speech Toolkit

Google AI 2018 BERT pytorch implementation

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

An open source implementation of CLIP.

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard

Lingvo

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

Implementation of BERT that could load official pre-trained models for feature extraction and prediction

A curated list of pretrained sentence and word embedding models

Library to scrape and clean web pages to create massive datasets.

LSTM and QRNN Language Model Toolkit for PyTorch

Russian GPT3 models.

Toolkit for efficient experimentation with Speech Recognition, Text2Speech and NLP