#

llm-inference

Here are 188 public repositories matching this topic...

nomic-ai / gpt4all

gpt4all: open-source LLM chatbots that you can run anywhere

Updated Dec 13, 2023
C++

microsoft / autogen

Enable Next-Gen Large Language Model Applications. Join our Discord: https://discord.gg/pAbnFJrkgZ

chat chatbot gpt chat-application agent-based-framework agent-oriented-programming gpt-4 chatgpt llmops gpt-35-turbo llm-agent llm-inference llm-framework

Updated Dec 14, 2023
Jupyter Notebook

bentoml / OpenLLM

Operating LLMs in production

Updated Dec 13, 2023
Python

mistralai / mistral-src

Reference implementation of Mistral AI 7B v0.1 model.

llm llm-inference mistralai

Updated Dec 13, 2023
Jupyter Notebook

openvinotoolkit / openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference

nlp natural-language-processing ai computer-vision deep-learning transformers inference speech-recognition yolo recommendation-system performance-boost good-first-issue openvino diffusion-models stable-diffusion generative-ai llm-inference optimize-ai deploy-ai

Updated Dec 14, 2023
C++

anything-llm

Mintplex-Labs / anything-llm

Open-source ChatGPT equivalent experience for both open and close source LLMs, embedders, and vector databases. Supports unlimited documents, threads, and concurrent users and management all in a very clean UI.

Updated Dec 14, 2023
JavaScript

SuperDuperDB / superduperdb

🔮 SuperDuperDB. Bring AI to your database; integrate, train and manage any AI models and APIs directly with your database and your data.

Updated Dec 14, 2023
Python

deepsparse

neuralmagic / deepsparse

Sparsity-aware deep learning inference runtime for CPUs

nlp performance computer-vision inference machinelearning pruning object-detection pretrained-models quantization cpus onnx sparsification llm-inference deepsparse

Updated Dec 14, 2023
Python

liguodongiot / llm-action

本项目旨在分享大模型相关技术原理以及实战经验。

llm llmops llm-serving llm-training llm-inference

Updated Dec 11, 2023
Jupyter Notebook

liltom-eth / llama2-webui

Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.

llm llm-inference llama2 llama-2

Updated Nov 27, 2023
Jupyter Notebook

InternLM / lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

llama cuda-kernels deepspeed llm fastertransformer llm-inference turbomind internlm llama2 codellama

Updated Dec 14, 2023
C++

intel / intel-extension-for-transformers

⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

cpu gpu chatbot xeon pc habana large-language-model chatpdf llm-inference 4-bits speculative-decoding llm-cpu streamingllm neurips2023 attention-sink intel-optimized-llamacpp neural-chat neural-chat-7b-v3-1 gaudi2

Updated Dec 14, 2023
C++

FasterDecoding / Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

llm llm-inference

Updated Dec 10, 2023
Jupyter Notebook

ray-project / ray-llm

RayLLM - LLMs on Ray

distributed-systems transformers ray serving large-language-models llm llmops llm-serving llm-inference

Updated Dec 6, 2023
Python

llmflows

stoyan-stoyanov / llmflows

LLMFlows - Simple, Explicit and Transparent LLM Apps

python machine-learning ai openai question-answering vector-database gpt-4 llm prompt-engineering llms chatgpt llmops llm-inference

Updated Oct 18, 2023
Python

lean-dojo / LeanCopilot

LLMs as Copilots for Theorem Proving in Lean

machine-learning theorem-proving lean lean4 llm-inference

Updated Dec 13, 2023
C++

predibase / lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

transformers pytorch llama gpt lora model-serving fine-tuning llm llmops llm-serving llm-inference

Updated Dec 14, 2023
Python

FlagAI-Open / Aquila2

The official repo of Aquila2 series proposed by BAAI, including pretrained & chat large language models.

llm llm-training llm-inference

Updated Dec 14, 2023
Python

anarchy-ai / LLM-VM

irresponsible innovation. Try now at https://chat.dev/

machine-learning deep-learning artificial-intelligence distillation distillation-model llm llm-agent llm-training llm-inference llm-local

Updated Dec 14, 2023
Python

ray-project / ray-educational-materials

This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.

deep-learning ray distributed-machine-learning ray-tune ray-train ray-distributed llm generative-ai ray-serve ray-data llm-serving llm-inference

Updated Dec 4, 2023
Jupyter Notebook

Improve this page

Add a description, image, and links to the llm-inference topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the llm-inference topic, visit your repo's landing page and select "manage topics."