#

llm-serving

Here are 15 public repositories matching this topic...

ray-project / ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a toolkit of libraries (Ray AIR) for accelerating ML workloads.

Updated Aug 12, 2023
Python

bentoml / OpenLLM

Operating LLMs in production

machine-learning deployment serverless transformers falcon mpt llama alpaca dolly fine-tuning bentoml vicuna ai-applications llm model-inference llmops llm-serving stablelm

Updated Aug 12, 2023
Python

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

inference pytorch transformer gpt model-serving mlops llm llmops llm-serving

Updated Aug 12, 2023
Python

skypilot-org / skypilot

SkyPilot: Run LLMs, AI, and Batch jobs on any cloud. Get maximum savings, highest GPU availability, and managed execution—all with a simple interface.

Updated Aug 12, 2023
Python

ray-project / aviary

Ray Aviary - evaluate multiple LLMs easily

distributed-systems transformers ray serving large-language-models llm llms llmops llm-serving llm-inference

Updated Aug 9, 2023
Python

mosec

mosecorg / mosec

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

python rust machine-learning deep-learning mxnet tensorflow gpu cv pytorch tts hacktoberfest model-serving nerual-network machine-learning-platform jax mlops llm llm-serving

Updated Aug 12, 2023
Python

liguodongiot / llm-action

本项目旨在分享大模型相关技术原理以及实战经验。

llm llmops llm-serving llm-training llm-inference

Updated Aug 13, 2023
Jupyter Notebook

ray-project / ray-educational-materials

This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.

deep-learning ray distributed-machine-learning ray-tune ray-train ray-distributed llm generative-ai ray-serve ray-data llm-serving llm-inference

Updated Aug 11, 2023
Jupyter Notebook

substratusai / substratus

Your cross-cloud AI substrate

kubernetes kubernetes-operator mlops ml-platform llmops llm-serving llm-training llm-inference

Updated Aug 12, 2023
Go

ray-project / llms-in-prod-workshop-2023

Deploy and Scale LLM-based applications

ray anyscale llm llms llm-serving llm-inference

Updated Jun 15, 2023
Jupyter Notebook

ray-project / anyscale-berkeley-ai-hackathon

Ray and Anyscale for UC Berkeley AI Hackathon!

hackathon berkeley-ai ray-distributed anyscale llm llm-serving llm-inference

Updated Jun 17, 2023
Jupyter Notebook

ray-project / llm-application

nlp scalable-machine-learning ray-distributed anyscale llm llm-serving

Updated Jun 14, 2023
Jupyter Notebook

mani-kantap / llm-inference-solutions

A collection of all available inference solutions for the LLMs

llmops llm-serving llm-inference

Updated Jul 24, 2023

LoopGlitch26 / Hinglish-AI-Mentor

Hinglish Chatbot powered by Azure Cognitive Services, Google Translate and Open AI

google azure nlp-machine-learning prompt-engineering llm-serving

Updated Jul 11, 2023
Jupyter Notebook

Stosan / commentator

generative-ai llm-serving

Updated Jul 5, 2023
Python

Improve this page

Add a description, image, and links to the llm-serving topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the llm-serving topic, visit your repo's landing page and select "manage topics."