inference

Cross-platform, customizable ML solutions for live and streaming media.

Making large AI models cheaper, faster and more accessible

ncnn is a high-performance neural network inference framework optimized for the mobile platform

Port of OpenAI's Whisper model in C/C++

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.

NVIDIA® TensorRT™, an SDK for high-performance deep learning inference, includes a deep learning inference optimizer and runtime that delivers low latency and high throughput for inference applications.

Plug and play modules to optimize the performances of your AI systems 🚀

💎1MB lightweight face detection model (1MB轻量级人脸检测模型)

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.

Runtime type system for IO decoding/encoding

🎨 The exhaustive Pattern Matching library for TypeScript, with smart type inference.

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

OpenVINO™ Toolkit repository

TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding features, including its cross-platform capability, high performance, model compression and code pruning. Based on ncnn and Rapidnet, TNN further strengthens the support and …

An easy to use PyTorch to TensorRT converter

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

Pre-trained Deep Learning models and demos (high quality and extremely fast)

TypeDB: a strongly-typed database

LightSeq: A High Performance Library for Sequence Processing and Generation