Triton Inference Server

Repositories

• python_backend

  Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.
  C++ BSD-3-Clause 13 44 0 1 Updated May 15, 2021

• server

  The Triton Inference Server provides an optimized cloud and edge inferencing solution.

  machine-learning cloud deep-learning gpu inference edge datacenter
  C++ 500 2,210 88 10 Updated May 15, 2021

• core
  C BSD-3-Clause 5 5 0 4 Updated May 15, 2021

• client

  Triton Python and C++ client libraries and example, and client examples for go, java and scala.
  C++ BSD-3-Clause 5 8 0 2 Updated May 14, 2021

• identity_backend

  Example Triton backend that demonstrates most of the Triton Backend API.
  C++ BSD-3-Clause 4 1 0 1 Updated May 14, 2021

• model_analyzer

  Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

  deep-learning gpu inference performance-analysis
  Python Apache-2.0 17 52 3 2 Updated May 14, 2021

• fil_backend

  FIL backend for the Triton Inference Server
  Cuda Apache-2.0 1 5 13 3 Updated May 14, 2021

• openvino_backend

  OpenVINO backend for Triton.
  C++ BSD-3-Clause 0 2 0 1 Updated May 14, 2021

• tensorrt_backend

  The Triton backend for TensorRT.
  C++ BSD-3-Clause 0 0 0 0 Updated May 14, 2021

• backend

  Common source, scripts and utilities for creating Triton backends.
  C++ BSD-3-Clause 9 31 0 0 Updated May 14, 2021

• common

  Common source, scripts and utilities shared across all Triton repositories.
  C++ BSD-3-Clause 6 2 0 0 Updated May 14, 2021

• dali_backend

  The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.

  python deep-learning gpu image-processing dali data-preprocessing nvidia-dali
  C++ MIT 5 34 4 2 Updated May 13, 2021

• pytorch_backend

  The Triton backend for the PyTorch TorchScript models.
  C++ BSD-3-Clause 5 13 0 0 Updated May 12, 2021

• onnxruntime_backend

  The Triton backend for the ONNX Runtime.

  backend inference triton-inference-server onnx-runtime
  C++ BSD-3-Clause 9 11 12 0 Updated May 11, 2021

• third_party

  Third-party source packages that are modified for use in Triton.
  C BSD-3-Clause 1 0 0 0 Updated May 7, 2021

• tensorflow_backend

  The Triton backend for TensorFlow 1 and TensorFlow 2.
  C++ BSD-3-Clause 4 12 0 1 Updated May 7, 2021

• square_backend

  Simple Triton backend used for testing.
  C++ BSD-3-Clause 0 0 0 0 Updated May 7, 2021

• repeat_backend

  An example Triton backend that demonstrates sending zero, one, or multiple responses for each request.
  C++ BSD-3-Clause 1 0 0 0 Updated May 7, 2021

• checksum_repository_agent

  The Triton repository agent that verifies model checksums.
  C++ BSD-3-Clause 0 0 0 0 Updated May 7, 2021

• hugectr_backend
  C++ BSD-3-Clause 2 8 0 1 Updated May 1, 2021

• model_navigator

  The Triton Model Navigator is a tool that provides the ability to automate the process of model deployment on the Triton Inference Server.

  deep-learning gpu inference
  Python 1 9 0 0 Updated Apr 13, 2021

• fastertransformer_backend
  C++ BSD-3-Clause 0 7 0 0 Updated Apr 12, 2021

• contrib

  Community contributions to Triton that are not officially supported or maintained by the Triton project.
  BSD-3-Clause 2 3 0 1 Updated Nov 4, 2020