GitHub 项目简介: TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on N…
README: TensorRT LLM is an open-sourced library for optimizing LLM and Visual Gen inference. It provides state-of-the-art optimizations, including custom kernels for common inference oper…
Release: v1.2.1
README: Beginning with version 1.0, TensorRT LLM has the following deprecation policy
README: TensorRT LLM now supports diffusion models for visual generation