特定架构和精度组合(如 CPU Native FP16、CPU BF16)存在 bug、不被支持或不推荐使用(评级为 C)。
GPU 支持是可选的,部署时需要根据目标设备确认所需后端是否被覆盖。
如何开始
需通过 CMake 从源码构建或使用预编译二进制文件和工具,难度为中等。
核验依据
GitHub 项目简介: MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
README: MNN is a highly efficient and lightweight deep learning framework. It supports inference and training of deep learning models and has industry-leading performance for inference an…
README: [MNN-LLM](./transformers/README.md) is a large language model runtime solution developed based on the MNN engine. The mission of this project is to deploy LLM models locally on ev…
README: - MNN-Converter: Convert other models to MNN models for inference, such as Tensorflow(lite), Caffe, ONNX, Torchscripts. And do graph optimization to reduce computation.
README: - Optimized for devices, no dependencies, can be easily deployed to mobile devices and a variety of embedded devices. - iOS platform: static library size will full option for armv…