模型推理

0 概述

https://openmlsys.github.io/chapter_model_deployment/model_inference.html

AI模型总的来说,其实就是两块东西,训练和推理

训练:就是利用数据,得到一个模型

推理:就是将训练好的模型应用到实际业务

1 OONX

Open Neural Network Exchange

跨框架的模型中间表达,模型的统一存储形式

2 推理框架

1 libtorch

2 TensorFlow Lite

3 TensorRT

NVIDIA® TensorRT™ is an SDK for optimizing trained deep learning models to enable high-performance inference.

https://docs.nvidia.com/deeplearning/tensorrt/quick-start-guide/index.html

4 NCNN

ncnn is a high-performance neural network inference computing framework optimized for mobile platforms.

5 onnxruntime


:D 一言句子获取中...