模型推理
0 概述
https://openmlsys.github.io/chapter_model_deployment/model_inference.html
AI模型总的来说，其实就是两块东西，训练和推理
训练：就是利用数据，得到一个模型
推理：就是将训练好的模型应用到实际业务
1 OONX
Open Neural Network Exchange
跨框架的模型中间表达，模型的统一存储形式
2 推理框架
1 libtorch
2 TensorFlow Lite
3 TensorRT
NVIDIA® TensorRT™ is an SDK for optimizing trained deep learning models to enable high-performance inference.
https://docs.nvidia.com/deeplearning/tensorrt/quick-start-guide/index.html
4 NCNN
ncnn is a high-performance neural network inference computing framework optimized for mobile platforms.