多任务学习中经常可以看到

参考

https://blog.nowcoder.net/n/600d5e96508d4b818009227fdc9b8cbc

https://zhuanlan.zhihu.com/p/138107999

https://blog.csdn.net/txwh0820/article/details/51791739

1 Gather function not implemented for CPU tensors

多卡训练时候，net的forward里面存在Tensor变成其它类型的操作，比如变成numpy，list

解决：改成Tensor操作

2 RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

https://blog.csdn.net/weixin_41990278/article/details/90311313

https://blog.csdn.net/wu_xin1/article/details/116502378

1 A Survey on Open Set Recognition

https://arxiv.org/abs/2109.00893

2 Open-Set Recognition: A Good Closed-Set Classifier is All You Need

https://arxiv.org/abs/2110.06207

3 Recent Advances in Open Set Recognition: A Survey

https://arxiv.org/abs/1811.08581

https://blog.csdn.net/gostman/article/details/107456597

https://zhuanlan.zhihu.com/p/37102973

随着TF2.0出现，TF也有动态图了

模型，数据需要指定设备

单机单卡

单机多卡

https://blog.csdn.net/weixin_38208912/article/details/105122668

https://blog.csdn.net/leviopku/article/details/109318226

https://blog.csdn.net/weixin_43750248/article/details/115698356

https://zhuanlan.zhihu.com/p/74792767

多机多卡

NO. there is no string tensor so you cannot directly convert string to tensor

https://zhuanlan.zhihu.com/p/113734249

当学习率过大的时候会导致模型难以收敛，过小的时候会收敛速度过慢，合理的学习率才能让模型收敛到最小点而非局部最优点或鞍点

经验值： 0.01 ~ 0.001

学习率衰减

原因：起初距离目标偏离大，可以设置较大，为了快速收敛，后续逐渐靠近目标，需要精细化一点，所以希望值小一点

分类

1.轮数衰减

每经过k个epochs后学习率减半

2.指数衰减

$$\alpha_t=decay\_rate^{epoch}*\alpha_{t-1}$$

3.分数衰减

$$\alpha_t=\frac{\alpha_{t-1}}{1+decay\_rate*epoch}$$

参考

https://blog.csdn.net/LiuPeiP_VIPL/article/details/119581343