[知乎]防止过拟合
1. 获取更多数据
- 从数据源头获取更多数据;
- 根据当前数据集估计数据分布，使用该分布生产更多数据;
- 数据增强;
2. 使用合适的模型
- 2.1 网络结构
- 2.2 训练时间-->Early stopping
- 2.3 权重衰减(Weight-decay)/正则化(Regularization)
- 2.4 增加噪声Noise
- 2.4.1 在输入中加噪声
- 2.4.2 在权重上加噪声
- 2.4.3 对网络的响应加噪声
3. 结合多种模型
- 3.1Bagging
- 3.2Boosting
- 3.3 Dropout
4.贝叶斯方法
综上：
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
- simpler model structure
- regularization
- data augmentation
- dropout
- Bootstrap/Bagging
- ensemble
- early stopping
- utilize invariance
- Bayesian
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%