autoDataAugmentation
AutoAug
背景：
Data augmentation is a effective technique to increase both the amount and diversity of data by randomly “augmenting” it.
直觉上讲，数据增强教会了模型数据不变性
目标：
we aim to automate the process of finding an effective data augmentation policy for a target dataset.
use Reinforcement Learning as the search algorithm
优点：
-
AutoAugment can be applied directly on the dataset of interest to find the best augmentation policy (AutoAugment-direct)
-
learned policies can be transferred to new datasets (AutoAugment-transfer). that policies found on one task can generalize well across different models and datasets.
- each operation being an image processing function such as translation, rotation, or shearing and the probabilities and magnitudes
- search algorithm to find the best policy such that
the neural network yields the highest validation accuracy
on a target dataset
过程
离散搜索问题
- A controller RNN （RNN控制器）predicts an augmentation policy from the search space
- A child network with a fixed architecture is trained to convergence achieving accuracy R（通过softmax进行决策）
- 奖励信号训练：The reward R will be used with the policy gradient method to update the controller
- the controller generate better policies over time.
两种成分
A search algorithm
At a high level, the search algorithm (implemented as a controller RNN) samples a data augmentation policy S, which has information about
-
what image processing operation to use
- policy S will be used to train a neural network with a fixed architecture, whose validation accuracy R will be sent back to update the controller.
- Since R is not differentiable, the controller will be updated by policy gradient methods.
-
the probability of using the operation in each batch
-
the magnitude of the operation.
A search space
- 包含五种子策略
- 对于小批量中的每一张图像，作者均匀地随机选取一个子策略去生成一张变换后的图像来训练神经网络。
- 每种子策略都包括两个有序的图像运算
- 每种操作包含两个hyperparameters:
- the probability of applying the operation 调用操作的概率
- the magnitude of the operation(除了Invert)
缺点
1、大规模采用这样的方法会增加训练复杂性、加大计算成本；
2、无法根据模型或数据集大小调整正则化强度。