【计算机科学】【2014.05】一种进化的深度学习网络自编码训练方法

本文为美国密苏里大学（作者：SEAN LANDER）的硕士论文，共35页。

自2006年以来，深度学习在监督和非监督学习领域都取得了长足的进步。深度学习的能力已经被证明胜过一般和高度专业化的分类和聚类技术，而对多层感知器的基本概念几乎没有改变。尽管这已经引起了人们对神经网络的兴趣，但在近30年后，此类系统的许多缺点和陷阱仍未得到解决：训练速度、局部极小值和人工测试的超参数。

在本论文中，我们提出使用进化技术来解决这些问题，并提升深度学习网路的整体品质与能力。在输入重建的自动编码器群体的进化过程中，我们能够以隐藏节点的形式为每个自动编码器提取多个特征，根据其输入重建的能力对自动编码器进行评分，最后选择以隐藏节点为染色体的自编码器进行交叉和突变。通过这种方法，我们不仅可以快速找到最佳的抽象特征集，而且还可以优化自动编码器的结构，以匹配所选择的特征。这也允许我们在数据分区和选择方面试验不同的训练方法，大大缩短了大型复杂数据集的总体训练时间。该方法可以快速有效地训练大型数据集，而用户只需进行少量的手动参数选择，从而更快、更准确地创建深度学习网络。

Introduced in 2006, Deep Learning has made large strides in bothsupervised an unsupervised learning. The abilities of Deep Learning have beenshown to beat both generic and highly specialized classification and clusteringtechniques with little change to the underlying concept of a multi-layerperceptron. Though this has caused a resurgence of interest in neural networks,many of the drawbacks and pitfalls of such systems have yet to be addressedafter nearly 30 years: speed of training, local minima and manual testing ofhyper-parameters.Inthis thesis we propose using an evolutionary technique in order to work toward solvingthese issues and increase the overall quality and abilities of Deep Learning Networks.In the evolution of a population of autoencoders for input reconstruction, we areable to abstract multiple features for each autoencoder in the form of hiddennodes, scoring the autoencoders based on their ability to reconstruct theirinput, and finally selecting autoencoders for crossover and mutation withhidden nodes as the chromosome. In this way we are able to not only quicklyfind optimal abstracted feature sets but also optimize the structure of theautoencoder to match the features being selected. This also allows us toexperiment with different training methods in respect to data partitioning and selection,reducing overall training time drastically for large and complex datasets. Thisproposed method allows even large datasets to be trained quickly andefficiently with little manual parameter choice required by the user, leadingto faster, more accurate creation of Deep Learning Networks.

1 引言
2 研究方法
3 性能测试
4 结果
5 讨论
6 结论
7 未来工作展望
8 小结

更多精彩文章请关注公众号：