《百面深度学习》试读 | 系列三：语言模型的前世今生

＊

导言

《百面深度学习》自上市以来，获得了众多读者的关注与支持，一直高居京东计算机与互联网类新书榜单的前列，大家的热情是我们精益求精的源源动力。为了更好地与大家进行分享与交流，我们从书中节选了几个关注度比较高的“热门“知识点，重新加以整理，内容涵盖推荐系统、计算广告、自然语言处理、计算机视觉、视频处理、生成式对抗网络等领域的相关知识，供大家试读。

“

《百面深度》试读第三篇

[1] BENGIO Y, DUCHARME R, VINCENT P, 等. A neural probabilistic language model[J]. Journal of Machine Learning Research, 2003, 3(Feb): 1137–1155.

[2] MIKOLOV T, SUTSKEVER I, CHEN K, 等. Distributed representations of words and phrases and their compositionality[C]//Advances in Neural Information Processing Systems. 2013: 3111–3119.

[3] PETERS M E, NEUMANN M, IYYER M, 等. Deep contextualized word representations[J]. arXiv preprint arXiv:1802.05365, 2018.

[4] RADFORD A, NARASIMHAN K, SALIMANS T, 等. Improving language understanding by generative pre-training[J]. 2018.

[5] DEVLIN J, CHANG M-W, LEE K, 等. BERT: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.

[6] YANG Z, DAI Z, YANG Y, 等. XLNet: Generalized Autoregressive Pretraining for Language Understanding[C]//Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8-14 December 2019, Vancouver, BC, Canada. 2019: 5754–5764.

[7] LIU Y, OTT M, GOYAL N, 等. RoBERTa: A Robustly Optimized BERT Pretraining Approach[J]. CoRR, 2019, abs/1907.11692.

[8] LAN Z, CHEN M, GOODMAN S, 等. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations[C]//8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.

长按关注Hulu动态

阅读更多《百面》好文

↓ 点击“阅读原文”

跳转购买《百面深度学习》