您的位置: 首页  >  文章  >  论文笔记：Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction

论文笔记：Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction

分类: 文章 • 2024-01-09 19:50:04

Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction

深度学习中的attention机制：
https://blog.****.net/guohao_zhang/article/details/79540014

机器翻译 RNN encoder-decoder
https://cloud.tencent.com/developer/news/158749

DenseNet
https://blog.****.net/u014380165/article/details/75142664

使用DenseNET

  1. 输入为：
    论文笔记：Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction3D stf f大小是目标单词向量和原单词向量的结合
  2. 中间网络论文笔记：Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction
  3. 输出
    由于三维feature，因此要二维折叠，也就是将s折叠起来，这样可以达到目标向量t，折叠方法用pool。
    折叠之后使用matrix E ∈ R|V|×fL 预测 output vocabulary V，得到
    论文笔记：Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction
    pi是词典v中的第i个。

相关推荐