TY - GEN
T1 - A novel design for a gated recurrent network with attentional memories
AU - Sun, Libin
AU - Biao, Gao
AU - Shi, Haobin
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Recurrent neural networks (RNN) are used for pro-cessing the temporal dynamics of complex sequential information. Existing recurrent neural networks, such as Long Short-Term Memory (LSTM) and its variant, Gated Recurrent Unit (GRU), are used to control current and historical information. This study proposes a novel design for a gated recurrent neural network with an attention mechanism, named the Attentional Memory Unit (AMU), which gives the recurrent neural network an attention capability. This design uses a gated mechanism and a generalized attention mechanism so that it can be adaptively adjusted in the spatial and temporal sense to allow the application of the attention mechanism for different scenarios. The experimental results verify that the proposed AMU significantly boosts the power of RNNs.
AB - Recurrent neural networks (RNN) are used for pro-cessing the temporal dynamics of complex sequential information. Existing recurrent neural networks, such as Long Short-Term Memory (LSTM) and its variant, Gated Recurrent Unit (GRU), are used to control current and historical information. This study proposes a novel design for a gated recurrent neural network with an attention mechanism, named the Attentional Memory Unit (AMU), which gives the recurrent neural network an attention capability. This design uses a gated mechanism and a generalized attention mechanism so that it can be adaptively adjusted in the spatial and temporal sense to allow the application of the attention mechanism for different scenarios. The experimental results verify that the proposed AMU significantly boosts the power of RNNs.
KW - Attention Mecha-nism
KW - Recurrent Neural Network
UR - http://www.scopus.com/inward/record.url?scp=85137177009&partnerID=8YFLogxK
U2 - 10.1109/ICCEAI55464.2022.00114
DO - 10.1109/ICCEAI55464.2022.00114
M3 - 会议稿件
AN - SCOPUS:85137177009
T3 - Proceedings - 2022 International Conference on Computer Engineering and Artificial Intelligence, ICCEAI 2022
SP - 522
EP - 527
BT - Proceedings - 2022 International Conference on Computer Engineering and Artificial Intelligence, ICCEAI 2022
A2 - Lin, Pan
A2 - Yang, Yong
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2022 International Conference on Computer Engineering and Artificial Intelligence, ICCEAI 2022
Y2 - 22 July 2022 through 24 July 2022
ER -