Complexity of Symbolic Representation in Working Memory of Transformer Correlates with the Complexity of a Task
Alsu Sagirova, Mikhail Burtsev
The addition of symbolic working memory to Transformer decoders improves machine translation by storing key concepts, with memory content correlating to text complexity.