Impact or applications of introducing attention in deep networks modelling multi-agent systems


I have been reading quite a lot about the research progress in the domain of self attention-based neural networks that were introduced by Google Inc. in their paper titled "Attention is all you need".

The concept of introducing attention to neural networks in order to free ourselves from a strict context vector being really unique on one hand and moreover using the same concept to model sequences without recurrent neural networks as introduced in the paper is extremely elegant.

I have been trying to figure out so has to how this concept of attention would aid deep networks that model multi-agent systems in which game-theoretic factors come into play for the network to learn.

I was looking for some direction or a toy example/explanation or even possible previous research done to try to test these concepts together.

P.S - I'm just tinkering with some ideas hoping to build something experimental.

Tuhin Khare

Posted 2020-03-01T06:35:48.837

Reputation: 21

No answers