Gated Neural Networks

Gating is a key feature in modern neural networks including LSTMs, GRUs and
sparsely-gated deep neural networks. The backbone of such gated networks is
a mixture-of-experts layer, where several experts make regression decisions and
gating controls how to weigh the decisions in an input-dependent manner.

Learn more

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.