In the previous article, we handled values in encoder-decoder attention, now we will simplify the diagram a bit add another set of residual connections.
This allows the encoder–decoder attention to focus on the relationships between the output words and the input words, without needing to preserve the self-attention and positional encoding from earlier.
Lastly, we need a way to take these two values that represent the token in the decoder and select one of the four output tokens: ir, vamos, y, or <EOS>.
To do this, we pass these two values through a fully connected layer.
We will explore this further in the next article.
Looking for an easier way to install tools, libraries, or entire repositories?
Try Installerpedia: a community-driven, structured installation platform that lets you install almost anything with minimal hassle and clear, reliable guidance.
Just run:
ipm install repo-name
… and you’re done! 🚀
Top comments (0)