Pytorch for Beginners #42 | Transformer Model: Implement Decoder

Transformer Model: Implement Decoder In this tutorial, we’ll implement the Decoder module of Seq2Seq Transformer model. First, we'll update the Multiheaded Attention module to accept arguments required for Cross-Attention - required to implement the Decoder. Also, we’ll see that Decoder is nothing but an extension of Encoder itself with added Cross-Attention module which accepts the Output of Encoder as Key and Value. In the next tutorial, we’ll combine the Encoder, and Decoder modules and complete the implementation of our Seq2Seq Transformer model. Stay tuned!! The code used in this tutorial is available here - Multiheaded Attention: https://github.com/makeesyai/makeesy-... Transformer Decoder: https://github.com/makeesyai/makeesy-... #pytorch #tutorial #transformer #decoder #seq2seq