a follow up on learning models

As an AI enthusiast, I have always been fascinated by the potential of transformers in generating images. Recently, I have been diving deeper into this field, building my code based on the "Attention Is All You Need" paper and making optimizations to my parameters while training. While I still have a lot to learn, I am already seeing promising results with stable diffusion-generated images.

One thing I have noticed is that my models are particularly good at maintaining word order and generating text similar to the input given to them. However, I still need to work on prompting out what I want and testing the weights and long-term knowledge of the models. With their impressive text generation capabilities, I believe a training loop of 50 logit/50 loss could be a good starting point.

Overall, I am excited to continue exploring the possibilities of transformers in AI image generation. While I have not set up a GitHub repository yet, I am committed to sharing my progress and insights with the community in the future. Stay tuned for more updates on this fascinating topic!