This week we'll explore sequence-to-sequence models in natural language processing (NLP), beginning with recurrent neural network (RNN)-based architectures and the introduction of attention mechanisms for improved alignment in tasks like machine translation. The module also covers best practices for training neural networks, including regularization, optimization strategies, and efficient model training. At the end of the week, you will gain practical experience in implementing and training sequence-to-sequence models.

This week explores transfer learning techniques in NLP, focusing on pretraining, finetuning, and multilingual models. You will first examine the role of pretrained language models like GPT, GPT-2, and BERT, and their challenges. We then explore multitask training and data augmentation, highlighting strategies like parameter sharing and loss weighting to improve model generalization across tasks. Finally, you will dive into crosslingual transfer learning, exploring methods like translate-train vs. translate-test, as well as zero-shot, one-shot, and few-shot learning for multilingual NLP.

This final week introduces large language models (LLMs) and how they can be effectively used through techniques like prompt engineering, in-context learning, and parameter-efficient finetuning. You will explore language-and-vision models, understanding how multimodal architectures extend beyond text to integrate visual and other data modalities. We will also examine non-functional properties of LLMs, including challenges such as hallucinations, fairness, resource efficiency, privacy, and interpretability.