My final bachelor's project is dedicated to studying multi-language models based on the transformer architecture. [IN PROGRESS]