1. Introduction

Natural Language Processing (NLP) helps computers understand and process human languages. My project focuses on translating Kinyarwanda into English using a small character-level GPT model. In my notebook, I also compare two versions of the model: one with Layer Normalization and another without it.

Although this model helps us understand how GPT works, improving translation quality may require a pretrained model that has already learned patterns from large amounts of language data. In this blog, I research three models that could help improve our project: Qwen3-4B-Instruct-2507, Llama 3.2 3B Instruct, and OPUS-MT Kinyarwanda-to-English.

  1. Overview of the Models

Model 1: Qwen3-4B-Instruct-2507

Qwen3-4B-Instruct-2507 was developed by the Qwen team. It has approximately 4 billion parameters and a context window of 262,144 tokens. It is released under the Apache 2.0 licence.

This model is useful for instruction following, reasoning, and multilingual text generation. It could be tested on Kinyarwanda-to-English translation by giving it clear instructions about the source language and the required English output. However, it is a general-purpose language model rather than a model specifically designed for this language pair. Its ability to translate Kinyarwanda accurately must therefore be tested.

Model 2: Llama 3.2 3B Instruct

Llama 3.2 3B Instruct was developed by Meta and contains approximately 3 billion parameters. Its context window is up to 128,000 tokens, and it is governed by Meta's Llama 3.2 Community Licence.

This model is designed for conversational tasks, text generation, and instruction following. It could be useful for translating short sentences when prompted appropriately. However, Kinyarwanda is not among the eight languages officially listed as supported in its model card. We would need to evaluate its translation quality carefully before selecting it.

Model 3: OPUS-MT Kinyarwanda-to-English

OPUS-MT is a machine-translation model developed by the Helsinki-NLP research group. The Helsinki-NLP/opus-mt-rw-en model is specifically designed to translate Kinyarwanda (rw) into English (en). It uses a Transformer-based encoder-decoder architecture and is available under the Apache 2.0 licence.

Unlike general-purpose LLMs, OPUS-MT focuses on a particular translation task. Its model card reports evaluation results on datasets including JW300 and Tatoeba, showing that it has been tested for Kinyarwanda-to-English translation. This makes it a strong candidate for our project, although its results on our own examples may differ from the published benchmarks.

  1. Comparison Table

Feature

Qwen3-4B-Instruct-2507

Llama 3.2 3B Instruct

OPUS-MT rw-en

Developer

Qwen team

Meta

Helsinki-NLP

Parameters

4 billion

About 3 billion

Smaller specialised model

Context window

262,144 tokens

Up to 128,000 tokens

Designed for sentence translation; not directly comparable to LLM context windows

Licence

Apache 2.0

Llama Community Licence

Apache 2.0

Main purpose

General language tasks

Conversation and text generation

Kinyarwanda-to-English translation

Translation suitability

Must be tested

Must be tested carefully

Specifically designed for the language pair

Colab suitability

Worth testing with memory-efficient loading

Worth testing on a T4 GPU

Likely the lightest option to run for this task

API key

Not inherently required for local use

Not inherently required for local use

Not inherently required for local use

The parameter count and context window do not alone determine translation quality. We should compare the models using the same test sentences and measure how accurately each preserves the original meaning.

  1. The Model I Choose and Why

After comparing these models, I choose OPUS-MT Kinyarwanda-to-English (Helsinki-NLP/opus-mt-rw-en) as the first model to use in our project.

The main reason is that it is trained specifically for the language pair we need. Our project does not require a model to answer every kind of question; it needs to translate Kinyarwanda sentences into understandable English. A specialised translation model is therefore a practical starting point.

Another reason is accessibility. OPUS-MT is available through Hugging Face and uses the Apache 2.0 licence. It can be loaded with Python and the Transformers library without requiring a paid model API. Its smaller, task-focused architecture should also make it a practical candidate for free Google Colab experiments, although actual memory use and speed should be measured.

To determine whether it improves our project, I will test OPUS-MT using a collection of Kinyarwanda sentences with reliable English translations. I will compare its outputs with those from my existing character-level GPT model, including the versions with and without Layer Normalization. I will examine grammar, word choice, preservation of meaning, and the handling of unfamiliar words.

We can also measure translation quality using an evaluation metric such as BLEU, while manually checking examples because a numerical score cannot fully measure meaning or naturalness. We should record the processing time and memory use as well.

OPUS-MT may not translate every sentence perfectly, especially informal expressions or uncommon words. If the results are not good enough, Qwen3 could be tested as an alternative. However, OPUS-MT is my first choice because its published purpose directly matches our project's objective.

  1. Conclusion

This research shows that different models are designed for different tasks. Qwen3 and Llama 3.2 are general-purpose language models, while OPUS-MT rw-en specialises in Kinyarwanda-to-English translation.

I choose OPUS-MT because it matches the language pair, is accessible for experimentation, and provides a clear baseline for comparison with my existing GPT model. Testing all models on the same examples will help us decide whether the chosen model genuinely improves translation quality.

References

Helsinki-NLP. OPUS-MT Kinyarwanda-to-English Model Card. https://huggingface.co/Helsinki-NLP/opus-mt-rw-en

Qwen Team. Qwen3-4B-Instruct-2507 Model Card. https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507

Meta. Llama 3.2 Model Card. https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md

Hugging Face. Transformers Documentation. https://huggingface.co/docs/transformers/index