Getting More from Small LLMs and Limited Data: Fuzzy-Match Retrieval and Candidate Scoring for Multitask Sorbian NLP

WMT 2026 (joint winner, Sorbian track)

Machine Translation
Low-resource
A QLoRA-fine-tuned Qwen3.5-2B with fuzzy-match retrieval and candidate scoring, joint winner of the WMT 2026 Sorbian multitask track.
Authors

Thomas Moerman

Arda Tezcan

Published

October 28, 2026

One 2B model serves five Sorbian tasks
One 2B model serves five Sorbian tasks
Fuzzy matches by stage, synthetic task data, scoring instead of writing
Fuzzy matches by stage, synthetic task data, scoring instead of writing
Official results, 22 rank points, first or second on every task
Official results, 22 rank points, first or second on every task

LT3’s submission to the WMT 2026 shared task on Multitask LLMs with Limited Resources for Upper Sorbian (hsb) and Lower Sorbian (dsb). A single QLoRA-fine-tuned Qwen3.5-2B model handles machine translation, question answering, spell checking, grammar checking, and maths reasoning, combining fuzzy-match retrieval (over authentic and back-translated pools), synthetic task data, and a scoring-based inference system. The primary system was the joint winner of the Sorbian track, ranking first or second across all five tasks.

QR code for the paper PDF in the WMT 2026 proceedingsPaperWMT 2026 proceedings QR code for the code on GitHubCodeGitHub QR code for the models on Hugging FaceModelsHugging Face QR code for the dataset on Hugging FaceDatasetHugging Face

Research theme: Machine translation

Citation

BibTeX citation:
@inproceedings{moerman2026sorbian,
  author = {Moerman, Thomas and Tezcan, Arda},
  publisher = {Association for Computational Linguistics},
  title = {Getting {More} from {Small} {LLMs} and {Limited} {Data:}
    {Fuzzy-Match} {Retrieval} and {Candidate} {Scoring} for {Multitask}
    {Sorbian} {NLP}},
  booktitle = {Proceedings of the Eleventh Conference on Machine
    Translation (WMT)},
  pages = {2667-2680},
  date = {2026-10-28},
  url = {https://www2.statmt.org/wmt26/pdf/2026.wmt-1.210.pdf},
  langid = {en}
}
For attribution, please cite this work as:
Moerman, Thomas, and Arda Tezcan. 2026. “Getting More from Small LLMs and Limited Data: Fuzzy-Match Retrieval and Candidate Scoring for Multitask Sorbian NLP.” Proceedings of the Eleventh Conference on Machine Translation (WMT), October 28, 2667–80. https://www2.statmt.org/wmt26/pdf/2026.wmt-1.210.pdf.