Parameter-Efficient Fine-Tuning of Large Language Models for Low-Resource Multilingual and Domain-Specific Applications

Abstract

Adapting large language models (LLMs) to underrepresented languages and specialized professional domains is constrained by annotation scarcity, computational expense, and linguistic mismatch. This paper proposes a modular multilingual domain-adaptation framework that combines quantized low-rank adaptation, language-conditioned adapter routing, balanced multilingual sampling, terminology preservation, and domain-aware evaluation. A formal optimization objective integrates supervised task loss, cross-lingual consistency, and a regularization term that limits deviation from a frozen backbone. The paper develops a reproducible experimental protocol for multilingual question answering, classification, and terminology-sensitive generation. To illustrate the intended analysis, a clearly labeled synthetic example compares full fine-tuning, conventional LoRA, QLoRA, and the proposed modular method in terms of macro-F1, exact match, memory, and trainable parameters. These constructed values are not measured findings. The contribution is a testable research design that prioritizes language equity, reproducibility, and practical deployment under limited hardware budgets.