МЕТОДЫ ДОМЕННОЙ АДАПТАЦИИ БОЛЬШИХ ЯЗЫКОВЫХ МОДЕЛЕЙ ДЛЯ ЗАДАЧ МЕДИЦИНЫ

Дмитрий Александрович Скреденас; Ирина Леонидовна Каширина; Дмитрий Андреевич Яковлев

doi:10.17308/sait/1995-5499/2026/1/114-134

Authors

Dmitry A. Skredenas MIRDA – Russian Technological University
Irina L. Kashirina MIRDA – Russian Technological University
Dmitry A. Yakovlev MIRDA – Russian Technological University

DOI:

https://doi.org/10.17308/sait/1995-5499/2026/1/114-134

Keywords:

large language models, domain adaptation, Causal AI, LoRA, RLHF, RAG, catastrophic forgetting, PEFT, question answering

Abstract

This paper examines domain adaptation methods for large language models (LLMs) that balance computational complexity, knowledge transferability, and risk minimization in critical fields such as medicine. Three key approaches are analyzed. LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that uses low-rank matrices and preserves the model’s original parameters. RLHF (Reinforcement Learning from Human Feedback) employs reinforcement learning based on feedback from users and experts to correct model behavior. RAG (Retrieval-Augmented Generation) integrates external knowledge bases into the response generation process, eliminating the need to modify the LLM’s parameters. The use of Causal AI methods to adapt LLM to domain-specific causal relationships was examined, which allows for increased reliability of responses. The comparative analysis yields the following conclusions. LoRA demonstrates modularity and training cost-efficiency by using a small number of parameters compared to the original model’s size but does not support feedback. RLHF enhances response accuracy and safety through dynamic adaptation; however, it requires expert data validation and is susceptible to catastrophic forgetting. RAG reduces the risk of «hallucinations» by providing access to up-to-date data, but its effectiveness depends on the quality of the retrieval algorithms and the completeness of the knowledge base. Combining RAG with fine-tuning methods leads to improved answer accuracy in tasks with dynamically updated information. It is concluded that the most promising direction lies in hybrid solutions that combine parameter-efficient fine-tuning (LoRA) with external retrieval mechanisms (RAG). For medical applications, the explainability and reliability of responses are crucial, making RAG a suitable tool.

Author Biographies

Dmitry A. Skredenas, MIRDA – Russian Technological University

2nd year master’s student
Irina L. Kashirina, MIRDA – Russian Technological University

Doctor of Technical Sciences, Professor
Dmitry A. Yakovlev, MIRDA – Russian Technological University

senior lecturer

References

1. Радюш Д. В. Методы интеграции знаний для разработки вопросно-ответных систем / Д. В. Радюш // Russian Technological Journal. – 2025. – С. 21–43.

2. Han Z. Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey / Z. Han, C. Gao, J. Liu, J. Zhang, S.Q. Zhang // Transactions on Machine Learning Research. – 2024.

3. Hu J. E. LoRA: Low-Rank Adaptation of Large Language Models / J. E. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang [et al.] // arXiv preprint. – 2021. – arXiv:2106.09685.

4. Dettmers T. QLoRA: Efficient Finetuning of Quantized LLMs / T. Dettmers, A. Pagnoni, A. Holtzman, L. Zettlemoyer [et al.] // arXiv preprint. – 2021. – arXiv:2305.14314.

5. Liu S. Y. DoRA: Weight-Decomposed LowRank Adaptation / S. Y. Liu, C. Y. Wang, H. Yin, P. Molchanov, Y. F. Wang, K. T. Cheng, M. H. Chen // arXiv preptint. – 2024. – arXiv:2402.09353.

6. Li D. MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA based Mixture of Experts / D. Li, Y. Ma, N. Wang, Z. Cheng, L. Duan, J. Zuo, C. Yang, M. Tang // arXiv preprint. – 2024. – arXiv:2404.15159.

7. Jiang T. MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning / T. Jiang, S. Huang, S. Luo, Z. Zhang, H. Huang, F. Wei, W. Deng, F. Sun, Q. Zhang, D. Wang, F. Zhuang // arXiv preprint. – 2024. – arXiv:2405.12130.

8. Zhang Q. Adalora: Adaptive budget allocation for parameter-efficient fine-tuning / Q. Zhang, M. Chen, A. Bukharin, N. Karampatziakis, P. He, Y. Cheng, W. Chen, T. Zhao // arXiv preprint. – 2023. – arXiv:2303.10512.

9. Huang C. LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition / C. Huang, Q. Liu, B. Y. Lin, T. Pang, C. Du, M. Lin // arXiv preprint. – 2023. – arXiv:2307.13269.

10. Shi S. ResLoRA: Identity Residual Mapping in Low-Rank Adaption / S. Shi, S. Huang, M. Song, Z. Li, Z. Zhang, H. Huang [et al] // Findings of the Association for Computational Linguistics: ACL 2024. – 2024. – P. 8870–8884.

11. Xia W. Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning / W. Xia, C. Qin, E. Hazan // arXiv preprint. – 2024. – arXiv:2401.04151.

12. Ouyang L. Training language models to follow instructions with human feedback / L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin [et al.] // arXiv preprint. – 2022. – arXiv:2203.02155.

13. Schulman J. Proximal Policy Optimization Algorithms / J. Schulman, F. Wolski, P. Dhariwal, A. Radford, O. Klimov // arXiv preprint. – 2017. – arXiv:1707.06347

14. Kreutzer J. Offline Reinforcement Learning from Human Feedback in Real-World Sequence-to-Sequence Tasks / J. Kreutzer, S. Riezler, C. Lawrence // Proceedings of the 5th Workshop on Structured Prediction for NLP (SPNLP 2021). – 2021. – P. 37–43.

15. Harrison L. RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback / L. Harrison, S. Phatale, H. Mansoor, K. Lu, T. Mesnard, C. Bishop, V. Carbune, A. Rastogi // Proceedings of the 41st International Conference on Machine Learning (ICML’24). – 2024. – P. 26874–26901.

16. Bai Y. Constitutional AI: Harmlessness from AI Feedback / Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones [et al.] // arXiv preprint. – 2022. – arXiv:2212.08073.

17. Fisch A. Robust Preference Optimization through Reward Model Distillation / A. Fisch, J. Eisenstein, V. Zayats, A. Agarwal, A. Beirami, C. Nagpal, P. Shaw, J. Berant // arXiv preprint. – 2024. – arXiv:2405.19316.

18. Rafailov R. Direct Preference Optimization: Your Language Model is Secretly a Reward Model / R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, C. Finn // arXiv preprint. – 2023. – arXiv:2305.18290.

19. Yuan Z. RRHF: Rank Responses to Align Language Models with Human Feedback without tears / Z. Yuan, H. Yuan, C. Tan, W. Wang, S. Huang, F. Huang // arXiv preprint. – 2023. – arXiv:2304.05302.

20. Luo Y. An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-Tuning / Y. Luo, Z. Yang, F. Meng, Y. Li, J. Zhou, Y. Zhang // IEEE Transactions on Audio, Speech and Language Processing. – 2023. – P. 3776–3786.

21. Lewis P. Retrieval-augmented generation for knowledge-intensive NLP tasks / P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. t. Yih, T. Rocktäschel, S. Riedel, D. Kiela // Proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS ‘20). – 2020. – P. 9459–9474.

22. Yan S. Q. Corrective Retrieval Augmented Generation / S. Q. Yan, J.C. Gu, Y. Zhu, Z. H. Ling // arXiv preprint. – 2024. – arXiv:2401.15884.

23. Jiang Z. Active Retrieval Augmented Generation / Z. Jiang, F.F. Xu, L. Gao, Z. Sun, Q. Liu, J. Dwivedi-Yu [et al.] // arXiv preprint. – 2023. – arXiv:2305.06983.

24. Asai A. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection / A. Asai, Z. Wu, Y. Wang, A. Sil, H. Hajishirzi // arXiv preprint. – 2023. arXiv:2310.11511.

25. Chan C. M. RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation / C. Chan, C. Xu, R. Yuan, H. Luo, W. Xue, Y. T. Guo, J. Fu // arXiv preprint. – 2024. – arXiv:2404.00610.

26. Edge D. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. / D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, J. Larson // arXiv preprint. – 2024. – arXiv:2404.16130.

27. Kaushik D. Learning the Difference that Makes a Difference with Counterfactually-Augmented Data / D. Kaushik, E. H. Hovy, Z. C. Lipton // arXiv preprint – 2019. – arXiv:1909.12434.

28. Jin Z. CLADDER: assessing causal reasoning in language models / Z. Jin, Y. Chen, F. Leeb, L. Gresele, O. Kamal, Z. Lyu, K. Blin, F. Gonzalez, M. Kleiman-Weiner, M. Sachan, B. Schölkopf // Proceedings of the 37th International Conference on Neural Information Processing Systems (NIPS ‘23). – 2023. – С. 31038–31065.

29. Jin Z. Can Large Language Models Infer Causation from Correlation? / Z. Jin, J. Liu, Z. Lyu, S. Poff, M. Sachan, R. Mihalcea, M.T. Diab, B. Scholkopf // arXiv preprint. – 2023. – arXiv:2306.05836.

30. Butcher B. Aligning Large Language Models with Counterfactual DPO / B. Butcher // arXiv preprint – 2024. – arXiv:2401.09566.

31. Wang N. CausalRAG: Integrating Causal Graphs into Retrieval-Augmented Generation / N. Wang, X. Han, J. Singh, J. Ma, V. Chaudhary // Findings of the Association for Computational Linguistics: ACL 2025. – 2025. – P. 22680–22693.