| ||||
| ||||
![]() Title:Robustness of Large Language Models to Meaning-Preserving Perturbations in Ukrainian Conference:ICTERI-2026 Tags:adversarial evaluation, meaning-preserving perturbations, paraphrasing and Ukrainian NLP Abstract: Large language models (LLMs) achieve strong performance on standard benchmarks but may show unstable behavior under small meaning-preserving input changes. This issue is particularly challenging for Ukrainian, a morphologically rich and relatively low-resource language, where synonym substitution must preserve word sense, inflection, and fluency. We evaluate the robustness of LLMs to Ukrainian synonym-substitution attacks using transferred adversarial examples generated against fine-tuned encoder models. Experiments on three classification tasks---sentiment analysis, news classification, and manipulation detection---show that such perturbations frequently transfer to LLMs, especially for sentiment and manipulation tasks. However, manual analysis of 300 adversarial failures reveals that standard robustness metrics substantially overestimate true brittleness: only 19\% of failures correspond to strict meaning-preserving prediction flips, while many are explained by semantic drift, grammatical errors, or label ambiguity. To improve perturbation quality, we introduce a WSD-based filtering method that increases the share of semantically meaningful perturbations. Our results suggest that the evaluated LLMs are sensitive to Ukrainian lexical variation. Overall, we provide a robustness evaluation focused on Ukrainian that highlights how lexical variation, morphology, and perturbation validity affect LLM behavior under synonym substitution. Robustness of Large Language Models to Meaning-Preserving Perturbations in Ukrainian ![]() Robustness of Large Language Models to Meaning-Preserving Perturbations in Ukrainian | ||||
| Copyright © 2002 – 2026 EasyChair |
