Kratek povzetek v slovenščini: Ker veliki jezikovni modeli (LLM) postajajo glavni posredniki informacij tudi v neangleško govorečih okoljih, je ocenjevanje njihove politične nevtralnosti v jezikih z manj viri, kot je slovenščina, še vedno ključnega pomena. Kljub temu vrednotenja pogosto temeljijo na enkratnem izpolnjevanju vprašalnikov, pri čemer se ideološka usmerjenost prepleta z artefakti merjenja. Predstavljamo prvo sistematično raziskavo politične pristranskosti in možnosti usmerjanja slovenskih LLM-jev, v kateri na testu Political Compass Test (PCT) ocenjujemo družine modelov Gemma, Qwen, GaMS in Zlatorog.
Povzetek v angleščini: As Large Language Models (LLMs) become primary information intermediaries in non-English contexts, assessing their political neutrality in lower-resource languages like Slovenian remains critical, yet evaluations often rely on single-pass questionnaires that conflate ideology with measurement artifacts. We present the first systematicstudy of political bias and steerability in Slovenian LLMs, evaluating the Gemma, Qwen, GaMS and Zlatorog families on the Political Compass Test (PCT). Using a Latin Hyper-cube Sampling framework across 300 configurations per model and ideological condition, we generate stable, distribution-averaged estimates that are decomposable byfactor. Every model default to the Libertarian-Left. Analysis of “semantic plasticity” — the ability to emulate assigned personas — reveals that while newer iterations of global models are increasingly steerable, smaller and natively adapted models (GaMS, Zlatorog) exhibit behaviour consistent with functional ignorance. This is confirmed by the consistency analysis, showing that for five of the fourteen evaluated models the responses along the economic axis are not far from random. Consequently, their near-center scoring cannot be read as a genuinely centrist political stance. Finally, we show that a single-pass administration of the PCT is an unreliable estimator of political position, given the structural asymmetry of the instrument and the variance introduced by instruction phrasing and other factors.
Avtorja: Luka Debevc, Matej Martinc; Institut “Jožef Stefan”
Prispevek je bil predstavljen na konferenci: LANGUAGE TECHNOLOGIES & DIGITAL HUMANITIES CONFERENCE 2026
Celoten članek je dosegljiv na povezavi (v angleškem jeziku): https://zenodo.org/records/22765800
