Comparative Analysis of Ideological Moderation and Bias in LLM Translation of Controversial Texts

This study investigates the presence and nature of political bias in the translation outputs of three state-of-the-art large language models (LLMs) ‒ ChatGPT-4, Claude 3.5 Sonnet, and Gemini 2.0 ‒ when translating between Hebrew and English in both directions. Focusing on politically sensitive terminology within the context of news reporting, the research examines how each model renders key lexical items such as “terrorist,” “Judea and Samaria,” and “West Bank,” using translation accuracy as a proxy for bias detection. The article presents promising preliminary results of the data-driven analysis of 100 politically diverse news excerpts and a comparative evaluation of translation tendencies across language directions. The findings reveal distinct patterns of ideological framing among the LLMs. ChatGPT-4 exhibited the highest rate of left-oriented bias in Hebrew-to-English translations and a notable degree of right-oriented bias in English-to-Hebrew translations. In contrast, Claude 3.5 Sonnet and Gemini 2.0 demonstrated greater consistency and neutrality, with minimal evidence of politically biased behavior. These results support prior research suggesting that LLMs’ outputs are sensitive not only to the prompt language but also to the underlying training data in specific languages and moderation systems. This study contributes to emerging discussions on AI ethics, multilingual fairness, and the role of post-processing in bias mitigation. The comparative framework presented here offers a foundation for evaluating political bias in multilingual LLM applications, particularly in low-resource language settings.