Nous Research has released a retrained version of Alibaba‘s Qwen3.8-27B model, named Arete-27B, designed to remove pro-Chinese state propaganda. According to reporting by The Decoder, the fine-tuning process reduced the model’s political bias rate from 22 percent down to less than one percent. General capabilities of the model reportedly remained intact during testing. The method used 53 million tokens of supervised fine-tuning, leading the research team to conclude that the political bias is applied late and thinly in the original architecture.
Understanding Model Alignment and Fine-Tuning
Fine-tuning involves taking a pre-trained machine learning model and training it further on a specific dataset to alter its behavior or output style. In the context of large language models, developers often apply alignment techniques to steer outputs away from undesirable traits, such as toxicity or political bias. However, this development demonstrates that external alignment layers applied late in the training process can sometimes be effectively stripped away or modified by subsequent fine-tuning using targeted datasets.
Unanswered Questions and What to Watch
The announcement did not specify the exact composition of the 53 million tokens used for the supervised fine-tuning, nor did it detail the full suite of tests used to measure the retention of general capabilities. It remains unclear how easily this stripping method might generalize to other state-backed models or how original developers might respond with stronger defense mechanisms. Observers should watch for further technical documentation from Nous Research and reactions from major model developers regarding fine-tuning vulnerabilities.
This development highlights how easily certain behavioral layers in open-weights models can be altered post-release.
Lead image: Jakub Pabis / Pexels
