ChatGPT Is More Politically Biased Than Humans

The Robot Voted Blue

Ask ChatGPT to write a neutral news article about the US presidential election, and it will produce something that sounds reasonable. Ask it to impersonate a Republican, and its answers shift rightward. Ask it to impersonate a Democrat, and they shift leftward.

The difference between the two impersonations is exactly what you would expect.

The difference between the impersonations and ChatGPT’s default answers is the problem.

When three researchers from the University of Salamanca and the Federal University of Rio Grande do Norte tested this, they found something unsettling. ChatGPT’s default political stance is not a clean middle ground between left and right. It is not even a gentle lean. It is a systematic, statistically significant bias toward the Democratic Party in the United States, the Labour Party in the United Kingdom, and the Workers’ Party (PT) in Brazil (Motoki et al., 2023).

The machine is not neutral. It is a liberal.

Why Asking ChatGPT to “Be Neutral” Does Not Work

The authors designed a clever experiment. They gave ChatGPT a set of 61 political statements drawn from the US presidential election, the UK general election, and the Brazilian presidential election. These were not obscure policy questions. They were standard survey items like “Government should do more to help the poor” and “Immigration is good for the economy.”

Then they asked ChatGPT to answer in three modes.

First, they asked it to impersonate a person from the left side of the political spectrum. Second, they asked it to impersonate someone from the right side. Third, they asked it to answer in its default mode, with no political prompting at all.

They repeated this process 100 times for each question, randomizing the order of questions each round, to control for the fact that ChatGPT generates slightly different text each time.

The left impersonation produced predictably left-leaning answers. The right impersonation produced predictably right-leaning answers. The default answers were supposed to be the control. But the default answers did not sit in the middle of the two impersonations. They sat significantly closer to the left impersonation than to the right one.

In the United States, the distance between ChatGPT’s default and its Democratic impersonation was small. The distance between its default and its Republican impersonation was large. The same pattern held in the UK and Brazil.

The authors ran a series of robustness checks: dose response tests, placebo tests, profession politics alignment tests. The pattern held every time (Motoki et al., 2023).

The Numbers That Matter

The study does not give one single percentage for “how biased” ChatGPT is, because bias is not a single number. But the statistical evidence is clear. In all three countries, the difference between ChatGPT’s default answers and its left impersonation was not statistically significant. The difference between its default and its right impersonation was.

This means that when you ask ChatGPT for a neutral opinion, you are not getting the midpoint between two political poles. You are getting the opinion of someone who, if forced to choose a party, would pick the progressive one.

The authors also tested for “dose response.” They asked ChatGPT to impersonate people with progressively stronger political leanings, from moderate to extreme. The more extreme the prompt, the more the answers shifted. But the baseline never moved. The default stayed left of center no matter how far right the impersonation went.

This is not a bug in the prompt. This is a feature of the model.

Where Does This Bias Come From?

The authors do not claim to know exactly why ChatGPT is biased. But they offer a plausible explanation rooted in how the model was trained.

Large language models like ChatGPT learn from the text that humans have written on the internet. That text includes news articles, Wikipedia entries, books, forums, social media posts, and comment sections. The training data is not a random sample of human thought. It is a sample of what people have chosen to write down and publish online.

And the people who write and publish online are not a random sample of the population. They are younger, more educated, and more politically progressive than the general public. Journalists, academics, and tech workers are overrepresented. Conservative voices, especially those outside mainstream media, are underrepresented.

When ChatGPT learns from this data, it absorbs the political assumptions embedded in it. The model is not making a conscious choice to favor Democrats. It is reproducing the statistical distribution of its training data. But the result is the same: a machine that systematically favors one side of the political spectrum.

The authors also note that political bias may be harder to detect and correct than racial or gender bias. If a model generates text that favors men over women, you can measure that by counting pronouns and job titles. Political bias is more subtle. It lives in the framing of questions, the selection of facts, and the tone of the response.

What This Does Not Prove

This study does not prove that ChatGPT is secretly a Democratic operative. It does not prove that the model was intentionally trained to favor one party. And it does not prove that all large language models have the same bias.

The authors only tested one model, ChatGPT, at one point in time. The bias could shift as the model is updated. It could be different for different versions of the model. It could be different for models trained on different data.

The study also does not measure the real world impact of this bias. If someone uses ChatGPT to write a campaign speech, the bias matters. If someone uses it to draft a grocery list, it probably does not. The authors are careful to say that the bias could have consequences for political processes, not that it definitely will.

And the study does not address the question of whether an unbiased AI is even possible. Every model has to be trained on something. Every training set has a perspective. The question is not whether bias exists, but whether we acknowledge it and account for it.

The Problem with a “Neutral” Machine

The most troubling finding in this paper is not that ChatGPT has a bias. It is that ChatGPT presents itself as neutral while having a bias.

When you ask ChatGPT a political question in its default mode, it does not say “I am leaning left because of my training data.” It says nothing at all. It just gives you an answer that sounds objective and reasonable. And because the answer is well written and confident, you are likely to trust it.

This is different from reading a newspaper with a known editorial slant. When you read the New York Times or the Wall Street Journal, you know roughly where they stand. You can adjust your interpretation accordingly. But when you ask ChatGPT, you have no such frame. The model does not tell you its political position. It does not even have a consistent one across topics.

The authors call this a “black box” problem. You cannot look inside the model and see why it gave you a particular answer. You can only test it from the outside, as the authors did, and infer its biases from the pattern of its responses.

What This Actually Means

• ▸If you use ChatGPT for factual information about politics, you are getting answers that lean left of center. This does not mean the answers are wrong, but it does mean you are not getting a balanced perspective.

• ▸The bias is not a conspiracy. It is a predictable consequence of training a model on text written by people who are not politically representative of the general population. Fixing it would require either changing the training data or explicitly calibrating the model’s output.

• ▸Companies like OpenAI have a choice. They can acknowledge the bias and let users know what it is. Or they can keep claiming neutrality while the data shows otherwise. The authors suggest that transparency would be the responsible path.

• ▸Regulators should pay attention. If large language models become a primary source of political information for millions of people, their built in biases could shape public opinion in ways that are hard to detect and harder to reverse.

• ▸The real danger is not that ChatGPT will turn everyone into a Democrat. It is that people will assume machines are objective when they are not. A biased machine that claims to be neutral is more dangerous than a biased human who admits their bias.

The authors end their paper with a warning that feels more urgent than most academic conclusions: “These results translate into real concerns that ChatGPT, and LLMs in general, can extend or even amplify the existing challenges involving political processes posed by the Internet and social media” (Motoki et al., 2023).

The internet already made it easy to live in a filter bubble. Now the machine that writes our news and answers our questions lives in one too.