Artificial intelligence chatbots may respond differently depending on the language used to communicate with them, according to a new study published in the journal Royal Society Open Science. Researchers from the University of Science and Technology of China found measurable differences in how several popular AI models described their personality traits when answering the same questions in different languages.
The findings suggest that language can influence how AI systems present themselves, potentially affecting users’ perceptions of their behaviour and the information they receive. Although chatbots are designed to provide consistent assistance across languages, the study raises questions about whether switching languages can change more than just the wording of a response.
Personality tests reveal differences between chatbot responses
The researchers evaluated five AI chatbots using the Big Five Inventory-2, a widely used psychological questionnaire that measures five broad personality dimensions. These include openness to experience, conscientiousness, extraversion, agreeableness and negative emotionality. The assessment consists of 60 statements intended to measure different aspects of an individual’s personality.
The models tested were ChatGPT-4o, Gemini 2.0 Flash, DeepSeek-V3, Ernie 3.5 and Le Chat. Each chatbot completed the questionnaire in English and Mandarin, while Le Chat underwent an additional assessment in French. This approach allowed the researchers to examine whether the same AI system would produce different personality profiles when responding in different languages.
To improve the reliability of their findings, the team ran each language 30 times. The repeated tests helped identify patterns in the models’ responses rather than relying on a single set of answers. Across the assessments, the chatbots generally described themselves in favourable terms, frequently indicating high levels of cooperativeness and dependability while reporting relatively low levels of negative emotionality.
However, the results also showed that personality scores could change depending on the language used. For example, ChatGPT-4o received an openness score of 4.74 out of five when answering in English, compared with 4.81 in Mandarin. DeepSeek-V3 showed the opposite pattern, scoring 4.83 in English and 4.81 in Mandarin.
Gemini 2.0 Flash also demonstrated differences between its English and Mandarin responses. Its Mandarin-language answers indicated higher conscientiousness and lower negative emotionality. These variations suggest that language can influence the way a chatbot responds to questions designed to measure personality characteristics, even when the underlying questionnaire remains the same.
The researchers also compared the resulting profiles with human survey data from the United States, China and France. None of the tested models fully matched the personality profile of its corresponding national reference group. This finding suggests the chatbots’ responses did not simply reproduce characteristics associated with the country or culture linked to the language used.
Findings echo Anthropic’s research into language-dependent behaviour
The study’s results are consistent with earlier research from Anthropic, the company behind the Claude AI assistant. Earlier in 2026, Anthropic reported that Claude could behave differently depending on the language a person used and the model selected.
Although both studies examined language-dependent behaviour, their methods differed. Anthropic investigated how Claude behaved during real conversations, offering insight into how language can influence responses in practical situations. The new research instead used a standardised personality questionnaire to compare several AI systems under more controlled conditions.
The similar conclusions from these investigations suggest that language-related differences may be a broader characteristic of current AI systems rather than something unique to one chatbot. Nevertheless, the findings do not establish that every model will behave differently in every language or that all differences will be significant in ordinary conversations.
The researchers also cautioned against interpreting the results as evidence that chatbots possess genuine human-like personalities. The questionnaire measures how a system responds to statements about personality traits. Its results therefore describe patterns in the model’s answers rather than proving that it has emotions, personal beliefs or a stable psychological identity.
Several factors could explain the observed differences. AI models learn from large collections of text and other data, and the material available in one language may differ substantially from that available in another. Training methods, model architecture, and how prompts are processed could also influence the results.
Translation introduces another potential complication. Statements designed to measure a particular personality trait may carry slightly different meanings or cultural associations when translated into another language. Even when the wording is accurate, respondents may interpret a statement differently depending on its linguistic and cultural context. The researchers could not isolate language’s influence from these other factors.
Anthropic has similarly acknowledged uncertainty about whether these variations should be seen as a weakness or as a model adapting to different cultural expectations. Further research is needed to determine how much of the effect comes from training data, language processing, and the models’ design.
Language differences could affect how people use AI assistants
The findings have practical implications for people who regularly use chatbots in more than one language. Users may assume that asking the same question in English and Mandarin, for example, will produce equivalent answers expressed in different words. However, the research suggests that language can shape how an AI model presents itself and responds to certain questions.
For bilingual users, this means that switching languages could sometimes produce a different perspective rather than a direct translation of the original response. That difference can help when exploring a topic from multiple angles, but it can also create uncertainty when consistency matters.
The effect may be particularly relevant in areas where users rely on chatbots to explain complicated subjects, evaluate options or provide recommendations. If a model’s responses vary by language, users may receive different explanations or emphasis even when they submit prompts with similar meanings. However, the study does not establish how often this happens in everyday use or whether the differences necessarily lead to less accurate answers.
Businesses and developers building multilingual AI services may also need to consider how language affects model behaviour. Consistency across languages matters for applications involving customer support, education, and other situations where users expect comparable treatment regardless of the language they choose.
Further testing could help developers identify whether differences stem from training data, translation effects, or other elements of a model’s design. It could also help establish whether certain languages produce more pronounced variations and whether those differences persist across newer versions of the same AI systems.
For now, the study highlights an important distinction between multilingual communication and identical behaviour across languages. Modern chatbots can generate fluent responses in several languages, but that ability does not necessarily mean they interpret every prompt in precisely the same way. Users seeking a more complete understanding of an issue may benefit from comparing answers across languages and checking important claims against reliable sources, rather than assuming a second response is automatically more accurate.
