So, what’s up with AI’s obsession with Islamic terrorist organizations?
Something weird has been going on with AI Arabic fan-translations.
Players keep noting how they keep mentioning an extremist organization of some sort despite the source never mentioning or even alluding to them.
What’s actually happening here is real simple, and real stupid.
AI - more accurately, Large Language Models (LLMs) - are pretty much fancy autocomplete. There’s no actual, quantifiable intelligence at play at the moment.
LLMs are trained on a wide selection of scraped data to predict, then output, what the most likely next THING (word, pixel, code, etc…) is. That training data then gets baked into the model’s parameters.
Whatever patterns, associations, and biases exist in that data, the model reproduces them while attempting to synthesize responses to the user’s query.
The content of said training data is key here. LLMs are largely trained on online content due to how plentiful and easily scrapable it is.
What dominates online content? News. And MENA news is rarely good news.
While synthesizing answers in Arabic, as is the case when LLMs are asked to translate into the language, they automatically associate terms like “Organization”, “Brotherhood”, etc… with crisis-adjacent contexts.
In contrast, English gets a more varied / less “wrong” output simply because the same terms are used in much more varied contexts in English news.
And that’s how Islamic terrorist organizations transcended lore and canon.
Maybe don’t put too much faith in autocomplete?
By Seif Chaib.
Originally posted on Linkedin.