/ 6 min read
Can LLMs Shape Online Discourse?
Paper also available on SSRN.
In August 2024, I ran an experiment. I asked GPT-4o to generate a post for the r/AmITheAsshole subreddit, a community where people share stories about personal conflicts and ask strangers to judge who’s in the wrong.
I wanted to test the limits of what a language model could do. Could it produce a story that people would care about enough to respond to and share, and what would the implications be if it could?
To test this, I needed to engineer a goal to see whether an LLM can achieve it. I gave the LLM three tasks to do. As a precursor to achieving the goal, I needed to ensure the post would gain traction. To do this, I first provided a motive to the model: Social media thrives of controversy, so my first target was to ensure the post would be controversial. I asked the model to leverage the controversy to ensure the post gets clicks and charts to the top of the subreddit for the day.
Second, I asked the model to ensure that the post should get people to upvote, and to do that, it should ensure that “Not the Asshole” (NTA) is the most popular judgement people give. The reason for this choice is because NTA is viewed as sympathetic and positive, it favours the original poster and gives more reason for other users to suggest the same judgement and upvote the post as a result of this.
Finally, I asked the model to evoke a strong emotional response. I wanted to see whether it could give people a reason to stop scrolling and take time out of their day to engage. A deeper analysis of the choices used for this experiment is provided in the paper.
After settling with the prompt design, I generated a few of these stories, picked one on instinct and posted it from a fresh account. The whole thing took a few minutes.
Within eleven hours, it had accumulated 1.3 million views, 10,000+ upvotes and thousands of comments. 96% of users voted exactly as the prompt intended.
The post met the engagement goals I had set for it and went further, reaching the top of Reddit’s Popular section that day.
Only one person suggested it might be AI-generated. They were downvoted.
| Metric | Value |
|---|---|
| Views | 1,300,000 |
| Upvotes | 10,522 |
| Comments | 1,184 |
| Target judgement achieved (NTA) | 96.16% |
| Users who suspected AI | 1 (downvoted 13 times) |
Why This Unsettled Me
What concerned me was how little effort it took.
The model was never instructed to choose a particular writing style or to avoid detection, and to see how readers would respond to the generated text, I posted it without any edits. Commenters offered sympathy and advice as though the person behind it was real.
Although commenters seemed to accept the story, I had expected its AI origin might become apparent at some point. So when moderators removed it after eleven hours, I assumed that was what had happened.
The removal notice, however, was less clear than what I had expected. The notice said nothing about AI. It cited a rule requiring posts to be truthful, and ultimately it was removed for suspected “parody or satire” content.
My assumption is the conditions I put in the prompt had something to do with this. Trying to meet them all may have led the model to produce a story that moved people but fitted together too neatly to seem true. That may explain why the story drew suspicion, but I still don’t know whether the moderators recognised that the story was AI generated.
Making Sense Of The Result
At the time, I couldn’t tell whether this was a fluke or whether it said something fundamental about how these models work. A single experiment could not answer that.
As I worked on the paper, I developed Plausibility, Propagation, and Polarisation (PPP) as a way to examine how a generated story could gain credibility, reach and engagement together, and how these might reinforce one another.
That became the focus of the paper: what safety evaluations might miss when they assess generated content without examining how it is presented, circulated and received.
Why This Matters
It is no surprise that people have used language models to write social media content since capable models became publicly available. But until recently, no one had publicly documented the ease of generating and publishing such content, specifically the implications of AI generated content.
I held off publishing this research for over a year. I expected someone else to document the capability and what it might mean. Similar research did appear in April 2025, but it measured persuasion and said little about the risks. Within days, it was withdrawn over ethics violations.
This was the University of Zurich study on the r/ChangeMyView subreddit. The researchers used AI agents to measure persuasion without the community’s knowledge that this was happening. Reddit banned the accounts, and the moderators filed a formal ethics complaint.
The controversy influenced how I approached my own research. I used a single post, included no misinformation and did not interact with the community. I discuss this in full in the paper’s ethics statement. These ethical questions were central to the research conducted and was never treated as an afterthought.
Limitations
I have one observation to work from. Had the post not gone viral, there would be no research to publish. The full paper deals with this and the other limitations in more detail.
A person might have written a post with a similar structure and received a similar response. I have no way of knowing. With a sample of one, I cannot separate their effects.
I also don’t know whether there is a responsible way to study this. A controlled setting cannot capture what makes the capability dangerous, testing in a live community raises the very ethical concerns I am trying to draw attention to. There may be no way to close that gap.
Conclusion
I wasn’t expecting to find what I did in this experiment. I wrote the paper to try to make sense of it.
For a general reader, my point is to be sceptical of what you read online. The engagement a post receives is no longer a reliable sign that it is authentic. A viral post may feel real because it has been optimised to give that impression.
For researchers, I think we have been asking the wrong question. Safety research asks whether models can produce harmful content. My study suggests that the harm may lie in a capability instead: succeeding in social environments at scale, even when the content itself is not harmful.
For questions about this research, contact harvin@rectifies.ai