I started to consider a theory while reporting on accounts that used AI-generated videos with young girls’ likenesses. During my research, I created multiple burner accounts and “seeded” their recommendation algorithms by interacting with AI accounts. Instagram’s Home page would recommend comparable AI-generated videos from similar accounts.
The Riddance team has used this technique on multiple investigations since, and a pattern is forming. In specific content categories where AI-generated accounts cluster, recommendation algorithms appear to have a latent ability to detect AI-generated videos. We can take advantage of this with what I call the recommendation feedback loop.
Specifically, Instagram’s recommendation algorithm groups AI content together, prioritizes AI content once it figures out you want more of it, and keeps serving it without any explicit AI-generated tagging or metadata.
I will explain our anecdotal evidence thoroughly to make the case that this hypothesis deserves serious research. This technique could be a powerful tool for both platforms and researchers trying to label or remove massive amounts of inauthentic content.
Seeding an account
Riddance focuses on finding inauthentic accounts, not individual AI-generated videos. Even though users primarily experience videos, we treat videos as downstream from accounts. Most platforms also work this way, penalizing accounts for inauthentic content instead of striking individual posts. I’m not claiming that recommendation algorithms can tell whether individual videos are AI-generated, but that they can reliably surface videos from inauthentic accounts when you point them in the right direction.
We can do this by seeding a burner account with content from confirmed inauthentic accounts. We’ve written about this technique before, but here’s how it works in practice.
First, I collect AI-generated videos from accounts that are part of a bigger trend. Followers and subscribers send me dozens of videos every week. Recently, I’ve had an alarming number of AI-generated health influencers sent my way. So, I pulled together ten of these accounts to use as seeds.
Then I created a burner account so I wouldn’t sacrifice my own algorithm. I used a proxy network to mask my IP address to keep the recommendations clean later on. I logged into the burner account and engaged with the seed accounts for a couple of minutes each. I watched their videos, liked and saved their videos, and followed the accounts. I’m trying to signal to Instagram that I really love AI-generated health influencers.
When I navigate to Instagram’s Home page, the recommendations are already onto the scent. I see more videos from accounts I followed and suggested AI videos from similar inauthentic accounts. If I only engage with the new inauthentic content, the loop tightens. The algorithm keeps the real creators in the mix, but it pushes them below the tenth recommendation. The top of the feed becomes exclusively inauthentic.
I used to think this was an anomaly, but we’ve repeated it on many other investigations. I’ve talked to other researchers who use a version of this technique, from those who study AI-generated kids videos on YouTube to another creator who investigates health influencers on Instagram.
I wanted to see if the loop was reliable enough to scale with simple automation. I tried to use a custom computer-use AI agent which only interacted with the top recommended posts. While clicking through the feed, the agent went too far down the feed, following real health creators by mistake. Instagram started surfacing other real creators inside the first ten posts, so I had to take control to readjust the algorithm. Ironically, this failure made me more confident in the hypothesis.
Even without automation, we’ve identified over 500 AI-generated health influencer accounts on Instagram alone, many with hundreds of thousands of followers, and the feedback loop keeps serving more. This was done in less than a day of work, by one human analyst working a single burner account.
A quick note on the term “recommendation feedback loop”: In machine learning, this usually describes the way a recommender drifts when it retrains on its own outputs. I’m using this term for something more deliberate: a researcher repeatedly engaging with one kind of content until the algorithm orients its top recommendations around that category.
What is the algorithm doing?
The obvious explanation is that Instagram is directly detecting AI metadata. For example, the C2PA standard is embedded by video models like those from OpenAI and Google. When a platform sees that metadata, it may apply a label (“Contains AI generated media” on TikTok, “AI Info” on Instagram) whether or not the creator disclosed it. So, maybe Instagram’s algorithm is just reading those tags and recommending more of that signal.
The flaw with this explanation is that we’ve seen the loop reinforce content that doesn’t carry any clear AI indicators. Sexualized AI videos come from open-source video models without C2PA data. A lot of the reposted, edited AI content circulating around the war in Iran doesn’t have these indicators, either. I haven’t tested these categories as thoroughly as others, but I’ve created recommendation feedback loops on both sexualized AI videos and inauthentic war footage. Something else is going on.
Here’s where I have to hand-wave. Recommendation algorithms balance between exploration and exploitation. Either the algorithm shows something that is likely to engage you (exploitation), or it shows you something new to test your preferences (exploration). Instagram’s Home page is leaning “exploitation-heavy” at the top of the feed but “exploration-heavy” farther down the feed. The top of the feed seems locked onto my preferences.
The deeper question is what the algorithm is locking onto. Modern recommendation algorithms represent videos as embeddings, which are vectors that pack together signals like video format, transcripts, captions, posting cadence, and even the engagement patterns of other users. So, if AI-generated health videos share distinct observable features, they may end up in their own region of the embedding space whether or not they were labeled as “AI”.
Whether this all combines into a latent AI vector, or it’s simply “the algorithm accidentally finding a bunch of AI content”, the effect is the same. The algorithm surfaces AI content without being asked to detect it.
I’m not claiming that this works for every category, for every platform, or for every kind of AI content. In fact, I’ve been unsuccessful when trying to create a recommendation feedback loop on TikTok, which has a very exploratory algorithm. But after observing this pattern on Instagram, it’s plausible that this latent ability could be harnessed on any platform that could dial in an exploitation-heavy feed for researchers and safety teams to use.
If only it were that simple…
This isn’t a silver bullet. I don’t see how a recommendation feedback loop could catch one-off AI-generated videos. Any version of this in production would still need forensic video analysis and network-based methods alongside it. The loop is also fragile, and inaccurate seed data can start it off in the wrong direction.
But the loop is well-matched to address the majority of AI-generated content by volume, which comes from accounts trying to capitalize on viral trends. They cluster around particular themes, using similar language to optimize for search. Furthermore, AI content workflows are usually much faster than real content workflows. These are characteristics inherent to AI; they won’t change even as AI videos improve.
I’ve asked academics at major research universities and independent research labs why this hasn’t been studied directly. The answer I get each time is that it’s hard to study from the outside. AI detection on its own is already an open problem, and trying to study a recommendation algorithm’s behavior without access to that algorithm is even harder. I believe open source intelligence techniques provide enough signal to study this.
Inside the platforms, the people who could look into this are split across the organization. The recommendation and Trust & Safety teams have different incentives and different tooling.
Part of the reason I’m writing this is that Riddance has built tooling to seed accounts reliably, track what the algorithm surfaces, and repeat the experiment across categories. That doesn’t solve AI detection on its own, but it means that researchers with the right methodology and tools can study recommendation algorithms without needing direct access to the models.
If the hypothesis holds up, it points somewhere interesting. The same property of recommendation algorithms that gets blamed for amplifying AI slop can also surface inauthentic content at scale. In the right hands, sandboxed recommendation algorithms could harness the detection work that no one asked them to do.
As ever I’m late to the party, just discovered the work of Riddance and this substack! This post piqued my interest, as an academic in astrophysics I know absolutely nothing about social media algorithms but I do know a bunch about scientific coding (read: throwing stuff at a wall and seeing what sticks) and the burden of statistical significance and this seems like the kind of data science that is prolific across the field of astrophysics (and probably many other academic fields, much to the horror of actual data scientists): how do you classify and infer latent hierarchy in a large, heterogenous dataset without knowing enough about the underlying generative process to model the likelihood function? The answer always seems to be to fucking about with clustering algorithms and seeing if you stumble upon any cool hierarchies in your galaxies (or whatever). But seems like the same principle could be applied here.
I abandoned Instagram a while ago, prior to AI generated video, so this may be a naive comment but do you have a feel for the proportion of authentic vs inauthentic accounts in these niches, is it possible there’s bias being introduced due to a skewed sample? And have you considered using an API scraper to pull all manner of account information for entire searches or hashtags (from an admittedly quick and dirty google search, it seems like there is a tool that reliably does this, although with a cost)? You could probably do a whole lot of probabilistic analysis on various metadata to start building a picture of whether there are any underlying patterns through which this kind of content could be classified for prioritisation in the algorithm fairly simply using python (although astronomers assume everything is Gaussian so i am no authority on how astronomy stats measure up in real world applications!)