Right now, a lot of commerce teams are trying to “win AI.”
They want their brand to show up when someone asks ChatGPT, Claude, or Google AI Mode questions like:
- “Best leggings for travel”
- “Best skincare for acne”
- “Best kitchen knife under $300”
- “Best Amazon supplements for sleep”
A new research project from SparkToro (Rand Fishkin + partner) put hard numbers behind something operators have felt for a while: AI tools are highly inconsistent when recommending brands or products. Ask the same prompt repeatedly and you get different lists, different ordering, and even different list lengths.
That matters if you’re spending money on “AI visibility tracking.”
But it matters even more if you’re building your growth engine on short-form video - because video is where most product discovery now happens (TikTok Shop, Reels, Shorts, Amazon video, paid social).
This post is for Shopify brands, Amazon sellers, TikTok Shop operators, and D2C teams using an AI video generator or AI video creator to scale content. The goal: help you avoid the wrong metrics, and build a content system that still wins even when AI recommendations are a lottery.
Main keyword: AI video generator
Supporting keywords: UGC video AI, AI video for TikTok Shop, Amazon product video, shoppable Reels, scale video production
What did the research actually prove (in plain English)?
SparkToro’s experiment ran thousands of repeated prompts across ChatGPT, Claude, and Google AI Overviews / AI Mode.
Key findings worth stealing for commerce:
- The same prompt rarely returns the same list twice (think: less than 1 in 100).
- The same prompt almost never returns the same list in the same order (closer to 1 in 1,000).
- AI outputs are probability engines, not stable rankings.
But there’s a nuance that matters:
- “Ranking position” is basically noise.
- Visibility percentage (how often a brand appears across many runs) can be directionally meaningful.
So the takeaway is not “ignore AI.”
It’s: stop treating AI like Google rankings.
Why this matters for social commerce teams (not just SEO teams)
Most brands are not losing because they lack a clever strategy.
They’re losing because they can’t produce enough video variations to keep up with:
- TikTok Shop creative fatigue
- Meta Reels placement diversity
- Amazon PDP video needs (and ad variations)
- Creator-style UGC expectations without creator budgets
Now layer in AI discovery.
If AI recommendations are inconsistent, then a strategy built around “we need to be #1 in ChatGPT” is fragile.
But a strategy built around being the brand that shows up everywhere people look (feeds, PDPs, AI answers, creator content, comparison videos) is resilient.
And that strategy is mostly a video production problem.
The trap: “AI visibility” dashboards can push you into the wrong work
If your dashboard says:
- “You dropped from #2 to #7 in ChatGPT for ‘best running shorts’”
…you will feel pressure to do something.
But the research suggests that “#2 vs #7” may be a random reshuffle, not a real change in market position.
The operational risk is that teams start optimizing for the wrong output:
- writing more “AI-friendly” copy
- chasing prompt hacks
- paying for visibility reports that can’t be reproduced
Meanwhile, the thing that actually moves conversion is still:
- better product demonstration
- clearer benefits
- stronger proof
- more variations
- faster testing cycles
In other words: more video, not one perfect video.
So what should you track instead (if you sell products)?
If you’re a Shopify merchant, Amazon seller, or TikTok Shop brand, here’s a more useful measurement stack than “AI rank”:
1) Track AI visibility as a distribution, not a position
If you do track AI at all, treat it like this:
- “We appear in 18% of responses across 200 prompts in this category”
- “Competitor A appears in 42%”
- “We’re strong in ‘travel’ prompts, weak in ‘wide fit’ prompts”
That aligns with the research: visibility percentage can be meaningful.
2) Track what actually pays the bills: creative throughput + conversion
For video-led commerce, the KPIs that matter are boring and powerful:
- videos shipped per week (by product, by angle)
- time from product drop to first 20 creatives live
- cost per creative variation
- hook retention (1s, 3s)
- thumbstop rate
- CTR to PDP
- PDP conversion rate (Shopify or Amazon)
- blended CAC / MER impact
AI visibility can be a side signal.
Creative output is the engine.
What this changes for AI video creation (the practical playbook)
If AI recommendations are inconsistent, your job is not to “win the list.”
Your job is to be the easiest brand to say yes to once you show up.
That’s a video problem.
Here’s the operator playbook.
Step 1: Build a “consideration set” content plan, not a single hero ad
AI tools pull from a messy corpus of what’s talked about, compared, reviewed, and repeated.
So your content should cover the moments where people decide:
- “Is this legit?”
- “Is it better than X?”
- “Will it fit me?”
- “Will it work for my use case?”
For fashion and apparel, that means scaling videos like:
- fit checks (true to size vs size up)
- fabric close-ups (stretch, thickness, sheerness)
- outfit formulas (3 ways to wear)
- before/after (wrinkle resistance, sweat, shaping)
- comparison clips (“vs Lululemon”, “vs Skims”) - carefully, but consistently
For CPG, it looks like:
- how to use
- taste/texture expectations
- routine placement (“morning vs night”)
- “who it’s for” and “who it’s not for”
- proof and constraints (allergens, certifications, results timeline)
This is where an AI video generator becomes infrastructure: you can generate video with AI from product images, existing clips, or templates, and scale variations without booking shoots every week.
Step 2: Produce in clusters so you can test faster
Instead of making 1 video per product, ship clusters:
- 10 hooks
- 5 benefit angles
- 3 formats (UGC selfie, product-only, text-led)
- 2 lengths (9-12s and 20-30s)
- 3 aspect ratios (9:16, 1:1, 16:9)
That’s 900 variations if you have 10 SKUs.
No team can film that traditionally.
But content teams using AI video creation can.
Step 3: Treat “UGC” as a format, not a person
A big chunk of “AI visibility” conversation is really about trust.
UGC works because it compresses trust into a familiar format.
You can create UGC-style videos without creators by standardizing:
- first-person language
- specific use cases
- objections + answers
- imperfect pacing (not overproduced)
- product-in-hand shots (real or simulated depending on workflow)
This is where UGC video AI is practical: not to fake influence, but to scale the format that performs.
Step 4: Match platform intent, not just creative style
AI recommendation inconsistency is a reminder: distribution is fragmented.
So your video system needs to output for each surface:
TikTok Shop video
- fast hooks, price anchoring, social proof
- “why this over alternatives”
- short demos that make the product feel obvious
Instagram Shopping and shoppable Reels
- aesthetic-first, but still benefit-clear
- creator-style voiceover + captions
- strong first frame for Reels grid
Amazon product video
- clarity beats vibes
- show what’s in the box, scale, usage, results
- answer top review objections visually
Paid social creatives
- iterate hooks aggressively
- build a library of winners by angle
- refresh weekly to fight fatigue
If you want a deeper read on how distribution is shifting toward social-first surfaces, see: The Social Media Shift Shopify Brands Can’t Ignore.
“If AI is random, should we stop caring about AI discovery?”
No.
You should stop caring about false precision.
Here’s the