Study Reveals Instability and Inconsistency in Google's AI Overview Results
A recent study conducted by WordStream by LocaliQ found that Google's AI Overview results are highly unstable over time. The experiment, spanning six weeks, revealed an average source retention of only 50%, with approximately three out of four cited sources changing weekly. This inconsistency challenges the effectiveness of optimizing content specifically for AI search features, despite observations of certain types of sites, like Reddit and educational institutions, frequently appearing in the overviews.
WordStream by LocaliQ conducted a six-week study to evaluate the stability and consistency of Google's AI Overview results. The research aimed to test the hypothesis that optimizing solely for AI search is misguided, largely due to concerns about low click-through rates and inconsistent citations.
Over the course of the study, researchers performed 14 common marketing searches every Thursday afternoon. They manually recorded the top 10 sources cited in the resulting AI Overviews each week to track changes.
The findings indicated that AI Overview results were largely unstable. The study reported an average source retention rate of only 50%, meaning roughly three out of every four cited sources changed from one week to the next. The layout of the AI Overview also varied, with the number of cited sites for a single query fluctuating significantly.
Despite the overall instability, the study identified certain types of content and platforms that frequently appeared. Reddit was noted as a significant winner, with social sites, videos, and content from educational institutions also presenting opportunities for visibility. Conversely, the study found that thought leadership content was not prominently featured in AI Overviews.
According to WordStream by LocaliQ, the experiment solidified the team's hypothesis that optimizing for AI search is unreliable due to the high volatility of its results.
