Blog
Generative engines break online analyzes and hurt their future
May
The search goes from classic search engines to generative engines, but the movement of many of these sites is not properly followed in analysis. It’s their fault, not yours.
I looked at our LLM filter at Ahrefs Web Analytics and noticed that the list lacks some typical generative engines. They are in our filters, but we do not see any data from them for websites.
This undetectable problem of movement comes from these systems without the value of the command. First, I noticed this problem with AI in Google, but this is a common problem for generative engines.
This is most likely a mistake on their part, but in some cases it may be purposeful. Some of these tools probably want a larger market share and simply made a mistake, while others may not want you to be able to measure traffic from systems. Google said that clicks from AI search are higher quality, but we can’t verify it.
If you have a website that sends traffic to other sites, you should want it to be properly followed. In the case of generative engines, I warned that these AI bots must send this information to meet their social agreement, in which they provide traffic on websites, and websites allow these bots to crawl and their data.
Bots suspending sites have costs, and between search engines and website owners there is a social agreement in which search engines augment value by sending a recommendation traffic to websites. This is preparing most of the websites from blocking search engines, such as Google, even if Google seems to be more likely to take more of this movement. This social contract includes generative engines.
I think that many site owners want to allow these bots to find out about their brand, their activities and their products and offers. But while many people assume that these systems are the future, now they risk that they do not add sufficient value to website owners.
The first LLM, which will add more values to users, showing the impressions and clicks to website owners, will probably have a substantial advantage. Companies will report indicators from this LLM, which will probably augment adoption and prevent more websites to block their bot.
The same sentiment concerns assignment. If these generative engines want to win a market share, they must be present in reports for companies. So far, many have not done great work.
I checked the value of the command by entering “Document.referrer” in the Chrome Dev Tools console to check if the command was transferred. If so, it displays the value of the saying where it comes from, and if not, it is empty.
Some generative engines send commands, others do not send them at all, and some send them to certain things, not others. I meant warning people to indicate partial results.
The link in my paid chatgpt account has a noreferrer attribute on the link. This would prevent sending the value of recommendation.
As expected, no referral was shown in the Chrome Dev Tools console. It returns empty.
document.referrer
''In Ahrefs Web Analytics it is registered as unknown, but in Google Analytics it would be classified as direct. Google combines traffic from unknown sources and internal movement of the website together as direct, while we divide them into unknown and internal.
It is fascinating that when I looked at the same type of link on the free account, he did not have a noreferrer attribute. He is properly followed.
In the case of links, they were also properly followed. 
Connections with content sources and at the bottom of the answer are also properly followed, and also add the URL parameter “? 
Network search
Most of the links in the network search mode had a command. I came across an fascinating example when there are many references. The best had a command, the remaining 2 no.
Deepresearch
In the Deepresearch mode, the links were included properly assigned, but the sources at the end were marked with noreferr.
HTTP headers
If you look at the http headlines, sometimes you will find Policy Policy headline To determine what and how much information is transmitted in the referral. You can utilize the SEO Ahrefs toolbar to display this information by going to the HTTP header card.
In the case of ChatGPT, they determined the value of “raw origin-night-cross-origin”. In this case, the reduction from HTTPS to HTTP would drop the command. All links to the pages using HTTP would not be properly assigned.
Most of the context and cited links in the twins were commanded.
One case, which was not the “Website research” section in deep research mode. They are marked as a norefer.
AI mode
The recent AI mode in the Google search engine is also powered by Gemini. Perhaps you saw my last article showing that AI is marked with noreferrer.
John Mueller from Google confirmed that this is a mistake and that they would probably fix it.
In the previous article, Louise Linehan mentioned that we may not appreciate the AI movement. She mentioned specifically how Copilot disappeared from our analytical tracking system. Since then, the movement has returned.
I suspect that these links have been marked as noreferrer during this period. This shows how code changes can affect your global tracking.
Everything here seemed properly followed.
This is not the case with Copilot in Windows. I did not find cases in which the recommender was transferred.
Their website seemed to send recommenders in everything.
Their computer application does not seem to send recommenders to anything. I haven’t tried the mobile application.
Claude seems to have the order of all links in all tested areas.
Grok does not transfer orders at all. I tried the independent groc and version on X.
Normal deep search and deep examinations have not undergone the recommender.
In the case of internet search, individual quotes transferred the command, but the links at the end are not.
Meta AI has transferred the internet version. I did not test it on any of the social media platforms.
Mistral passed the command in all cases I checked.
Final thoughts
Attribution problems are not unique to generative engines. Many movement is attributed to the unknown or direct in your analyzes. This movement came from them.
There is a enormous part of the traffic on the website that is never recorded in the analysis due to people blocking analytics or javascript, some sites are waiting for the acceptance of cookies before release, or people leave the page before they ignite your analytics.
The assignment is becoming more and more challenging. If you are a generative engine and you want to make sure that people know that they receive traffic from you, test all your links to make sure that the data is sent. Your survival may depend on your reputation in the marketing community and visibility in marketing reports.
