Each time you employ a web site like Google or ChatGPT, your browser and a group of backend providers have a protracted and surprisingly candid dialog about what you requested for. That dialog sits in plain sight within the Community tab of Chrome DevTools, a single F12 away, and most of the people by no means have a look at it. I’ve spent an unreasonable quantity of my time watching this knowledge, and need to share what I’ve discovered with you.
Staring into this community abyss is what turned up a Google search exploit value a $13,337 bug bounty. It offered precise knowledge that gave some perception into how Google was working, together with not solely names of its parameters, however precise values for issues equivalent to Google’s site_quality_score, and Google’s personal rq_semantic_class. It additionally led me to be one of many first to find the totally different supply units ChatGPT pulls its outcomes from, and later, the freshness home windows it asks for when it searches, and proof that Reddit continues to be being learn by ChatGPT simply as a lot as ever, despite the fact that it has largely stopped being cited.
None of this wants a knowledge science crew. It wants curiosity, a browser, and as of late an AI agent prepared to do the tedious bits. The purpose is just not that community knowledge is cool (it’s, mildly). The purpose is that realizing issues your rivals don’t is the closest factor web optimization has to a sustainable edge, and this is likely one of the final locations these issues are nonetheless mendacity round free of charge.
Why Anybody Would Do This To Themselves
“Why?” is an inexpensive query, and one I think most SEOs ask the lavatory mirror most mornings. Anybody that is aware of me, is aware of I get pleasure from taking part in video games, significantly video video games, and I like a brand new problem of determining the best way to win. That’s the recreation for me.
Living proof, my finest pal not too long ago taught me backgammon. As with every new recreation, I discovered the foundations after which misplaced repeatedly; studying the foundations of a recreation is just not the identical as realizing the best way to play nicely. I believe I misplaced round 20 or 30 video games on the trot. They are saying 50. The determine is contested. Sooner or later throughout this run of humiliation, quite than merely having fun with a pleasing board recreation like a well-adjusted grownup, I began determining the best way to get small edges. For example, figuring out which distances on the board are literally reachable from two cube, and the way probably each is. That issues, as a result of it tells you the place you should block your opponent and the place you possibly can safely go away a bit uncovered.
As I see it, web optimization works the identical means. You’ll be able to study each finest follow within the information and you’ll nonetheless be dangerous at it till you’ve got expertise and learn to play the sport.
Optimization is the sum of cumulative positive aspects and edges. Gaining this data lets you stack 1% in your competitors over time in a means that’s unattainable for that to mimic. This isn’t some “hack” method like a listicle they’ll copy in 10 minutes.
The Paper, Scissors, Rock Analogy
I found an analogy that I do know others discover helpful to elucidate this strategy to shoppers or managers with the sport paper, scissors, rock.
Right here’s the state of affairs:
Think about you might be taking part in infinite rounds in opposition to somebody whose said technique is to play rock each single time. What’s the optimum option to play in opposition to them?
Most individuals instantly say “play paper each time” is the optimum option to strategy this drawback, and they might be incorrect. Whereas you’ll win the primary 5 or ten video games, your opponent, who is just not a houseplant, notices they’re dropping each recreation and modifications what they’re doing. This then steals any benefit you’ve got and places you again onto a good footing.
The optimum strategy is to play randomly however play paper barely extra usually. Your “optimization” will probably not be perceptible by your opponent, and with no apparent purpose to adapt, you slowly, quietly carve out an ever-increasing lead over time.
Blackhat Vs. Whitehat; Anticipated Worth (EV)
The paper, scissors, rock analogy primarily encapsulates the argument between blackhat and whitehat web optimization approaches.
Blackhat web optimization completely works, and you may make quite a lot of cash. Within the analogy we outlined above, blackhat web optimization is the “taking part in paper each time” technique. It’s efficient; your short-term anticipated worth (EV) is excessive, however you pressure the opposite facet (Google, ChatGPT, whoever) to vary what they do, at which level your benefit evaporates and sometimes takes your area with it.
Whitehat web optimization is the “play paper barely extra” strategy, which focuses on producing worth with out anybody needing to vary the foundations, and to stack it up over time.
Community knowledge is likely one of the finest locations I do know to search out this sort of data to offer you an edge over the competitors.
What Community Information Really Is
Whenever you use an online app like Google or ChatGPT, your browser is sending knowledge to backend providers and receiving knowledge again. The least technical means I can describe it’s delivery crates touring to and from a warehouse.
Some crates have clear labels, so you realize roughly what’s inside. Some have labels that make no sense in any way. And a few you possibly can merely open, look inside, and infer what’s going on from the contents.
Getting at these crates is just not tough. Press F12, do a Google search or ChatGPT immediate, and you might be, for all sensible functions, within the Matrix. Click on the Community tab, filter to Fetch/XHR, and also you get a listing of requests. Click on on any one in all them, and also you get a set of tabs that look intimidating and aren’t:
- Headers: directions and metadata in regards to the request.
- Payload: what you despatched.
- EventStream: what continues arriving dwell, as knowledge bounces forwards and backwards between your browser and the backend (ChatGPT makes use of this closely whereas it’s producing a solution).
- Response: what you bought again.
The rationale that is attention-grabbing quite than merely nerdy is the way in which these programs are constructed. Mario Fischer’s diagram of how Google’s rating programs match collectively, assembled from the leaked Content material Warehouse documentation and DOJ trial displays, makes the purpose visually: what seems from the surface like one monolithic search engine is definitely an enormous variety of small subsystems speaking to one another.
Techniques on the entrance finish, like Google’s Superroot, have to know what you looked for, they usually pull in scores and alerts generated elsewhere within the stack. These scores and alerts must journey someplace. Generally they journey by means of your browser, and generally no person remembered to take the labels off.
How We Discovered The Google Exploit (A Guide Instance)
A small confession earlier than we start. The rationale we spend a lot time doing that is that we run AlsoAsked, which includes quite a lot of work with Google’s Folks Additionally Ask knowledge. I point out this so that you simply decide me barely much less for the period of time I spend watching Google’s community requests. Solely barely.
Whereas digging by means of request headers, we got here throughout a callback URL with a huuugee and attention-grabbing parameter string connected to it.
Naturally, the primary thought was to repeat the request right into a terminal, fireplace it off with curl, and see what Google despatched again. (This exercise can also be a really useful Tinder first date exercise; YMMV). We obtained lots of… stuff again, however it didn’t actually make lots of sense.
We observed a parameter known as _fmt, set to prog. We guessed, with no actual justification, that it’d imply “format.” For causes I genuinely can not now keep in mind, in what actual knowledge scientists would describe as “dumb luck,” puzzled what would occur if we merely deleted it.
What occurred was that Google despatched again structured knowledge. Plenty of it. Web site high quality scores. Inverse doc frequency scores. When the web page was final considerably up to date, in epoch time. The server IP. All types of data that Google very a lot didn’t intend us to have.
Which is how we ended up with a bug bounty from Google. We did report it, in equity. A couple of 12 months after we discovered it. We is not going to dwell on that.
The Google Vulnerability Reward Program panel awarded $13,337.00, noting that “exploitation chances are excessive” and that the problem “certified as an abuse-related methodology with excessive affect.” I’ve chosen to take that as a praise.
Getting Worth From the Information
Whereas there was lots of direct worth in that knowledge, I need to decide a barely extra uncommon instance of the way it was one thing helpful.
One of many issues we observed whereas hoovering the whole lot up was that Google classifies just about each question into one in all eight “refined question” semantic courses. Throughout a pattern of 4.8 million English queries, the distribution appeared like this:
A pile of labelled knowledge like that’s precisely what you should practice a classifier. I went to Hugging Face, the AI neighborhood and mannequin hub, and downloaded DistilBERT, a smaller, quicker model of Google’s BERT mannequin (40% smaller, 60% quicker at inference, and retaining round 96% of BERT’s efficiency). It’s sufficiently small to run by yourself pc, which is technically true within the sense that I managed to finish the fine-tuning based mostly on all 4.8 million queries and Google labels in about 50 hours on a gaming laptop computer (which was doing a fantastic job heating my room).
The result’s QueryClassifier, which we put on-line free of charge. You give it any search question, and it predicts how Google goes to categorise it.
This output is extra helpful than it seems for a content material pipeline. Understanding the question class tells you what kind of content material Google is anticipating to reply it with, and there’s a sturdy correlation between question class and the SERP options Google chooses to point out. So you possibly can know forward of time, and at scale, which options you should be optimizing for. You too can discover out, with 99.9% confidence, that “what occurs if john mueller eats cheese” is a CONSEQUENCE question, which I think about a public service.
Getting ChatGPT Fan Outs First
I’ve been doing this sort of guide evaluation for years. Over a 12 months in the past, I used to be posting tips about the best way to watch ChatGPT’s fan-out queries within the Community tab: Open the dialog request, have a look at the Response, and seek for search_model_queries. On the time, I don’t assume anybody else had that.
That particular trick not works, as a result of the format has since modified (extra on that shortly). However the precept stands. The attention-grabbing data is just not within the reply ChatGPT reveals you. It’s within the crates.
Now You Can Vibe Code Your Strategy to This
Happily(?), we now dwell in a world of vibe coding, which suggests you not must do all of this by hand.
My weapon of selection is Perplexity Pc, Perplexity’s agent orchestration system. I gave it what Gen Z would describe as a “fairly beg” immediate. In essence, I’m telling Perplexity that I’m attempting to work out how ChatGPT works by analyzing its community knowledge in Chrome. I give it some examples of the forms of issues I’m on the lookout for (search_model_queries), and I would like it to assist me run some searches, analyze what comes again, and determine the rest which may trace at how the system works.
Perplexity was very adept at this job. Within the background, it used Playwright to fireplace up actual Chrome cases, ran searches on ChatGPT, captured the community knowledge, and produced a twenty-something-page report that we may then have a dialogue about.
That’s how I used to be one of many first to find that ChatGPT was pulling its outcomes from totally different supply units, labelled in a result_source discipline:
On the time, with the restricted dataset, I believed these to be:
- serp: dwell net search outcomes, scraped from a search engine within the background. Basic retrieval-augmented era.
- labrador: which I didn’t recognise on the time, and which seems to be OpenAI’s personal index.
- vivid: which I consider is Vivid Information, a supplier of structured business knowledge packs.
I ended up publishing this analysis earlier than I’d actually had an opportunity to dig into it for the noble purpose of blind panic. Metehan Yesilyurt had teased he had an enormous reveal on LinkedIn round how ChatGPT works, and I had a horrible feeling I knew precisely what it was going to be about. So I messaged him to apologize upfront, defined I had been doing the identical analysis, and posted mine instantly. That is what passes for etiquette in technical web optimization.
To be clear, folks have since accomplished significantly better, deeper work on this than I did. Tomek Rudzki‘s analysis for Peec AI discovered that result_source solely ever carried 4 values (Labrador, Vivid, Oxylabs and SERP) and that Labrador is just not a single index however a household of vertical indexes protecting the overall net, PDFs, YouTube, information, arXiv, Wikipedia, native, finance, authorized, medical, procuring and pictures. If this topic pursuits you, learn that, and observe Metehan, who writes quite a lot of good materials on it.
Getting Across the Scraper Bouncers
If you find yourself taking part in this sort of recreation at scale, there’s a drawback you’ll run into rapidly: getting blocked. Google blocks you. ChatGPT blocks you. Automated browsers pushed by vibe-coded scripts aren’t refined, and the platforms have observed.
The workaround I landed on was to get Perplexity Pc to construct me a customized Chrome extension to automate my native browser. This implies I may help with human checks; I may have simple management over which fashions, accounts, and settings I used to be utilizing (which all affect outcomes).
The cool factor is, you don’t want to publish it to the Chrome Net Retailer and wait for somebody to overview it. You go to chrome://extensions, swap on Developer mode, click on “Load unpacked,” and level it on the folder. That’s it. No hoops.
Other than pulling aside the community knowledge of ChatGPT, the extension “I” constructed features a “analysis mod” which lets me paste in a complete checklist of prompts, then automates the browser for me: with repeats and gaps between runs, saving all the community knowledge that comes again. That permit me do that evaluation at a a lot bigger scale than poking at DevTools by hand.
I used to be going to tidy up a model of this for public launch, however it turned out I by no means wanted to, as a result of my good pal Suganthan Mohanadasan had been doing very related analysis on the similar time and launched FanoutFox, a free Chrome extension that does primarily what mine does, minus the majority checker, and does it extraordinarily nicely. It reads your individual ChatGPT session and reveals you the fan-out queries, the sources fetched, who obtained cited, and who solely obtained talked about.
What FanoutFox Information Confirmed
1. ChatGPT’s Freshness Home windows
Certainly one of Suganthan’s findings is the form of factor I might beforehand have anticipated to wish a leak to study. When ChatGPT runs a search, the community knowledge contains the question it ran adopted by a vertical pipe and a quantity. That quantity seems to be a time window, in days.
Ask whether or not Nvidia is overvalued and ChatGPT searches for inventory value data with a window of two days. Premier League weekend outcomes get seven. “Finest AI dwell chat software program” will get 30. Nvidia earnings steerage will get 90. And when it goes taking a look at Reddit for dwell chat assist opinions, it targets reddit.com particularly, with a window of three hundred and sixty five days.
There isn’t a scarcity of assured opinion within the trade about how recent content material must be. This is likely one of the only a few locations you will get an goal reply, from the system itself, in regards to the window it’s really wanting in for a given matter. That may be a excellent start line for a content material pipeline: For this sort of question, that is roughly how updated the data must be to qualify in any respect.
2. Whether or not ChatGPT Is Even Looking out
FanoutFox can even inform you whether or not retrieval is going on in any respect for a given immediate, or whether or not the reply got here fully from the mannequin’s parametric reminiscence, that means its coaching knowledge.
I requested it to advocate self-help ebook (one other dependable option to inform somebody works in web optimization). Considerably to my shock, the reply got here fully from coaching. No net search ran. ChatGPT was merely recommending books based mostly on what it had learn.
That’s genuinely helpful to know, as a result of it means it’s near unattainable so that you can affect that reply within the brief time period. If the response is coming from coaching knowledge, altering it’s a long-term challenge, not one thing you repair with a brand new touchdown web page subsequent Tuesday. Understanding which of your goal prompts set off retrieval and which don’t tells you the place effort will repay now and the place it’s a sluggish burn.
3. Reddit Didn’t Go Anyplace
You’ll most likely have seen that Reddit is being cited far much less by ChatGPT. Petra Labs’ evaluation of 113,500 ChatGPT runs confirmed Reddit’s share of ultimate citations falling roughly 80% in a single day in mid-August 2026, shortly after GPT-5.6 turned the default mannequin.
It will be very simple to take a look at that chart and conclude Reddit is not value bothering with for AI visibility. The community knowledge says in any other case. Suganthan’s analysis, and the information popping out of FanoutFox, confirmed that whereas Reddit is being cited a fantastic deal much less, it’s nonetheless being used simply as a lot within the background. The most important change since GPT-5.6 is that the fan-out searches lean closely on website: queries, and Reddit continues to be very a lot amongst them. ChatGPT continues to be studying what folks say on Reddit to determine whether or not to advocate a model. It has merely determined to quote the seller’s personal web page as a substitute of the thread.
That may be a aggressive benefit for anybody who is aware of it, and a lure for anybody who solely reads the quotation charts. It is usually instance of the qualitative perception that large quotation research are inclined to miss: counting the hyperlinks in a solution tells you what the mannequin selected to point out, not what it selected to learn.
However Wait, There’s Extra…
Just a few takeaways to encapsulate the broad and particular:
This knowledge is functionally useful to your optimization. Understanding the freshness window to your matter, or that Reddit continues to be being learn, is just attention-grabbing till you employ it to determine what content material to construct, when to replace it, and the place to spend effort. Then it’s cash.
These programs change repeatedly, and there’s a nice deal to study. The way in which ChatGPT sends its community knowledge has modified radically over the previous few months, apparently at random. Suganthan has documented ChatGPT rebuilding its search device to the purpose the place the search_queries discipline he and I each relied on vanished fully. I put up about this lower than I used to, partly as a result of issues appear to vary shortly after folks put up about them. Make of that what you’ll.
Use the instruments that now exist. You don’t should be a developer to do any of this anymore. Perplexity Pc can run the browsers, seize the visitors, and summarize what’s attention-grabbing. It may well construct you a Chrome extension that you simply load in two clicks. And FanoutFox will do lots of the ChatGPT studying for you, free of charge.
The information is all there, transferring between your browser and the backend each time you search. Most individuals by no means open the crates. You need to.
Extra Assets:
This put up was initially revealed on Mark Williams-Prepare dinner.
Featured Picture: TippaPatt/Shutterstock
