If you work in AI search (GEO/AEO/SEO), you have probably seen the frustrating pattern this guide is about: a page is indexed, ranks for relevant terms, and may even be retrieved by an AI system, yet another source gets quoted. While traditional SEO reporting often collapses this into one outcome called "visibility," AI search does not work as one decision. Discovery, passage selection, source selection, and answer generation are separate decisions.
Why you should care about rerankers at all
A reranker sits in the middle of that chain. After a search system has found a shortlist of candidate documents or passages, a neural reranker usually reads each candidate against the user's query and assigns a relevance score. Not every production system works the same way: some reranking also draws on click or engagement signals, as public Google search trial evidence has suggested for classic web ranking. Neural relevance scoring is still the useful mental model for AI-answer pipelines, but it is an important gate, not the only gate. Authority, freshness, source quality, retrieval coverage, diversity rules, and the final LLM can all change which source is cited.
In many retrieval systems your page is not evaluated only as one URL. Parts of it can be retrieved and scored as passages. A strong page can therefore contain a weak candidate passage, while a less authoritative page can contain the clearest extractable answer. That’s why page-level SEO metrics alone can’t explain every citation win or loss.
Content SEO Learn why an on-topic section can still fail an answer-seeking query, and how to improve answer completeness without keyword stuffing. | GEO/SEO Specialist or Reputation Manager Separate retrieval coverage from passage selection and final citation use, so a visibility loss is diagnosed at the right stage. | Technical GEO/SEO or Relevance Engineer Choose a realistic set of models, interpret scores safely, and avoid dashboards built only on attention heatmaps. |
By the end of this guide, you should be able to answer four questions:
Why can a relevant, well-ranking page still lose the citation?
Why do listicles appear so often as sources?
What can BGE, MiniLM, SPLADE, ColBERT, and Qwen each tell you about a passage?
What should you actually change, measure, and report?
The animated cards in this post replay measured outputs from 12 open rerankers reading the same two passages. They are included to make the models easier to understand. Nothing is fetched while you read the page. A static side-by-side Passage A/B card and score table appear later for social, email, and print.
Reading tip: You can read this guide in order. Parts 2–3 are optional background if you already know BERT basics; Part 6 is the practical “which model when” guide; Parts 7–8 are the content actions. |
Scope: This guide describes how modern retrieval systems commonly work. It’s not a claim that ChatGPT, Google AI Overviews, or Perplexity use any specific open-source model. A high open-model score helps you diagnose content but it doesn’t prove a URL will be cited. |
First, watch one reranker think
This is BGE-reranker-large reading a passage about Peec, an AI search visibility platform, for the query "what are the best aeo tools". It first reads a product description, then a direct shortlist. Word colors are a rough hint about where the model looked; they are not a precise explanation. Watch how the score moves from Passage A to Passage B.
Part 1: The 30-second mental model of AI search
What you’ll learn here: Before editing content, identify the stage that failed. If the page was never retrieved, rewriting a paragraph for a reranker can’t fix the problem. If it was retrieved repeatedly but rarely selected or cited, passage-level relevance is more likely to be the issue. |
A common pattern in documented search and RAG systems looks like this, although proprietary products can add more stages and signals:
Query planning. The user's question gets rewritten and often split into several search queries (this is called query fanout). The prompt "Best CRM for small business" might become three or four separate searches.
Retrieval (the wide net). Two kinds of search run, usually together. Firstly, keyword search (BM25 or similar). Classic matching on exact words, like a smarter version of Ctrl+F. It’s fast and literal. Secondly, vector search (embeddings). Every text is turned into a list of numbers that captures meaning, so "car" and "automobile" land near each other. This catches things keyword search misses. The two result lists get merged, commonly with a method called Reciprocal Rank Fusion (RRF), which rewards documents that rank well in both lists.
Reranking (the fine sieve). A shortlist, often with dozens or hundreds of candidates, gets re-scored by a model that reads the query against each passage. The strongest candidates move on.
Generation. The LLM writes the answer using the surviving passages and may cite some of them.
In short, retrieval decides whether a passage enters the candidate set, reranking helps decide which candidates survive, and generation decides what the answer uses and cites.
Quick diagnose: No retrieval → indexing, coverage, authority, or freshness. Retrieved but losing → answer shape or passage relevance. Strong passage but no citation → source quality, diversity, or generation-stage behavior. Fix the failed stage first. |
AI search to-do: Diagnose first (15 minutes)
☐ List 5–10 target prompts and label each answer category: brand, definition, best-of, comparison, or how-to. (These are prompt categories, not classic commercial/informational SEO intents.)
☐ For each prompt, save the section on your page that should answer it. If a competitor URL is cited or mentioned, open that URL and capture the section that seems to supply the answer. If only a URL is shown, use the most answer-like section on that page.
☐ Assign one failure bucket: not retrieved, retrieved but weaker, or mentioned/cited with the wrong claim.
☐ Only then choose an edit. Don’t start with keyword density or model-score chasing.
Why not just use the big LLM for everything?
This question is about the systems behind AI answers, not about asking ChatGPT to rewrite your blog post. A ranking stage at OpenAI, Anthropic, Google, or a similar stack may evaluate many passages per query. Running a large general-purpose LLM on every pair can be slow and expensive, so those systems often use smaller models specialized for relevance. Their job is narrow: compare a query and a passage, then return a score.
Part 2: What SEO specialists actually need to know about BERT
What you’ll learn here: Why exact keywords, surrounding context, chunk length, and query-passage interaction change model scores. You don’t need enough knowledge to train BERT. |
Many rerankers are based on BERT or related encoder architectures, which is why just five minutes on BERT makes the rest of this guide easier to follow.
What is BERT?
BERT (Bidirectional Encoder Representations from Transformers) is a language model that Google released in October 2018, Containing two sizes at launch: BERT-base with 110 million parameters and BERT-large with 340 million. It was a breakthrough because it reads text in both directions at once.
Older models read a sentence left to right, like a human reading aloud. BERT looks at every word in the light of every other word around it, simultaneously. In the sentence "the bank of the river", BERT understands "bank" means a riverbank, not a financial institution, because "river" is right there influencing it. This mechanism is called attention, where every word constantly asks "which other words in this text matter for my meaning?"
How BERT was trained (and why you should care)
BERT learned language through a fill-in-the-blank game called masked language modeling. Take billions of sentences, hide 15 percent of the words, and make the model guess them. "The capital of France is [MASK]" teaches the model facts, grammar, and word relationships without any human labeling.
This training style helps BERT-family models represent contextual meaning. Relevance-ranking still needs an extra fine-tuning stage on query-document examples. In 2019, both Google and Microsoft said BERT or other transformer-based methods were used in search ranking.
The obi-encoder vs cross-encoder distinctionBi-encoder (embedding models). The query and the document are processed separately, each turned into one vector, and then the vectors get compared with simple math (cosine similarity or, when vectors are normalized, an equivalent and often faster dot product). Because documents can be embedded ahead of time, you can search millions of them in milliseconds. That speed costs precision. Each text was compressed into one vector before the model ever saw the other text, so fine-grained interactions ("does this passage answer this exact question?") get blurred.
Cross-encoder (most rerankers). The query and the passage are fed into the model together, as one combined input. Every word of the query can attend to every word of the passage through the whole network. This is far more accurate, but far slower, because nothing can be precomputed. Every query-passage pair needs its own full model run.
That is why the pipeline runs in two stages. Bi-encoders (plus BM25) cast the wide net cheaply, cross-encoders re-score the shortlist expensively. Nearly every production search system, including the ones behind AI chatbots, is built this way.
Takeaway: A bi-encoder answers "could this passage be relevant?" A cross-encoder answers "how well does this passage satisfy this query compared with the other candidates?" If you only test embeddings, you are measuring discovery, not the final relevance check. |
Where does the reranker's score come from?
In a classic cross-encoder, the model compresses the query-passage pair into one relevance score and often shows it as a percentage so candidates are easy to sort. That percentage is for ranking passages against each other.
Important: These percentages don’t guarantee a citation. They are mainly used to sort available passages by relevance within one model and query. |
Part 3: Which model family matters for which SEO task?
What you’ll learn here: Related model families often share training assumptions. Part 6 gives a practical “which model when” guide; this section is the map. |
"BERT family" is not one model. It is a group of related encoder architectures and their descendants. The backbone tells you how text is processed; the ranking fine-tune tells you what the model learned to call relevant. You don’t need to memorize every name.
MS MARCO, early: Many classic English rerankers are fine-tuned on MS MARCO, a Microsoft dataset built from anonymized Bing queries and human-judged answers or passages. A model name with "ms-marco" usually points to that training lineage. It doesn’t mean the model learned from raw click logs, nor does it mean Google or ChatGPT uses that exact checkpoint. |
BERT (Google, 2018)
The original encoder-only transformer: 512-token context, English. Included here for completeness and historical accuracy. Most classic MS MARCO cross-encoders sit in this immediate family. Its practical legacy for SEO work is contextual, passage-level evaluation.
DistilBERT, MiniLM, TinyBERT (2019-2020): The distilled children
Distillation means training a small "student" model to imitate a big "teacher" model. You keep most of the accuracy at a fraction of the size and cost.
MiniLM (Microsoft, 2020) is a widely used distilled family in search. The workhorse rerankers ms-marco-MiniLM-L-6-v2 (about 23M parameters, 6 layers) and ms-marco-MiniLM-L-12-v2 (about 33M, 12 layers) are compact cross-encoders fine-tuned on MS MARCO passage-ranking data. They run comfortably on a CPU. Part 6 covers when to use L6 versus L12.
TinyBERT (Huawei, 2019) goes even smaller. ms-marco-TinyBERT-L-2-v2 has about 4 million parameters and 2 layers. It’s small enough to triage many candidates before a stronger model reviews the shortlist. Part 6 covers safe use.
MS MARCO detail: Relevance labels tend to reward passages that contain an answer to the query, whatever the answer format is. There is no special "listicle" query class inside MS MARCO. |
RoBERTa (Facebook, 2019): BERT, trained properly
RoBERTa is the same architecture as BERT but trained longer, on ten times more data, with the ineffective parts of BERT's training recipe removed. It reliably beats BERT at the same size. Many later models start from RoBERTa rather than BERT.
RoBERTa also explains why two BERT-shaped tools can differ before relevance fine-tuning even begins: ranking dataset and language coverage often matter more than the backbone name alone.
XLM-RoBERTa (Facebook, 2019): The multilingual one
RoBERTa trained on 2.5 terabytes of text across about 100 languages. This is the backbone of the most popular open reranker in the world:
BGE-reranker-base (278M) and BGE-reranker-large (560M) from BAAI (Beijing Academy of Artificial Intelligence) are fine-tuned XLM-RoBERTa models. If you have used any open-source RAG tutorial, you have probably used these.
BGE-reranker-v2-m3 (568M) is the newer version built on BGE-M3, which itself descends from the XLM-RoBERTa lineage. It handles 100+ languages and longer inputs, and is the default recommendation for multilingual work.
For international comparisons, score query and passage in the language users actually use. An English score doesn’t transfer unchanged to Turkish, German, or Japanese. Part 6 has the multilingual picks.
ELECTRA (Google, 2020): The efficient learner
ELECTRA changed the training game. Instead of guessing masked words, it learns by detecting which words in a sentence were secretly replaced by fakes. This gives more learning signal per training hour. ms-marco-electra-base (110M) is the ELECTRA representative among classic rerankers, and as you’ll see in its card below, it behaves noticeably differently from its BERT cousins on the same input.
ELECTRA is a useful second opinion from a different backbone. If it disagrees with MiniLM, inspect the passage rather than averaging scores.
DeBERTa (Microsoft, 2020-2021): The benchmark champion
DeBERTa improved how position and content information are mixed inside attention. It topped academic leaderboards for years. Fewer popular open rerankers are built directly on it, but its ideas leaked into everything that came after.
A strong language backbone is not automatically a strong search reranker. Check the relevance fine-tune, languages, and context length before choosing DeBERTa for content tests.
ModernBERT (December 2024): The modern revival
For six years the encoder side of the transformer world barely moved while decoder LLMs got all the attention. ModernBERT brought encoders up to date: an 8,192 token context window (16 times the original BERT), modern positional embeddings, and much faster inference. This matters for SEO tooling because 512 tokens is only about 350 words, so classic rerankers literally can’t see most of a long article at once. ModernBERT-generation rerankers can.
Ettin (Johns Hopkins, 2025) is a family of encoders trained with the ModernBERT recipe at multiple sizes. ettin-reranker-150m is the biggest dissenter in the measured example below. Keep an eye on its card.
Longer context reduces hard truncation, but production systems may still retrieve smaller sections. A whole-page score can hide the exact section that answers the query.
The family tree, in one picture
Outside this encoder tree there is a newer branch: decoder LLMs such as Qwen3-Reranker (Alibaba, 2025), which reuse a generative LLM as a relevance judge. They matter because they can follow ranking instructions and accept longer contexts. Section 4.4 covers them; Part 6 says when to use them.
Why combine more than one reranker?
One model is one opinion. In practice a good check combines a few different families so you’re not repeating the same training assumptions: one strict cross-encoder, one multilingual or modern encoder, SPLADE (a sparse lexical model that shows exact missing terms), ColBERT (a late-interaction model that shows which query words lack a strong partner), and optionally an LLM-based reranker. Part 6 turns this into a decision guide.
Part 4: What each reranker can and cannot tell you about content
What you’ll learn here: Which model output supports which content decision. Cross-encoders are best for relative relevance, SPLADE for lexical gaps, ColBERT for uncovered query facets, and LLM rerankers for complex or instructed relevance. |
BERT-family cross-encoders are one way to build a reranker. The other families expose different evidence. We tested four families on the same sentence so you can separate "what the model scored" from "what its explanation can legitimately tell you."
4.1 Cross-encoders: The judges
Examples: ms-marco-MiniLM, BGE-reranker, Jina-reranker v1/v2, mxbai-rerank-v1, GTE-multilingual-reranker
Cross-encoders take the query and passage in together and return one relevance score, with the highest accuracy per parameter for short passages.
Two practical quirks an GEO/AEO should know:
Attention sinks. BERT-family models park a large share of their internal attention on punctuation and structural tokens (periods, [SEP] markers). This is a documented phenomenon (Clark et al., 2019, "What Does BERT Look At?"). In our measurements, BGE-reranker-large puts roughly 24 percent of its attention on punctuation and special tokens for a typical passage. If a tool shows you "the model paid most attention to this one word," be skeptical. Clusters of highlighted words are meaningful but a single top word often is not.
Below are the cross-encoder-style models in this run. Each card uses the same Peec AI passage and query. Watch the relative change between the descriptive and direct-answer versions, not the absolute percentage across different models. For “which model when,” jump to Part 6.
Reading the results: The strict MS MARCO models gave the descriptive passage near-zero scores; BGE-large and Qwen3 were also low; Ettin treated the same passage as highly relevant. This one example can’t tell us which model is universally "right." The robust observation is within-model: every model scored the direct-answer version higher or already near its ceiling. Disagreement is a reason to inspect the passage, not to average incompatible scores. |
Reranker | Passage A (description) | Passage B (direct shortlist) |
MS-MARCO MiniLM-L6-v2 | 0.0019% | >99.9% |
MS-MARCO MiniLM-L12-v2 | 0.0018% | >99.9% |
MS-MARCO TinyBERT-L2-v2 | 0.0015% | >99.9% |
MS-MARCO Electra-base | 0.0026% | 99.4% |
Ettin-reranker-150M | 99.9% | >99.9% |
BGE-reranker-base | 11.6% | >99.9% |
BGE-reranker-large | 0.20% | 99.8% |
Qwen3-Reranker-0.6B | 0.27% | >99.9% |
MS-MARCO MiniLM-L6 EN-DE | 0.12% | >99.9% |
mMARCO mMiniLM-L6-v2 | 0.37% | >99.9% |
SPLADE-cocondenser | 2.57 overlap | 24.83 overlap |
Jina-ColBERT-v2 | 0.36 avg cos | 0.57 avg cos |
Scores are normalized within each model for display. They're not comparable across families as one universal percentage.
Static view: Both passages at once
Use this for social, email, or print.
Query: what are the best aeo tools?
Scores below are within-model normalized values from the same run as the animated cards.
Passage A · product description “Peec is an AI answer engine optimization platform that tracks brand mentions…” | Passage B · direct shortlist “The best AEO tools in 2026 are Peec AI, Competitor A, Competitor B and Competitor C.” |
4.2 SPLADE: The transparent keyword machine
Example: naver/splade-cocondenser-ensembledistil (110M)
SPLADE (from Naver Labs Europe, 2021) is one of the most interpretable models for content diagnosis because its lexical contribution to the score can be decomposed exactly.
It takes a text and expands it into a weighted bag of vocabulary terms. Not just the words that appear, but the words the model thinks should be associated with the text. The query gets the same treatment. The relevance score is then just: for every term both sides share, multiply the two weights, and add them up. Measured lab data · auto-playing
SPLADE reading the Peec AI sentence
The exact math behind that score
Green rows are terms the query and the passage share (their product IS the score). The "triggered by" column shows which of the passage's actual words activated the term. Orange rows are the opposite: terms the model strongly associates with the query that the passage never activates.
The passage's word "platform" is doing heavy lifting: it triggered both "software" and "tools" for SPLADE, even though the passage never says "software." That’s vocabulary expansion working in your favor.
Now look at the orange rows: the model's two highest-weight query terms by far are pieces of "aeo" itself and the word "best," and the descriptive passage activates neither. Terms such as “ae” and “##o” are WordPiece fragments of "aeo", not separate missing concepts. SPLADE is telling you, in arithmetic you can verify by hand, exactly why this passage can’t win this query: it never says it is about AEO tools, and it never engages with "best."
SPLADE is fundamentally a very smart keyword system. It expands vocabulary, but it doesn’t deeply model whether a sentence answers a question. Use it as your explainability tool and your "what terms am I missing" tool, not as your final arbiter of quality.
Takeaway: Use SPLADE's missing high-weight concepts as editorial questions, "Does this passage genuinely address this facet?" Add a term only when the answer is yes and the supporting claim belongs in the section. Don’t read it as proof that inserting every expanded term, repeating a keyword, or maximizing a SPLADE score will improve rankings or citations. |
4.3 ColBERT: The word-matchmaker
Example: jina-colbert-v2 (137M, multilingual)
ColBERT (Stanford, 2020) sits between bi-encoders and cross-encoders, which is why it’s called "late interaction." It embeds every individual token of the query and every individual token of the document separately (so documents can still be indexed in advance, keeping it fast), then at query time each query token searches the document for its best-matching token. The score is the sum of these best matches. This mechanism is called MaxSim.
ColBERT reading the Peec AI sentence
The matchmaking table: Which passage word "wins" for each query word
Each query token picks its best partner in the passage. Green bar = strong partner found, short red bar = the query word is essentially unanswered.
Read the table like a matchmaking report. The query word "best" could not find anything better than "most" at a weak similarity; "tools" ended up paired with "helps". The average match quality per query word was 0.36 on a 0-to-1 scale, which is a weak result. On the direct-answer passage, the same model's average jumps sharply because "best" finds "best" and "tools" finds "tools." Like SPLADE, this decomposition is exact, not an approximation, so you can trust it for diagnosis.
One reading tip: ColBERT's raw score grows with query length (it’s a sum over query tokens), so never compare raw ColBERT scores across different queries. Instead, compare the average similarity per query token.
Takeaway: Inspect the lowest-quality query-token alignments and classify each as intentional or missing. A weak match for "pricing," "for small teams," or "2026" often reveals an unanswered decision criterion. Don’t read it as proof that every query word needs an exact-match copy. A strong semantic partner can be enough, and function words are rarely editorial priorities. |
4.4 LLM-based rerankers: The new generation
Examples: Qwen3-Reranker (0.6B/4B/8B, Alibaba, 2025), mxbai-rerank-v2 (built on Qwen2.5)
LLM-based rerankers are the newest branch, and they abandon BERT entirely. Take a decoder LLM, give it a prompt like "Query: ... Document: ... Does the document answer the query? Answer yes or no," and derive a relevance score from its output-token probabilities. Unless it has been calibrated for your data, this is still a model score rather than an observed probability of citation.
These models inherit everything the LLM learned about the world. They handle instructions ("prefer recent sources," "prefer official documentation"), long documents (32,000 tokens for Qwen3-Reranker versus 512 for classic BERT rerankers), 100+ languages, and code. Qwen3-Reranker ships in 0.6B, 4B, and 8B sizes and currently sits at or near the top of multilingual retrieval benchmarks.
You saw Qwen3-Reranker-0.6B's card above: a low normalized score on Passage A and a high one on Passage B. Those score bars are measured. We didn’t obtain reliable passage-token attribution from this implementation, so the word colors use a lexical visualization fallback to keep the animation readable. Judge this model by the controlled score comparison and validation labels first; treat the word colors as a readability aid, not a causal explanation of the decision.
They come at a cost as they’re 10 to 100 times heavier than a MiniLM, and their inner workings are the hardest to inspect of all four families. You trade transparency and speed for capability.
Takeaway: Use an LLM reranker when relevance depends on instructions such as recency, source type, audience, or comparison criteria, then validate those instructions on labeled examples. Qwen3-0.6B is the lightweight entry point; larger variants trade more compute for capacity. Don’t infer that its attention heatmap explains the decision or that sharing a decoder architecture makes it a replica of ChatGPT Search. |
Part 5: What is ChatGPT probably using?
What you’ll learn here: What is documented about ChatGPT-related retrieval, what behavior suggests, and what remains unknown. This section is about understanding the stack, not rewriting content yet. |
The evidence splits into three levels: what vendors document, what observable behavior is consistent with, and what remains unknown.
What is documented (high confidence)
OpenAI's File Search (the retrieval feature in the OpenAI API, used when you upload documents) is publicly documented. Documents are chunked (default 800 tokens per chunk with 400 token overlap), embedded with text-embedding-3-large, and at query time the system runs both a semantic (vector) search and a keyword search, then can apply a reranker to the combined candidates.
The API exposes ranker options ("auto" and dated versions) and returns per-chunk relevance scores between 0 and 1. You can also turn the reranker off. Independent tests with the reranker disabled are consistent with a classic Reciprocal Rank Fusion merge of the form w_e/(60+r_semantic) + w_t/(60+r_kw), using the common RRF constant of 60. OpenAI doesn’t say what the neural reranker is, but its existence and position in the pipeline are official.
ChatGPT's web search is documented (in OpenAI's own help pages) to use a mix of third-party search providers, with Microsoft Bing named among them, plus content from OpenAI's partners and its own crawling (OAI-SearchBot).
Google and Bing are two retrieval systems that can feed AI answers. They have both publicly described BERT-based or transformer-based ranking technology, and Google has also described passage-based ranking inside pages. That supports passage-level relevance as a real search concern. It doesn’t disclose the full source-selection stack behind ChatGPT or AI Overviews. Public Google search trial evidence has also shown reranking trained with click data in classic web search; neural AI-answer pipelines may combine relevance models with other signals.
What behavior suggests (medium confidence)
Observed citations are consistent with a passage-selection stage, but behavior alone can’t identify a specific architecture. Three patterns show up often:
1. Answer-bearing passages often beat passages that are only topically related. Our controlled passage comparison reproduced this across the measured models.
2. Lists, definitions, and comparison tables can perform well for matching intents. The mechanism is not a universal preference for formatting. These structures often contain extractable answers, named entities, and coverage for many prompt variants.
3. Citations frequently resolve to a specific section or claim. This fits passage-level retrieval or scoring, while page-level authority may still matter elsewhere.
What remains unknown
Which open model is closest to what ChatGPT runs internally? There isn’t enough public evidence to name one. What we can say is narrower:
The retrieval-side shape (hybrid keyword plus vector search, then rerank) matches the industry-standard pattern documented in OpenAI File Search and many open RAG stacks.
Modern production systems can use encoder cross-encoders, decoder LLM-based rerankers, custom models, or ensembles. Public behavior doesn’t tell us which.
Qwen3-Reranker is a useful open example of an LLM-based relevance judge. There is no evidence that OpenAI uses Qwen3-Reranker.
BGE and MiniLM are useful encoder baselines for your own tests. There is no evidence that ChatGPT uses those checkpoints.
Practical model-selection advice continues in Part 6.
Part 6: A practical reranker decision guide for AEO/GEO and SEO teams
What you’ll learn here: Which reranker to reach for depending on the decision. A content editor diagnosing missing concepts needs a different tool from an engineer ranking 100 passages under a latency budget. This is the executive “which to choose and when” guide. |
Choose by the decision you need to make, not by the largest parameter count or the most attractive benchmark headline. Agreement across a few different families is stronger evidence than success on one model; still validate against real retrieval and citation outcomes.
"I want to understand WHY my content scores low" (explainability)
Use SPLADE first, then ColBERT. SPLADE gives you the exact list of matched and missing terms with weights, which becomes a set of editorial questions. ColBERT shows which query concepts fail to find a strong partner in your text. Cross-encoder attention maps look impressive but are partly artifacts; treat them as hints, not evidence. Prefer exact score components before approximate attention.
"I need the most accurate ranking, cost is secondary"
Benchmark an LLM-based reranker such as Qwen3-Reranker-4B or 8B, alongside a strong managed rerank API if available. These models can handle longer inputs, multiple languages, and ranking instructions. The 0.6B version is the more accessible local baseline. "Most accurate" must still be measured on your query types and relevance labels.
"I need speed on cheap hardware" (CPU, high volume, low latency)
Start with ms-marco-MiniLM-L-6-v2 (23M) locally, or a low-cost rerank API if you don’t want to self-host. MiniLM is a common workhorse with strong quality for its size and can run on a laptop CPU. Managed options such as Jina Reranker are often priced around $0.00001 per request, which can be cheaper than GPU time for bursty workloads. Actual throughput depends on hardware, sequence length, and batching. Test L12 when the quality gain on your labels justifies its extra latency. Use TinyBERT only as a coarse pre-filter.
"My content is not in English"
Use BGE-reranker-v2-m3, or Qwen3-Reranker if you have the compute. Both are multilingual (XLM-RoBERTa lineage and Qwen lineage respectively). Don’t use the ms-marco English models on non-English text and trust the numbers; they degrade quietly rather than failing loudly. For broad multilingual work, the mMARCO family (trained on machine-translated MS MARCO in 13+ languages) is the classic option.
"My documents are long" (guides, documentation, whitepapers)
Test a ModernBERT-generation reranker (Ettin family) or an LLM-based one. These are the long-context options. Classic BERT-family rerankers commonly accept only up to 512 tokens, so later text is truncated unless you split it. ModernBERT/Ettin and LLM rerankers accept much longer inputs. Even then, test smaller passages because production retrieval systems may still return sections rather than the whole document. Aim for locally understandable sections, not robotic repetition in every chunk.
"I'm building a two-stage search pipeline"
A strong baseline is Stage 1: BM25 plus a bi-encoder, merged with RRF. Stage 2: a reranker on a measured candidate depth such as the top 50 or 100. This resembles the documented hybrid shape of OpenAI File Search and many public RAG stacks. Public Google materials have discussed smaller candidate depths in the roughly 20–30 range for some ranking stages, so treat 50–100 as a common RAG default rather than a universal law. Tune depth with recall and latency tests. MiniLM-L6 is a speed baseline; BGE or Qwen are quality candidates to benchmark.
"I want to check my content the way AI search sees it"
Run a few models from different families and look for agreement. One model's score is an opinion but five models from three architectures agreeing is a diagnosis. When MiniLM (BERT distilled), BGE (XLM-RoBERTa), SPLADE (sparse), ColBERT (late interaction), and Qwen3 (LLM) all score your passage low for your target query, the problem is your passage, not the model. The Ettin dissent you saw above is the exception that proves the rule: single-model scores can mislead you in either direction.
Quick reference table
GEO/AEO/SEO question | First model to test | What the output supports |
Explain a low score | SPLADE | Exact term-level decomposition, directly editable |
See which query words you miss | ColBERT (jina-colbert-v2) | Per-word alignment table |
Best accuracy, have GPU | Qwen3-Reranker 4B/8B | Benchmark leader, long context, instructions |
Best managed quality, API budget | A current managed rerank API | Production ranking without self-hosting; validate version and cost |
CPU, high-volume scoring | ms-marco-MiniLM-L-6-v2 | Fast baseline; measure throughput on your hardware |
Coarse pre-filter | ms-marco-TinyBERT-L-2 | Very small model for triage, not final editorial decisions |
Multilingual content | BGE-reranker-v2-m3 | Broad language coverage; validate each locale |
Long documents | Ettin / ModernBERT family | More context before truncation; still test passage windows |
Content diagnosis | A few different families | Agreement raises confidence; disagreement reveals review cases |
Part 7: What this changes in an AEO/GEO and SEO content strategy
What you’ll learn: Translate the model mechanics into content decisions while keeping retrieval, reranking, and citation generation separate. |
Finding 1: "About the topic" and "answers this query" are different tests. The descriptive passage received very low normalized match scores from the strict models, while the direct-answer version moved above 99 percent on their own transformed scales. SEO decision: give each query-targeted section a clear answer-bearing sentence, then support it with evidence and nuance. It doesn’t prove that copying the query verbatim creates a 99 percent citation chance.
Finding 2: Why listicles often appear in citations doesn’t require a universal "list bias." A good listicle about "best AEO tools" names several entities, covers several attributes, answers comparison sub-queries, and contains passages that can satisfy multiple fanout queries. It also has the answer shape the query requests: a shortlist. Those properties expand retrieval opportunities and make extraction easier. The same logic predicts a definition for "what is AEO" and ordered steps for "how to track AI citations." The useful principle is intent-to-answer-shape alignment, not "turn every page into a list."
What to do: Map each target query to the information shape it requires. Use a shortlist for selection queries, criteria and evidence for comparisons, a concise definition for entity questions, and ordered actions for procedural queries. Then make sure the relevant section contains that answer before the supporting detail. |
Query intent | Useful answer shape | What makes the passage citeable |
What is X? | Concise definition | Clear entity name, category, distinguishing facts, and sourceable support |
Best X for Y | Shortlist with criteria | Named options, inclusion criteria, trade-offs, audience fit, and current evidence |
X vs Y | Like-for-like comparison | The same criteria applied to both options, with differences stated directly |
How to do X | Ordered procedure | Prerequisites, steps, expected outcome, and exceptions |
Price, date, or current status | Direct fact with timestamp | Exact value, effective date, primary source, and update context |
Insight for content SEO: The cheapest high-impact edit is usually one direct-answer sentence in the first two lines of the relevant section, followed by criteria, entities, and evidence. Don’t rewrite the whole page until that sentence exists. |
Finding 3: In our counterfactual test, missing concepts mattered more than shuffled word order for most tested models. We ran the model again with a scrambled query and with an unrelated control query. SEO decision: cover the decision criteria and entities the intent genuinely requires. It doesn’t prove that syntax, phrasing, and sentence quality never matter across all queries or models.
Finding 4: Passage quality can become the local bottleneck. Documented RAG products such as File Search retrieve chunks, while web systems can combine page-level and passage-level signals. SEO decision: make important sections locally understandable, name the subject explicitly, and keep the answer close to its evidence. It doesn’t prove that authority, links, freshness, or whole-page quality stop mattering.
Finding 5: Cross-encoder attention is not an editorial instruction list. About a quarter of BGE-large's measured attention landed on punctuation and structural tokens in the example. SEO decision: use attention to form hypotheses, then use occlusion, counterfactual rewrites, SPLADE terms, or ColBERT alignments for stronger evidence.
Finding 6: Model families can disagree sharply while agreeing on a relative edit. Ettin was permissive on the descriptive passage while strict models rejected it, yet the direct-answer rewrite remained strong across those models. SEO decision: compare versions within each model and look for directionally consistent improvements. Never average raw scores from different models into a fabricated universal "GEO score."
Part 8: A practical workflow for AEO/GEO and SEO teams
Who this is for: Use the workflow that matches your role. A content editor shouldn’t need to deploy Qwen, and an AEO/GEO analyst shouldn’t report one reranker score as AI visibility. |
GEO/AEO master to-do list Use this as a working checklist. Complete the diagnosis items before the writing items. ☐ Pick 10 money prompts and label answer category + expected answer shape for each. ☐ Extract your current passage and the winning competitor passage for each prompt. ☐ Add or sharpen one direct-answer sentence in the first two lines of every target section. ☐ Name the subject explicitly in each section; replace vague “it / this tool” references. ☐ Cover missing decision criteria (audience, price, year, alternatives) only when they belong in the answer. ☐ Run a few different families (one cross-encoder + SPLADE or ColBERT; add Qwen when intent is complex). ☐ Track three metrics separately: retrieval exposure, candidate strength, citation/mention rate. ☐ Re-test after each rewrite against the same competitor passages and the same model versions. ☐ Segment reporting by intent. Don’t average brand, definition, and “best X” queries into one score. |
Start with the same seven steps regardless of tool:
Define the query cluster and intent. Keep brand, definition, comparison, selection, and how-to queries separate.
Collect passage-level candidates. Save your relevant section and the passages that are currently cited or consistently outrank it.
Locate the likely failure stage. Distinguish discovery/retrieval, passage relevance, and final source use before proposing an edit.
Run a few different model families. Use one cross-encoder plus SPLADE or ColBERT; add an LLM reranker when the intent is complex.
Read the strongest available evidence. Prefer exact term contributions and token alignments, then controlled rewrites; treat attention as a hypothesis.
Make the smallest useful rewrite. Add the missing answer, criterion, entity, qualifier, or evidence without changing unrelated prose.
Validate twice. Re-score against the same competitors, then monitor real retrieval, mention, and citation outcomes over time.
Then apply the role-specific checks below.
When writing or editing content:
For every query you target, write one sentence that answers it directly, and place it in the first two sentences of the relevant section.
Make each section self-contained: full names instead of pronouns, the topic restated, one sub-question answered per section.
Use question-form subheadings for sections that target questions.
Cover concepts, not phrasings. Use SPLADE or another term-gap tool to find concepts the passage misses, then decide whether each belongs in the answer. Repetition is not a substitute for coverage or evidence.
Prefer answer-first structure. Elaboration after the answer, never instead of it.
When reviewing content:
Test passages at several plausible window sizes and also preserve page-level context. You don’t know the proprietary system's exact chunker.
Use at least two model families. Trust their agreement, not any single number.
Treat sigmoid-transformed logits as normalized model scores unless you have calibrated them. They’re not observed citation probabilities.
Check your best-performing competitor's chunk for the same query and compare structures. The difference is usually one direct-answer sentence, not five hundred words of extra content.
When reporting AEO/GEO performance:
Separate retrieval exposure, candidate/reranker strength, and final citation or mention rate. A single visibility score hides the failed stage.
Segment by query intent. Brand, definition, comparison, "best X," and how-to queries expect different answer shapes.
Report model version, query set, locale, date, and passage window. Without them, a reranker score is not reproducible.
Use competitor and rewrite controls. The most useful output is "version B outranked version A consistently" not "this page is 83 percent GEO optimized."
When building tools or pipelines:
Use a two-stage baseline: wide retrieval with BM25 plus embeddings, followed by reranking at a candidate depth chosen through recall and latency tests.
Benchmark MiniLM-L6 for speed, BGE-v2-m3 for multilingual work, Qwen3 for instructed or complex relevance, and SPLADE for lexical explanations.
Log per-chunk scores over time. Reranker score regressions after a content edit are the earliest warning you’ll get that you broke a winning passage.
Part 9: Should you write in "reranker-friendly chunks"?
The content-strategy question answered here: Should an SEO writer change prose to fit imagined AI chunk boundaries? Usually no. The useful goal is local clarity across plausible passage windows, not a fixed token count. |
There is a growing piece of advice in AI-SEO circles to write your content in reranker-friendly chunks. Like most advice that spreads fast, it contains a real insight wrapped in a myth. Let’s separate them, because doing this wrong wastes your time and can make your content worse for humans.
The myth part
Myth: You can control the chunks.
You can’t. Every system chunks differently. OpenAI's File Search cuts at 800 tokens with a 400 token overlap by default. (This is not web search; it is vector-database File Search.) Other systems cut at 256 or 512 tokens, some cut at sentence boundaries, some at fixed character counts, and web search pipelines often work from whatever the crawler and snippet extractor produced. Your carefully designed 800-token sections will be sliced mid-paragraph by one system and merged with neighbors by another. Writing to match one specific chunker is optimizing for a target you can’t see and that changes without notice.
Myth: Chunk boundaries are fatal.
They mostly are not, because of overlap. When systems chunk with 50 percent overlap (as OpenAI does by default), every piece of your text appears in two windows. A good answer sentence that gets cut in one window survives intact in the next. Overlap exists precisely because engineers know arbitrary cuts damage passages, and they compensate for it.
Myth: Every paragraph must be a self-contained answer machine.
If you rewrite an entire article so every paragraph opens with a definition and restates the full context, you get robotic text that humans stop reading. Humans still assign the links, the mentions, and the brand searches that get you into the retrieval pool in the first place. Content that wins the reranker but loses the reader is a net loss.
What helps instead
Local self-containedness is what actually helps. Whatever window a system cuts, that window is judged alone. So the useful property is not "my sections match the chunker" but "any few-hundred-word window of my page makes sense on its own."
In practice, name things explicitly instead of using "it" and "this tool" across paragraph boundaries, keep an answer and its subject in the same neighborhood, and don’t build arguments that only work if the reader (or the model) saw three sections earlier. This helps under every chunking scheme simultaneously, which is exactly why it’s worth doing.
One direct-answer sentence per target question is the other move that helps. Our measurements show this is the single highest-leverage edit that exists. It doesn’t require restructuring anything. Instead, it requires adding or sharpening one sentence in the section that targets each query.
When chunk-conscious writing pays off
Long informational content (guides, documentation, comparison pages, FAQs). These get chunked aggressively, are retrieved for many different sub-questions, and each section competes alone. This is where self-contained sections and question-form subheadings earn their keep.
Content targeting many distinct queries on one page. Each target query needs its own answer-bearing passage. Structure is your friend here.
Anything you want cited by RAG products (AI assistants reading your docs, support bots, File Search style systems), because passage retrieval and scoring are central there. Metadata, permissions, freshness, and source filters can still matter.
When it is unnecessary (or actively a waste of effort)
Short pages. A 400-word page fits in one or two chunks no matter what. Chunk-conscious structure changes nothing; the whole page is the passage. Focus on answering the question well once.
Brand and navigational queries. Explicit brand terms make retrieval easier, so passage structure is usually less of a bottleneck than it is for generic discovery queries. Accuracy and source identity still matter.
Content that wins on uniqueness. Original data, quotes, and facts create a stronger reason to retrieve and cite the primary source. They don’t guarantee citation, but they usually deserve more investment than token-count micro-optimization.
Pages whose job isn’t to be cited. Landing pages, product pages, conversion pages. Their job is converting humans. Sacrificing persuasive flow for chunk structure on these pages is optimizing the wrong metric entirely.
Micro-optimizing chunk sizes. Rewriting sections to hit specific token counts, splitting paragraphs to imagined boundaries, and duplicating context into every paragraph. Overlap and multi-window retrieval can absorb some boundary damage. Past local clarity and answer completeness, returns diminish quickly.
The one-line version: Don’t write for the chunker. Instead, write so that any window of your page can survive being read alone. Add one direct-answer sentence per target question. Spend the saved effort on being the source worth citing, because the reranker can only choose among the passages that exist. |
Glossary
Attention: The mechanism inside transformers where every word weighs how much every other word matters to it. The core of BERT and GPT alike.
Attention sink: The tendency of BERT-family models to dump unused attention on punctuation and special tokens. Makes attention visualizations partly misleading.
BM25: The classic keyword-scoring formula used by search engines for decades. Still everywhere, usually alongside vector search.
Bi-encoder: Embeds query and document separately, compares vectors. Fast, less precise. Used for retrieval.
Chunk: The unit of text (a few hundred tokens) that retrieval and reranking actually operate on. Your real competitor in AI search.
Cross-encoder: Reads query and document together, outputs one relevance score. Slow, precise. Used for reranking.
[CLS] token: A special summary token BERT-family models prepend to the input; the classification head reads the final score from it.
Distillation: Training a small model to imitate a big one. How MiniLM and TinyBERT were made.
Embedding: A list of numbers representing a text's meaning, so similar meanings are near each other mathematically.
Late interaction: ColBERT's approach: embed tokens separately (fast indexing), match token-to-token at query time (precision).
Logit: a model's raw output before a sigmoid or softmax. Its sign and scale only have meaning within that model and task.
MaxSim: ColBERT's scoring rule: each query token takes its best-matching document token; sum the similarities.
MS MARCO: Microsoft's dataset of real Bing queries with human relevance labels. The training ground of almost every classic reranker.
Masked language modeling: BERT's fill-in-the-blank pretraining game.
Reranker: A model that re-scores a shortlist of retrieved passages against the query. The citation gatekeeper.
RRF (Reciprocal Rank Fusion): A simple formula for merging multiple ranked lists, rewarding items ranked high in several of them.
Sigmoid: A function that maps a number to 0-to-1. The result looks probability-like, but it is not a calibrated citation probability unless validated as one.
SPLADE: A sparse model that expands texts into weighted vocabulary terms; its scores decompose exactly per term.
Sources and further reading
• Devlin et al., "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" (Google, 2018)
• Google, "Understanding searches better than ever before" (BERT in Search announcement, October 2019)
• Microsoft Bing, "Bing delivers its largest improvement in search experience using Azure GPUs" (transformer ranking on all queries, November 2019)
• Nguyen et al., "MS MARCO: A Human Generated MAchine Reading COmprehension Dataset" (Microsoft, 2016)
• Liu et al., "RoBERTa: A Robustly Optimized BERT Pretraining Approach" (Facebook AI, 2019)
• Conneau et al., "Unsupervised Cross-lingual Representation Learning at Scale" (XLM-RoBERTa, 2019)
• Clark et al., "ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators" (2020)
• Clark et al., "What Does BERT Look At? An Analysis of BERT's Attention" (2019), source of the attention sink findings
• Wang et al., "MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression" (Microsoft, 2020)
• Khattab and Zaharia, "ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT" (Stanford, SIGIR 2020)
• Formal, Piwowarski, Clinchant, "SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking" (Naver Labs Europe, 2021)
• Warner et al., "Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder" (ModernBERT, 2024)
• Qwen team, Qwen3-Embedding and Qwen3-Reranker model cards and technical report (Alibaba, 2025)
• BAAI, BGE reranker model cards (bge-reranker-base, large, v2-m3)
• OpenAI, File Search documentation (chunking defaults, hybrid search, ranker options)
• OpenAI Help Center, documentation on ChatGPT search providers
• Google, "How AI is powering a more helpful Google" (passage ranking, 2020-2021)
