From a monitored question to an evidence-backed AEO result
Understand question portfolios, model tests versus AI search, provider responses, evidence extraction, and the eligibility of a measurement.
On this page
Define the thing being measuredQuestion portfolios and objectivesModel tests and AI search are different surfacesCollection and checkpointsEvidence extractionSix independent observationsWhat a failed or incomplete run can still showReproduce a finding before actingDefine the thing being measured
AEO measures your presence in answers to a defined set of questions. Configuration includes the business identity and aliases, website, business type, topics, competitors, country, language, providers, models, and measurement method. These settings determine the observation population. The module is not a crawler of all questions asked by every user of an AI product.
Question portfolios and objectives
A useful portfolio separates informational, comparison, and purchase-oriented questions. A retailer needs evidence of recommendations to buy from that retailer; a manufacturer may instead need its brand or product recommended. Exact-product checks, branded confirmation questions, neutral discovery questions, and optional experiments answer different research questions. Store question versions and their objectives rather than silently treating every wording as an interchangeable baseline. Generated proposals need review for relevance before measurement.
Model tests and AI search are different surfaces
A model test records an answer from a configured model request. AI-search collection additionally depends on the supported search surface and its available grounding or citation output. Neither necessarily reproduces the consumer interface, personalisation, location history, or a private conversation in the provider's own app. A change of method, model, search configuration, or locale can change results even if the question text is unchanged.
Collection and checkpoints
Workers dispatch the configured requests and retain returned answers with their run and question context. Provider failures and refusals need separate handling from a completed answer that gives no recommendation. Collection progress and extraction progress are different counts. Checkpointing saved outputs allows supported retries to continue analysis without unnecessarily buying the same answer again.
Evidence extraction
The assessment separates literal identity mentions, positive brand or product recommendations, retailer recommendations, explicit ranks, shop links, and visible citations. Positive claims need supporting passages that match the saved answer. An analyst cannot establish a recommendation merely because a familiar name appears in a URL or in a negative comparison. Ambiguous identities, truncated output, or unsupported passages can leave an assessment unresolved instead of forcing a false negative.
Six independent observations
These observations are related but are not substitutes for one another. A page may show several because they answer different business questions.
| Observation | Evidence needed | Common mistaken inference |
|---|---|---|
| Retailer mentioned | The shop identity is present in an assessable answer | The shop was recommended |
| Brand mentioned | An applicable own or stocked brand identity is present | The retailer carrying it won the recommendation |
| Product recommended | Positive recommendation of the identified product | The product is in stock in your shop |
| Retailer recommended | Positive evidence of buying from the identified shop | Any manufacturer link counts as a shop recommendation |
| Retailer linked | A verified shop-domain URL in the answer | The link proves first-choice preference |
| Retailer cited | Assessable visible citation output pointing to the shop | Every retrieved search URL was visible to the user |
What a failed or incomplete run can still show
A run can contain usable answers alongside failures. A metric must state its eligible sample, unresolved assessments, and coverage rather than presenting the requested answer count as proof that every output was assessed. Re-extracting a saved answer may change its classification; it is not a fresh observation of the external model. Formula and extraction versions matter when comparing old and new displays.
Reproduce a finding before acting
Open the question, original answer, relevant excerpt, provider/model, method, country, language, and timestamp. Check whether the name refers to your shop, a manufacturer, a similarly named business, or a stocked brand. Then inspect the exact numerator and denominator used by the card. Keep this evidence with any action you create.
Still need help?
Tell us what you expected and what happened.