Methodology
Where every number on this product comes from.
Every figure AnswerLift shows counts something that actually happened: an answer a paid API returned, or an answer a person read with their own eyes. There is no estimate and no computed score anywhere in the product, and every count is made in code from stored text, never by a model. This page states the method in enough detail that reading the source code afterwards holds no surprise.
An earlier version of this product showed a score out of 100 derived by arithmetic from the brief the customer typed. It measured nothing, and a number in that position is indistinguishable from a measurement to the person reading it. It has been removed entirely rather than relabelled.
Part one
What is queried, and how.
A Sprint asks up to 50 buyer questions, agreed at kickoff, of a fixed panel of answer engines. The question never names the client’s brand: the measurement is whether the brand comes up unprompted. Tracked prompts can also be re-asked on a cadence, from every 6 hours upward, so a second round shows whether answers moved. The panel, in the order every report lists it:
The model behind ChatGPT
OpenAI GPT API (web search)
OpenAI's current mid-tier GPT model, asked through OpenRouter with OpenAI's own web search tool switched on, because chatgpt.com searches the web for this kind of question. The answer is stored verbatim with the exact model id and every source the search returned.
The model behind Claude
Claude API (web search)
Anthropic's Claude Sonnet model, asked through OpenRouter with Anthropic's own web search tool switched on, as claude.ai does. Stored verbatim with the model id and the sources the search returned.
The model behind Gemini
Gemini API (assistant panel)
Google's Gemini Flash model, asked through OpenRouter from model knowledge, with no web search. The Gemini app grounds on Google Search, and that grounding is not available through this route; a different search engine under a Gemini label would misdescribe the answer, so we do not add one.
The model behind Perplexity
Perplexity Sonar API
Perplexity's Sonar Pro model, which always searches the web before answering. Stored verbatim with the model id and every source it cited.
DeepSeek, Qwen, Kimi, GLM and MiniMax
Five open-API models (assistant panel)
Asked from model knowledge with no web search, through OpenRouter, from hosts outside China. Together they carry well under one percent of UK AI-chat use; they are kept as a comparison set, and listed after the four above in every report.
An API answer is not the consumer product’s answer. A model API is a separate product from the assistant app built on top of it: different system prompt, different memory, often a different model version. Every record collected this way is stored and displayed under the API’s name — “OpenAI GPT API (web search)”, never “ChatGPT” — and carries the exact model id that answered it. The engines asked with web search measure whether the brand is named and whether its own pages are among the sources retrieved; the engines asked from model knowledge measure whether the brand is one of the names a model produces unprompted. Both are real, neither substitutes for the other, and every record says which it is.
A run that fails, or that finds no API key configured, is stored as a failed or skipped run and shown as one. If an engine’s primary model is down, the next model from the same vendor answers and the record says it was a failover; an engine is never answered by a different vendor’s model. If no gateway key is configured on the deployment, collection is simply off and every surface says so.
Deliberately not done
- The consumer ChatGPT, Claude, Gemini and Perplexity apps are never scraped, automated, or driven by a headless browser. We ask the models behind them through official APIs. The apps add their own system prompts, memory, personalisation and model routing, so a person asking the same question there may see a different answer. Where the app itself matters, a person opens it and records what they saw.
- Microsoft Copilot is not measured. It has no API of its own; it answers with OpenAI models plus Bing retrieval, and the OpenAI GPT row is the nearest measurement we can honestly make.
- Grok and Meta AI are not queried. Together they are under one percent of UK AI-chat use.
- Google AI Overviews is never collected automatically. There is no route to it we are willing to use. A person may record what they saw in a Google result by hand, and that record is labelled as a hand observation.
- A URL a model writes from memory is never stored as a source. Only URLs a web search returned as structured citations are recorded as cited sources; the knowledge-mode engines therefore have none.
- No estimated, modelled or computed score exists anywhere in the product. There is no fallback number when collection is off.
- Nothing is inferred about a prompt that was not asked, or a date on which nothing ran.
Part two
What is recorded, and how it gets verified.
Collected answers and hand-recorded observations land in the same ledger, in the same shape, and pass the same gate. A person can also bulk-import evidence they already gathered as CSV — up to 500 rows at a time, validated row by row, with a preview before anything is written and no partial imports. That is an entry path for human-gathered evidence, not a second collection path.
| Recorded on every observation | What it means |
|---|---|
| Prompt | The exact question that was asked, stored verbatim. |
| Prompt version | A normalised form of the prompt keys a version counter, so re-asking the same question later appends v2, v3 … instead of overwriting v1. |
| Engine or provider | For collected records: the API name and the model that answered. For hand-recorded records: which consumer interface a person looked at. The two are never conflated. |
| Answer text | For collected records, the provider's answer stored verbatim, so a reviewer checks the actual response and not a summary of it. |
| Cited sources | Every URL the engine's own web search returned as a source for that answer. Empty for engines asked from model knowledge. |
| Source label and URL | What the snapshot is, and where it can be re-checked. |
| Captured at / observed at | When the source material was captured, and when the answer was seen. |
| Collection method | Manual observation · imported snapshot · API collection. |
| Brand classification | Not mentioned · mentioned but not cited · cited. Mention and citation are recorded separately and never conflated. |
| Classification basis | Whether a person made the call, or it is a provisional machine match awaiting review. |
| Confidence | High, medium or low. Machine-provisional records always start at low. |
| Volatility | Stable, emerging or volatile — derived only from repeated observations of the same prompt on the same provider, never from a single answer. |
| Evidence note | What the record establishes, in plain words, including any qualification. |
| Reviewer and review time | Set only by the human QA step. |
The provisional call, and why it is not a finding.
When an answer is collected, the software makes a first-pass call: if any cited source is on the client’s own domain the record is marked cited; if the brand name appears in the answer text with no such citation, mentioned but not cited; otherwise not mentioned. That is a string match, and string matches are wrong in both directions — a brand named “Apex” matches text that has nothing to do with it, and a brand described without being named is missed.
So the provisional call is stored as exactly that: flagged machine provisional, at low confidence, with QA status pending, and with the full answer text saved so a reviewer checks the response rather than the guess.
The QA gate.
A person reviews the record, enters their own name, and sets it verified or needs review. The reviewer name and time are stored on the record. The gate is enforced where the data is read, not in the page markup: the public share-report query returns only verified observations, so a pending record cannot reach a client screen, and a QA decision never rewrites the snapshot it was made about.
If nothing has been verified yet, the client report says so in those words. It does not fall back to a projection, a placeholder or a score.
Volatility, and why it is usually blank.
Volatility is derived only from the classifications already recorded for the same prompt on the same provider. Fewer than two prior records and it stays “emerging”; three or more that agree makes it “stable”; any disagreement makes it “volatile”. A single answer never produces a volatility claim.
AI-crawler logs.
A customer can import a summary of their own server or CDN logs — up to 5,000 rows, by date, bot, path and status — covering crawlers such as GPTBot, PerplexityBot, ClaudeBot and Google-Extended. AnswerLift does not fetch or obtain those logs itself. A crawler hit records that a bot fetched a page: it is not a citation and not a mention, and it is labelled that way everywhere it appears.
Part three
What is software and what is a person.
The software does
- Ask each agreed prompt of every engine on the panel, through paid official APIs.
- Store each answer verbatim with its engine, model, search method, cited sources and time.
- Count, in code, which engines named the brand, where, and who was named instead.
- Make a provisional classification for a reviewer to check.
- Derive volatility from repeated observations only.
- Enforce the human-QA gate on what can reach a client report.
- Render, export and expire the client report.
A person does
- Choose the prompts and the cadence.
- Ask questions in the consumer interfaces, where no API exists.
- Read every collected answer and confirm or correct the classification.
- Sign off every record before a client can see it.
- Write the interpretation and the prioritised remediation work.
- Export and supply the crawler logs from their own servers.
Concretely, a Visibility Sprint covers 1 brand in 1 market, briefed with 3–5 priority topics. Prompts tracked and reviewed: At least 10, up to the 50 per audit the product enforces — the working set is agreed at kickoff. Delivery: 20 business days from kickoff. Work performed by: AnswerLift — a named AnswerLift reviewer reads every stored answer and signs off each record before it can reach your client. Provider API usage costs: Included in the fee — collection runs on AnswerLift's own gateway accounts under a fixed spending ceiling per Sprint, and provider usage is never billed on to you. Delivery and the reviewer are commitments we make; the 50-prompt ceiling and the provider-cost position are properties of the software described above.
What this method cannot tell you.
- It cannot tell you exactly what the consumer ChatGPT, Claude, Gemini or Perplexity apps say to a given person. Records from the APIs describe the APIs, asked the same way every time; records from the consumer apps describe one person’s session at one moment.
- It cannot measure Copilot or Google AI Overviews automatically.
- It cannot give you a share of voice or a rank. Those need a defined denominator across a sampled prompt universe, and this product samples only the prompts you choose.
- It cannot tell you whether a change you made caused a change in an answer. There is no controlled before-and-after here, and answers move on their own.
- It cannot promise citations, placement, traffic or revenue, and nothing in the product should be read as doing so.
Still to build. Collection today is the honest core: a nine-engine panel, verbatim storage, provisional classification checked in code, and the QA gate. Not yet built, and not claimed: competitor share-of-voice across a sampled prompt universe, automated alerting on classification changes, and any coverage of Copilot or Google AI Overviews. This page will be rewritten the day any of that ships.
Still deciding?
See what the Sprint delivers.
The engines, models and search modes are defined in convex/answerPanel.ts; collection runs in convex/sprintRun.ts and convex/collection.ts. The QA gate is in convex/audits.ts, in the share-report query.