Guide · Measurement
Compare businesses fairly
Compare businesses within a defined AI answer sample, using the same questions and a clear counting rule.
Decide the comparison first
Choose alternatives that fit the customer decision. Explain the group rather than assuming familiar competitors are the only businesses an answer might name.
Not every business named in an answer is a direct rival.
Include the businesses customers actually choose between, not only the largest names. A national brand and a local independent may not compete for the same customer at all.
Write the customer question the comparison serves at the top of your notes. It keeps the group and the prompts tied to one purpose.
Keep the group fixed for the period. Adding a bigger brand halfway through changes the comparison even if your own recordings stay the same.
Your evidence is the saved answers. A name appearing in an answer isn't permission to copy or collect a competitor's website content.
Keep the group small enough that someone can read each answer that names its members. A long list of names you never read becomes a count without context.
Hold the prompt set steady
Keep the questions, location and collection method consistent across the comparison. Record a changed condition before interpreting a difference.
If conditions differ, label the records separately. Don't pool an API response with a consumer app recording without saying so.
Choose prompts a customer would ask before they know any names. Mix broad questions with specific needs, such as a service in a place or with a constraint.
Leave failed recordings out of the successful answer totals, but report them alongside so coverage is visible.
If one period has fewer completed checks, say so. A percentage means little until readers know how many checks lie behind it.
Leave out prompts that name one of the businesses. A prompt that asks about your business by name favours you, and tells you nothing about the comparison.
Pick a period long enough to hold several runs, and keep it the same for each comparison. Compare like with like, such as one month against the next.
Record the conditions that change answers
Answers depend on more than the wording of the prompt. Location, the product used and the way the answer was collected can all change what comes back.
OpenAI's API guide lets a developer pass an approximate user location, by country, city, region or time zone, to refine web search results. Anthropic's web search tool has a similar setting.
Google says AI Overviews may use different models and techniques, so their responses and links vary. Treat them as two separate surfaces, not one.
Google also describes a query fan-out technique, where a system runs several related searches to build one response. The links behind an answer can come from searches nobody typed.
So record the location setting, the product or interface, and whether you were signed in. Two answers collected under different settings aren't a fair pair.
Report each assistant separately before you combine them. A total across assistants can hide one where your business is never named.
Show counts with their context
If you're mentioned in four of ten recorded answers, show both numbers and say which ten answers they are.
Explain missing or failed captures. A changed total can make a percentage look better when there are no more mentions.
Share of voice is a comparison of how often each business was named, under a stated counting rule and set of businesses. Always show which businesses and answers were included.
Repeat each prompt before you compare
The same prompt can return a different answer on a different day, or on a second run. One recording per business is a snapshot, not a pattern.
Record each prompt more than once across the period, under the same conditions. Then compare how often each business was named across all the runs.
Space the runs across the period, so one unusual day doesn't decide the result.
Report the number of runs beside every figure. Named in three of five runs means something different from named once in a single run.
If a business is named in one run only, treat it as a lead to look at rather than a finding.
Some answers decline to name any business, or suggest checking a directory. Count these as answers with no business named, and keep them in the total.
Read what the answers say
Compare what the answer says about each option. Being named alongside an unavailable service is different from being recommended for the stated requirement.
Use the records to choose a prompt or page to look at. A small sample doesn't tell you about the whole market.
Note anything an answer gets wrong about your own business while you read. A comparison can be where an outdated fact first comes to light.
Read the answers in full before you look at the counts. The counts make more sense once you've seen what lies behind them.
Check your own business facts against your records. Don't state something about another business as fact unless you've verified it.
Keep descriptions apart from scores. Say what you counted and what needed a person to read the answer.
Sentiment is the tone of a description, such as positive, neutral, negative or mixed. If you label it, write the rule down and keep the passage beside every label.
Confirm each name is the business you mean
Check names that could refer to more than one organisation. Keep an uncertain identity unresolved instead of assigning its mentions to a preferred competitor.
Check the clues in the answer before you count the mention. The town, the service, a linked website or a phone number can confirm or rule out a match.
Write down the names that count as each business, such as a short name or the name with Ltd. Write down the names that don't, such as a similarly named firm elsewhere.
Treat branches of a chain with care. Decide whether you're comparing the brand or one branch, and count every business on the same basis.
If you can't confirm a match, record the mention as unresolved rather than guessing either way.
Record when a business changes its name or merges with another. Link the old and new names in your notes, so the history still adds up.
Treat the order of a list with care
Record explicit order where an answer presents an ordered list. Do not turn paragraph order or one appearance into a persistent search ranking.
If you record position, write down the rule. Count only answers that give a list, and note where each business sat in each one.
An answer sometimes states its own basis for the order, such as distance or alphabetical order. Note that basis where the answer gives one.
Keep list answers apart from answers that name a business in passing. A name inside a sentence of advice isn't the same as a place in a list.
Report the spread as well as any average. An average position can hide a business that came first in some answers and was missing from the rest.
Compare the sources behind each business
Many answers link to sources. A citation is a link attached to an answer, and the linked page may support only part of what the answer said.
Record the websites linked in answers that name each business. Note which websites are linked for a rival and never for you, and what kind of site each one is.
These lists may include directories, review sites, industry bodies and local news. Each is a place to check your own details, not proof of why an answer named someone.
A source linked for all the businesses in the group tells you less than one linked for a single business.
Open only public pages, and read rather than copy. A linked page shows what an answer may have drawn on, not what you should publish.
Repeat the source check in each period. A website linked often in one month may drop out the next, and another may take its place.
Write the comparison up so others can check it
Describe the finding within the limits of the recorded sample.
Include the prompt set and its version, dates, assistants, collection method and number of recordings. Add the businesses, why each was included and the counting rule.
Show failed or missing recordings beside the totals, with short excerpts that illustrate the main differences. Mark any illustrative example as fictional.
Store the raw answers with the write-up. A summary without the answers behind it can't be checked later.
A comparison made for planning should stay internal. If you ever publish one, state only what the records show and have the facts checked first.
Keep earlier versions of the prompt set when you expand the review.
Treat a change as a cue to look at the answers. It doesn't tell you why the movement happened or how many customers chose a business.
Before you read meaning into a change, check whether anything in the method moved. A new prompt, a new assistant or a different location setting can explain it.
Compare your own results period against period as well as against rivals. Your own trend uses the same prompts and settings, so it's less affected by who else was named.
A worked example
Business A and Business B appear in the same prompt set.
| Detail | Observation | Review or next step |
|---|---|---|
| Business A | Named in two of four complete answers. | Keep the four answers to hand for review. |
| Business B | Named in three of the same four answers. | Describe this sample, its dates and its limits. |
The gap is what showed up in four recordings. It says nothing about either business's quality or market share.
Common questions
Should I call the most mentioned business the best?
No. Mention frequency describes the sample you checked. It doesn't measure service quality, satisfaction or suitability. Keep the original answers so a reader can see the context.
What if a new business appears halfway through?
Record the new name as an observation. If you add it to the group, start a new version. Don't quietly change the basis of the earlier comparison.
Should I call the most mentioned business the best?
No. Mention frequency describes the sample you checked. It doesn't measure service quality, satisfaction or suitability. Keep the original answers so a reader can see the context.
What if a new business appears halfway through?
Record the new name as an observation. If you add it to the group, start a new version. Don't quietly change the basis of the earlier comparison.
Check your understanding
What belongs beside a comparison?
Read the answer and explanation
- The prompt set, surface, period and counting rule.
- Only the largest percentage.
- A claim that all users see the same answer.
Answer. The prompt set, surface, period and counting rule.
These details describe the sample you observed. They let a reader judge its limits.
Answer and explanation
The prompt set, surface, period and counting rule.
These details describe the sample you observed. They let a reader judge its limits.
Your next steps
Sources and method
A Carqo review method. Provider statements link to the relevant documentation.
- OpenAI API documentation, Web search Link checked 4 November 2026
- Claude API documentation, Web search tool Link checked 4 November 2026
- Google guidance on AI features Link checked 4 November 2026