Experiments
Test website changes with a fair comparison.
Write down your prediction before you make a change, hold something back to compare against, and see how much weight the result can bear.
Example visibility, the share of counted answers naming each business.
Your workflow
- 01Register the study
- 02Set the evidence floor
- 03Make the change
- 04Read it honestly
01 / Experiments
Real effect or coincidence
A change followed by an improvement is an association, not proof. AI answers move for many reasons, including new models and ordinary variation from week to week.
A study needs a period without the change to compare against and a scope that says what was touched. Inconclusive means more data might settle it. Invalid means the design cannot answer it.
02 / Experiments
Write the prediction first
Name the study, say what you expect to happen and why, and list the questions it will follow. Once it starts, the prediction, questions, scope and period before are fixed. The result cannot be fitted to the prediction later.
03 / Experiments
Four designs and their limits
The hold-back design changes some items and leaves the rest alone as a comparison group. It only holds while those items stay untouched. The staged design rolls a change out in steps, so timing does the comparing. Anything else changed on the same schedule cannot be told apart.
The before-and-after design compares the whole website with the period before the change. There is no control, so it can show a change and cannot show its cause. A study with no design is recorded as an association only.
04 / Experiments
Rules that keep it fair
A study may not hold back a fix your website needs. Corrections, and security and privacy fixes, reach every page, and a study that includes one is refused.
By default a study needs at least ten answers in each group, fourteen days and five questions. You can ask for more evidence, and never for less.
A site-wide change during a hold-back study treats the comparison group too. When that happens, the study is shown as one whose comparison cannot be believed.
How it works
- Register the study Name it, write the prediction and choose the scope and design.
- Set the evidence floor Keep the default minimums, or ask for more.
- Make the change You or your web developer make the change, then mark the study as running.
- Read it honestly Mark the study completed, inconclusive or invalid.
Who it is for
Service businesses
Hold the other pages back and compare like with like.
Practices
Change some branch pages and leave the rest as a comparison group.
Shops
Template changes reach every page, so the study flags when a comparison has been spoiled.
How Carqo works
You connect your website, review what AI search says and receive drafts for your next steps.
Carqo checks the facts
Facts are checked against their source where Carqo holds one. Conflicts and out-of-date facts are flagged to you.
Carqo shows where AI search mentions you
It follows your customer prompts and shows what the answers said about your business.
You decide what to do with each draft
A person on your side reviews each draft and decides whether to use it.
Common questions
Why not just compare before and after?
Without a comparison group, a rise, a fall or a flat line can each be explained away. A registered design says in advance what counts.
Can a study hold back a correction my website needs?
No. Corrections reach every page. Only optional changes can differ between groups, and the form refuses anything else.
How much evidence does a study need?
By default, at least ten answers in each group, fourteen days and five questions. You can raise any of those numbers.
Will I see an uplift figure?
No. A study shows its design, its status and whether the comparison can support a conclusion. It cannot promise that a difference will last.
Why not just compare before and after?
Without a comparison group, a rise, a fall or a flat line can each be explained away. A registered design says in advance what counts.
Can a study hold back a correction my website needs?
No. Corrections reach every page. Only optional changes can differ between groups, and the form refuses anything else.
How much evidence does a study need?
By default, at least ten answers in each group, fourteen days and five questions. You can raise any of those numbers.
Will I see an uplift figure?
No. A study shows its design, its status and whether the comparison can support a conclusion. It cannot promise that a difference will last.
See the answers.
Then choose your next step.
Start with a free AI visibility report.