Open a clean session and ask an assistant to describe your agency and what makes it different from its competitors. Don’t help it. Don’t paste your homepage in first. Just ask, and read what comes back.
Most agency owners get a paragraph that could describe forty other companies. Full-service, results-driven, strategic partner, works across industries. The instinct at that point is to treat it as an AI problem and go looking for a fix that makes the model smarter about you.
When an assistant describes your agency in generic terms, it’s reporting on your public record rather than failing to read it. The model is summarizing what’s available about you, so a vague answer is a positioning diagnosis rather than a technical one.
Quick Take
- The model has no private knowledge of your business. It can only summarize what you and other people have published, so its answer is a fair reading of your footprint.
- A generic description means the differentiators aren’t on the page anywhere. That’s fixable, and it’s the same fix whether or not you care about AI.
- Being described accurately and being recommended are separate problems. My own site wins citations on definitional questions and gets named in zero of the four buying questions I track.
- Buyers are already running this exact test on agencies. I can see them doing it in my own search data.
In this article
- What the test measures
- Why the answer comes back generic
- Buyers are already running this on you
- How to run it properly
- What to change when the answer is vague
What does the test measure?
It measures how legible your positioning is to someone reading everything you’ve published and nothing else. That’s a harsher reader than a prospect, who arrives with context from a referral or a conversation, and it’s a fair proxy for a cold buyer landing on your site at eleven at night.
The model can’t see your client roster, your delivery process, or the reason your best accounts stay. It sees your homepage, your blog, your LinkedIn, whatever directories list you, and any third-party page that mentions you. If your difference lives in how you actually work but never made it into writing, the model has no way to know, and neither does the buyer.
So the output is a mirror. When it comes back as a list of services and adjectives, that’s usually accurate reporting on a footprint made of services and adjectives.
Why does it come back generic?
Because the claims on most agency sites can’t be verified before a sale, so there’s nothing distinctive to summarize.
“We do better work” is the clearest case, and it’s never a real differentiator. Anyone can say it, nobody can check it, and a buyer discounts it to zero. A model does the same thing for the same reason. It has no basis to repeat a quality claim, so it falls back on the category description, which is the part of your site that reads like everyone else’s.
The differentiation that survives this test is almost always structural, the kind of thing the 11 Differentiation Levers are built to surface, and owners walk past it because it doesn’t feel like marketing. How you charge. What you refuse to take on. What you guarantee. A hard cap on client count. Month-to-month terms in a category that demands annual ones. Those are checkable facts, they can be stated in a sentence, and a model can repeat them without evaluating anything.
The uncomfortable version of this: if the answer about your agency is vague, the honest read is that your positioning is vague, and the assistant just told you faster than a buyer ever would.
Buyers are already running this on you
This isn’t a hypothetical exercise. It’s showing up in my own search data as queries I can read.
Across ninety days, five separate queries reached my site in the form “evaluate the agency company [name] on competitors and differentiation,” each one naming a different real agency. Somebody is running a differentiation audit as a repeatable prompt, swapping in company names. Another query asked an assistant to “summarize [company]‘s positioning and differentiators.”
Those are buyers, competitors, or prospective hires doing exactly what I described in the opening, against companies that have no idea it’s happening. Whatever your public record says is what they got.
Two things follow. The first is that the answer about you already exists and is already being read, whether or not you’ve looked at it. The second is more useful: the questions people ask assistants are phrased differently from the keywords they type into search. Roughly a quarter of the queries reaching my site last quarter were full sentences, and the vocabulary in them is buyer vocabulary, not category vocabulary. That’s free research into how your market describes its problem.
How to run the test properly
Do it in an incognito window, logged out of everything. An assistant you have been using for months knows who you are and is built to give you the answer it thinks you want, so it will happily put your own company at the top of a list it would never put you on for a stranger. Logged in you are measuring your chat history rather than the market.
Ask four questions and save the answers verbatim. Ask what your company is and who runs it. Ask what makes your company different from its competitors. Ask for the best providers in your category, phrased the way a buyer would phrase it rather than the way you’d describe yourself. And ask for alternatives to a named competitor, which tells you whether you’re in the consideration set at all.
Then read for two specific things rather than for a general impression. Look at whether the description contains a single fact a competitor couldn’t also claim. And look at which pages the answer cites, if it cites any, because that tells you which URL the model considered worth quoting.
Run it again in a month with the same wording. Changing the questions between runs leaves you with two unrelated snapshots and no way to tell improvement from drift.
What to change when the answer is vague
Put the checkable facts in writing, in plain language, on a page a crawler can reach.
That means a sentence a buyer could verify before signing rather than a promise about quality, so write down who you work with and who you turn away, what the engagement includes and what it costs, and what happens if it doesn’t work. Owners resist this because it feels like giving away negotiating room, and in practice it’s the part that gets repeated back to you.
Then get those facts onto pages you don’t own. My own experience here was instructive: the thing that eventually corrected how an assistant described me wasn’t anything on my site, it was a third-party podcast page where my name, my company and my subject all appeared together. Third-party pages carry weight that self-published claims don’t, which is the same reason a testimonial outperforms a slogan.
One caution before you build a strategy on being described well. Being described accurately and being recommended are different games with different fixes. Two of my pages get cited in AI Overviews, both at the top of the answer, and across two months of tracking I’m named in zero of the four buying questions I care about. Explaining things well earns citations. Getting into a recommendation set is a separate job.
If you run the test this week and get a paragraph that could describe anyone, you’ve just been handed the clearest read on your positioning you’ll get for free, and that’s the work we do inside the Dynamic Agency Community.
FAQ
Will ChatGPT know about my agency if we’re small?
Often, yes, and usually less accurately. Smaller footprints mean thinner source material, so the model leans harder on category assumptions and on anyone else sharing your name. A short, confident, wrong answer is more common for small agencies than a blank one.
Should I paste my homepage in and ask it to evaluate that instead?
That’s a useful separate exercise, but it answers a different question. Pasting your copy tests whether the writing is clear. Asking cold tests whether the positioning has reached the world. Do the cold version first, because that’s the one a buyer experiences.
What if the assistant describes us accurately but still doesn’t recommend us?
Then your positioning is legible and you have a presence problem in the buying questions, which is a separate piece of work. Look at which pages the model cites when it answers those questions. In my category, nearly every cited page turned out to be published by a vendor inside the category rather than by an independent reviewer.
How is this different from just asking a friend to read my website?
A friend brings context and charity, and both distort the result. The model brings neither, which is closer to how a cold buyer reads you. It also reads everything about you at once, including the third-party pages you’ve forgotten exist.
Does this replace real positioning work?
No. It’s a diagnostic, and a fast one. It tells you whether your positioning has made it into the world in a form someone else can repeat. Deciding what the positioning should be is still the founder’s job and still the harder half.