Aggregator scores or auditable criteria, reading SAAS marketing agency reviews

What to take away

A Manchester software team shortlists three agencies, then books a call on the strength of a five-star profile. The listing was compiled by a directory that earns commission on introductions, and nobody checked whether the scores were earned or bought.

What counts as an agency review?

A review is a documented assessment against stated criteria. A directory profile is not. Nor is a testimonial the agency asked a happy client to write.

Ask who did the assessing, when, and against what. If the answer is a commission-driven platform, weight it accordingly. If the answer is a named procurement lead with a scoring sheet, you can audit it.

The SAAS marketing best providers in England page covers how shortlists are built. Read it alongside this piece before you score anything.

Scores drift for reasons unrelated to quality: a ranking algorithm, a sponsored placement, a burst of reviews after a conference.

What was assessed here, and what was not?

Nothing was tested. No agency was briefed, no campaign was run, no invoice was paid. This is a reading method for reviews you find elsewhere.

What was assessed: how review claims are structured, and which sources can be verified. What was not: any named agency's performance, pricing or client results.

That distinction matters because a review that mixes tested facts with borrowed claims is unusable for procurement. Separate the two columns before you decide.

How do vendor claims differ from independent evidence?

Vendor claims come from the seller: pitch decks, homepage metrics, award badges, conference talks. They may be true, but they are not evidence until someone outside the seller verifies them.

Independent evidence includes statutory accounts at Companies House, published complaint records, and regulatory guidance. The Competition and Markets Authority sets out how it promotes competition and acts on misleading commercial practices, which is the frame for any claim about market conduct.

For data handling, the ICO's data protection principles guidance is the baseline for SAAS compliance and ethical marketing. An agency that cannot describe its lawful basis for processing prospect data fails a basic check.

Verification settles what is true. It does not tell you what matters for your shortlist. The weighted method in SAAS marketing: reviews and comparison methods for 2027 handles that second job.

Where does editorial judgement fit?

Judgement is unavoidable, so make it visible. State the criteria, the weighting and who signed off. A score with no method behind it is an opinion wearing a number.

Paid search and organic creative claims are a common weak spot. The IAB UK search creative best practice principles give a published reference for judging whether a campaign approach is sound.

A workable test: choose two criteria, such as reporting clarity and lawful data handling, and score both agencies yourself. If your order differs from the review's order, you have learnt something about the review.

Glossary

Common questions

Can I trust a review platform's star ratings?

Only as a starting signal. Check whether the platform takes commission, whether reviews are verified, and whether criteria are published. If not, treat the rating as marketing.

Should I ask an agency for client references?

Yes, but expect curated names. Ask for a client who left, and ask what changed in the last year. Compare the answers with the agency's public claims.

How many reviews should I read before shortlisting?

Enough to see a pattern, not one loud outlier. Three independent sources with different incentives beat thirty reviews on one commission-driven site.

What if the reviews contradict each other?

That is normal. Weight the sources by incentive and evidence, then run your own scoring sheet with your own criteria before any call.