Knowledge · How we work · 5 minutes
How to read a vendor security assessment
How a number differs from a decision, why missing evidence doesn't mean “secure” and which four questions to ask about any report someone shows you.
Last updated: August 21, 2026
The problem with a single number
A score looks good on a slide and answers none of the questions you need to ask before a decision.
Seventy-two out of a hundred can mean the system is average. It can also mean the system is fine but the vendor didn't share half of the information. Those are two completely different situations, leading to two different decisions, and a single number makes them look the same.
That is why we split it into four measures reported separately: how severe the impact is, how strong the evidence is, who can reach that surface at all and what decision follows from it.
Missing evidence is neither good news nor bad news
When something could not be confirmed, the honest answer is “we don't know”. Not “probably fine” and not “we assume the worst”.
A report without a section on what was not checked isn't shorter. It is less useful, because you can't tell where its validity ends.
A practical tip: with any report, first find the list of areas not tested. If there isn't one, that is your first piece of information about the quality of the report.
Risk depends on the use case, not on the vendor's name
The same vendor can be suitable for scheduling posts and unsuitable for handling HR. The technical assessment is the same, the weight of the impact is completely different.
That is why an assessment that doesn't state the use case is incomplete. If someone shows you a report on a tool, check what use the tool was assessed for, and whether that is your case.
Four questions for every report
First: what are the scope and the date. Without these two, the report says nothing about today.
Second: what was not checked and why. The answer “everything was checked” is a warning sign, not reassurance.
Third: how do we know. Every claim should have a source, and those that come from a test should be reproducible.
Fourth: when does it stop being valid. An assessment without an expiry date suggests the state of the system doesn't change, and it changes with every deployment.
What not to expect from any report
A security certificate. No honest party issues one, because security is not a state, but the result of specific checks at a specific moment.
A guarantee that nothing will happen. A report reduces uncertainty and sets out conditions. It doesn't remove risk.
A legal ruling. Pointing out what a given contract clause means for your use case is useful, and it is not a legal opinion.
In short
- Scope and date matter more than any number in the report.
- The section on what was not checked is a sign of the report's quality.
- An assessment without a stated use case doesn't answer your question.
- Every claim should have a source, and test results should be reproducible.
- A security certificate doesn't exist. Whoever offers one is selling something else.
Have a website, app or email address that looks suspicious?