Companies buy penetration tests for two reasons: a customer asked for one, or something went wrong. Both produce a report. Only one tends to produce better security, and the difference comes down to how the engagement was scoped and what happened in the two weeks after the findings landed.
01Know what you are buying
An automated vulnerability scan finds known CVEs and misconfigurations. It is cheap, should run continuously, and is not a penetration test. A real test has a human trying to chain weaknesses into something that matters - a low-severity information leak plus a weak permission check becoming full account takeover.
If the report you receive reads like scanner output with severity ratings attached, you paid for manual testing and got a scan. That is unfortunately common at the cheaper end, and the tell is the absence of any finding that required understanding your business logic.
| Type | Finds | Cadence | Misses |
|---|---|---|---|
| Dependency scanning | Known vulnerable packages | Every build | Everything bespoke |
| SAST | Code-level patterns | Every build | Logic and authorisation flaws |
| DAST | Common web weaknesses | Nightly | Chained and business-logic issues |
| Manual pen test | Chained and logic flaws | Annual or per release | Regressions between tests |
| Bug bounty | Long-tail, ongoing | Continuous | Needs mature triage capacity |
02Scope for the damage, not for the surface
Testers given a login page and a week will find what a week of black-box testing finds. Give them credentials for each role, an architecture overview, and a clear statement of what would hurt most - customer data crossing tenants, payment manipulation, privilege escalation - and the same week produces findings that matter.
Grey-box testing is nearly always better value than black-box for this reason. You are not simulating a specific attacker profile; you are trying to find the flaws before someone else does, and information asymmetry only makes that more expensive.
03Multi-tenant isolation deserves explicit scope
For any SaaS product, the highest-consequence flaw is one tenant reaching another's data. It should be called out as a specific objective rather than left to whatever the tester happens to try.
Set up two test tenants with realistic data and ask them to break the boundary from every direction: object identifiers, API filters, search, exports, webhooks, background jobs and file storage. Isolation bugs concentrate in the paths nobody thinks of as user-facing, and a background job that ignores the tenant filter is a much more likely bug than a broken login.
04Fix, verify, then look for the pattern
Findings should land in your normal issue tracker with owners and dates, not sit in a PDF. Agree the retest window in the original contract - most firms include one - because a fix that was never verified is a fix you are assuming worked.
Then look past the individual bugs. Three separate authorisation findings in different endpoints is not three bugs, it is one missing pattern in how authorisation is applied. Fixing the three and moving on guarantees a fourth next year; fixing the pattern is what makes the next test cleaner.
| Finding pattern | Point fix | Systemic fix |
|---|---|---|
| Several IDOR findings | Add checks to those endpoints | Authorisation enforced in a shared layer |
| XSS in a few fields | Escape those outputs | Framework-level escaping, CSP |
| Secrets in config | Rotate them | Secret manager plus CI scanning |
| Outdated dependencies | Update them | Automated dependency updates |
05Once a year is a compliance cadence, not a security one
An annual test satisfies an auditor and tells you about the state of an application you have since changed hundreds of times. Continuous automated scanning between tests catches regressions, and running a focused manual test after any significant architectural change catches what scanners cannot.
The most cost-effective arrangement for most product companies is continuous automation, one substantial annual manual test, and targeted testing whenever the authentication, authorisation or payment paths change materially. Those three areas produce a disproportionate share of serious findings.
Topics
Priya Iyer
Staff Engineer · SyncTrix
Writes about the engineering decisions behind production systems - architecture, delivery and the trade-offs that only show up at scale.
Building something like this?
SyncTrix engineers AI, SaaS, platform and cloud systems for enterprises and high-growth teams. Tell us what you're shipping and we'll scope it with you.
Talk to an engineer