Penetration Testing — Rules of Engagement & Cadence
Owner: Security. Review cadence: per engagement, at least annual.
External penetration testing is a SOC 2 / FedRAMP expectation and the highest-
signal validation of the technical controls in SECURITY.md.
This document is the standing RoE to hand a testing firm; engaging the firm
and scheduling the test is an organizational action (see Procurement).
Cadence
- Annual full-scope external test (baseline requirement).
- On material change — a new auth flow, a new externally-reachable
surface, or a major dependency/runtime bump (e.g. the K8s client rewrite).
- Re-test of any Critical/High finding after remediation, before close-out.
Scope
Targets:
- Control plane
/cp/* — auth, sessions, MFA, invites, RBAC, org lifecycle.
- Tenant API
/v1/* — metering, entitlements, billing, credits, ledger.
- The dashboard SPA + its session/cookie handling.
- The Helm chart + images: PodSecurityStandards, NetworkPolicies (cross-tenant
isolation — see
tests/integration/cross-tenant-isolation.integration.test.js),
secret handling, Postgres RBAC.
Emphasis areas (where our threat model concentrates risk):
- Tenant isolation — can org A reach org B's data plane, Postgres rows, or
Redis keys? (We don't run tenant code, so cross-tenant data exposure, not
sandbox escape, is the crown-jewel risk.)
- AuthN/AuthZ — session fixation/idle/absolute limits, MFA bypass, the
admin-key vs session vs JWT boundaries, IDOR on
/v1/customers/:id/*.
- Billing integrity — metering/credit/ledger tampering, negative-amount and
replay abuse on grants + webhooks.
- Webhook + SSRF surfaces — outbound webhook dispatcher, OIDC redirect/callback.
Out of scope: subprocessors (Stripe/Resend), volumetric DoS, and
findings requiring a pre-compromised operator cluster.
Environment & rules
- Test against a dedicated staging cluster seeded with synthetic orgs —
never production tenant data (honor the in-cluster data-locality rule).
- Provide the firm two test orgs (so cross-tenant tests have an A and a B) +
an admin key scoped to staging.
- No destructive teardown of shared infra; coordinate any reclaim/teardown
testing with the team.
Findings → remediation
Findings map onto the standard remediation SLA (Critical 24h / High 1 week /
Medium 1 month / Low 3 months — SECURITY.md). Each
finding gets a tracked ticket; criticals are re-tested before the engagement
closes. The final report is retained as SOC 2 evidence
(see soc2-evidence-map.md, CC4.1).
Vendor selection criteria
When picking a testing firm, the Security lead evaluates against:
- Credentials — at least one of CREST, OSCP, OSCE, or equivalent
certifications held by the lead tester. Listed in the engagement
proposal.
- SaaS + Kubernetes experience — demonstrated prior engagements on a
multi-tenant SaaS API + a Helm-deployed control plane. References on
request.
- Methodology — published methodology (e.g. OWASP WSTG, PTES) the
firm follows + how they map findings to CVSS v3.1 / v4.0.
- Insurance + indemnity — professional indemnity + cyber-liability
coverage commensurate with the data the staging cluster holds (synthetic,
but the firm gets admin keys).
- Contractual terms — MSA + NDA + a per-engagement statement of work;
results owned by MeterBox; findings exclusive to MeterBox for the
reasonable-disclosure window in the SOW (default 90 days).
- Independence — firm has not provided product-development services
for MeterBox in the past 12 months; no individual tester moonlights
as a MeterBox contractor.
Tie-breaker on otherwise-equal candidates: a firm that publishes anonymized
case studies + recent CVEs they've credited reporters on.
Expected deliverables
Each engagement closes with these artifacts; failure to deliver any blocks
sign-off and the second tranche of payment:
- Executive summary — one page; risk posture in plain English for the
CEO + auditor.
- Findings report — per-finding section with: title, CVSS, affected
surface (
/cp/*, /v1/*, SPA, chart, image), reproduction steps,
observed impact, recommended remediation, references.
- Raw evidence — request/response captures, screenshots, exploit
scripts. Provided via the firm's encrypted portal; MeterBox retains
copies in the company password manager.
- Remediation retest letter — issued after the team closes Critical /
High findings; confirms the original issue is no longer reproducible.
Findings track + retest workflow
- Security lead ingests the findings into the same workflow as
CI / Dependabot advisories (one ticket per finding).
- Severity-driven SLA from SECURITY.md §
Vulnerability management applies.
- Each finding's ticket links back to the engagement id + the firm's
raw-evidence reference.
- Critical / High findings get a retest from the same firm after
remediation, before the engagement closes.
- Medium / Low findings are retested internally; the firm signs off on
the close-out summary.
Report retention
Pentest reports are Restricted artifacts per the
InfoSec policy § 3. Stored in the
company password manager (not the repository). Retention per the
data retention policy under the
legal-obligation exception: 7 years to support SOC 2 / FedRAMP /
regulator follow-on review windows.
Annual program review
Security lead reviews the program once per fiscal year:
- Were the agreed engagements held on the agreed cadence?
- Did the firm deliver against § Expected deliverables?
- What were the recurring themes across the year's findings? (Drives the
annual training refresher per
security-training.md § 4.)
- Renew, replace, or rebid? (Default: rotate firms every 2–3 years to
vary the methodology.)
Review notes are committed alongside the year's annual training log.
Procurement (the human steps)
- Select a reputable firm against the criteria above; sign an MSA + NDA
- SOW. Budget owner: CEO; engagement owner: Security lead.
- Share this RoE + read-only access to SECURITY.md and the architecture
overview in README.
- Schedule the window; stand up the staging cluster + seed test orgs.
- On delivery: ingest findings into the remediation workflow, re-test
criticals, file the report in the password manager as evidence,
update the soc2-evidence-map CC4.1 row with the engagement date.
Related docs