From Pipeline Findings to a Risk Register

Everything in the SOC 2 as Code post ends at the same place: a pass/fail line in a terminal. [FAIL] CC6.6 No security group exposes a sensitive port to the internet. That’s a true statement, and it’s useless to almost everyone outside of me. A VP doesn’t read gapcheck output. What they read is a risk register: what’s wrong, how bad is it, who’s fixing it, and by when. Writing one of those from the real findings my pipeline caught, instead of a fabricated example, is what this post is.

What a risk register actually is

A risk register is a running list of identified risks, each scored and owned. At minimum it has:

• A description of the risk, in plain language, not just a control ID
• Likelihood — how probable is this actually happening or being exploited
• Impact — how bad is it if it does
• A risk rating — likelihood combined with impact, into an overall bucket
• An owner — a named person, not a team
• A remediation plan and target date
• A status — open, in progress, accepted, or closed

The scoring part is where a lot of registers go wrong. Likelihood and impact get treated as objective numbers when they’re really structured opinions. I’m not an auditor and I don’t have loss data to calibrate against, so I’m sticking to a simple 3-tier scale (Low/Medium/High) for both axes instead of a fancier 1–5 scale that would just be pretending the number is more precise than my actual basis for it. Here’s the matrix I’m using to turn the two scores into one rating:

Likelihood \ Impact	Low	Medium	High
Low	Low	Low	Medium
Medium	Low	Medium	High
High	Medium	High	High

No Critical tier. With three levels on each axis, calling anything “Critical” would need a fourth bucket I don’t have the loss data to justify — High is the ceiling here.

The findings, as risks

These are pulled straight from real gapcheck.py runs against the live account, not invented for this post.

Risk 1: SSH open to the internet on the default security group

• Found: live, unplanned, while working on the lab (see the gap-catching screenshot in the SOC 2 post)
• Likelihood: Medium. The account isn’t a high-value target, but “someone finds an open port and tries the default creds” is exactly the kind of opportunistic scan that happens to anything with a public IP, not a targeted attack that requires someone caring about this specific account.
• Impact: High. SSH access to a box in this environment is a foothold into everything else on the account it can reach.
• Rating: High
• Owner: me (in a real company, whoever owns the AWS account or the platform team)
• Remediation: remove the 0.0.0.0/0 inbound rule, restrict to a specific IP or a bastion. Already done — see CC6.6 in the assessment table.
• Status: Closed

Risk 2: IAM user without MFA

• Found: early gapcheck.py run, real 5/7 pass rate, not staged
• Likelihood: Medium. Credential-stuffing and phished-password attacks against AWS accounts are common; an account with a password but no second factor is one leaked password away from full access.
• Impact: High. IAM access is the whole account, not one system.
• Rating: High
• Owner: me
• Remediation: enroll the user in MFA. Closed.
• Status: Closed

Risk 3: CloudTrail not multi-region

• Found: same early run as Risk 2
• Likelihood: Low on its own — this isn’t something an attacker exploits directly.
• Impact: High. Not because a logging gap is dangerous by itself, but because of what it does to every other risk on this list: if something goes wrong in a region outside the one being logged, there’s no trail to investigate it with, and every other row in this register becomes harder to detect or prove.
• Rating: Medium — low likelihood pulls the combined rating down even though the impact is high.
• Owner: me
• Remediation: enable a multi-region trail. Closed.
• Status: Closed

Risk 4: single point of knowledge on the pipeline itself

This one isn’t a gapcheck.py finding — it’s a risk about the pipeline itself, not something the pipeline detected. Right now exactly one person knows how soc2-as-code works, where its credentials live, and what its false-negative cases might be. If that person is unavailable, the evidence collection stops and nobody notices until the next audit.

• Likelihood: Medium over a long enough timeline for any single-person system.
• Impact: Medium. Evidence gaps during an actual audit window are a real problem, not a hypothetical one.
• Rating: Medium
• Owner: me
• Remediation: document the runbook (README already covers setup; add an incident-style “what to do if this breaks” section), and treat the GitHub Actions cron run as a canary — if it goes red and nobody notices for a week, that’s the actual failure mode to fix.
• Status: Open

The register

ID	Risk	Likelihood	Impact	Rating	Owner	Status
R1	SSH open to 0.0.0.0/0 on default security group	Medium	High	High	me	Closed
R2	IAM user without MFA	Medium	High	High	me	Closed
R3	CloudTrail not multi-region	Low	High	Medium	me	Closed
R4	Single point of knowledge on the evidence pipeline	Medium	Medium	Medium	me	Open

Markdown tables are fine for a blog post, but a register is meant to be a working document, so I built the actual thing in a spreadsheet, color-coded by rating, the way I’d actually hand it to a stakeholder:

Same four rows, but this is the format someone would actually open and act on.

Four rows is a small register, but the shape scales. A real company’s register for this same scope would run to dozens of rows once you add the controls that can’t be automated: vendor risk, physical security, HR offboarding timing, the stuff controls.py can’t see because it isn’t an API call.

Why this is a different artifact than the evidence pack

The evidence pack from the SOC 2 post answers “did this control pass.” The risk register answers “why do we care, and what happens next.” An auditor mostly wants the first one. A CISO deciding what to fix this quarter, or a board asking what the company’s actual security exposure looks like, wants the second one. Same underlying facts, but the register is the version built for a decision, not an attestation.

It’s also the artifact that doesn’t automate away. gapcheck.py can hand me the raw fact that CC6.6 failed. It can’t tell me the SSH rule matters more than the CloudTrail gap because SSH is a direct foothold and CloudTrail is a detection gap one layer removed. That prioritization is a judgment call, and it’s the actual GRC work underneath the automation.

Pros and cons

Pros	Cons
Turns a pass/fail line into something a non-technical stakeholder can act on	Scoring is subjective no matter how structured it looks
Forces prioritization instead of a flat list of findings	Stale within a quarter if nobody re-reviews it
Gives every finding an owner and a deadline, not just a status	A register with no follow-through is worse than no register — it’s a paper trail proving you knew and didn’t act

Wrap-up

The findings in this register aren’t hypothetical. They’re the same ones from the SOC 2 post, just run through a different lens: not “did this pass,” but “how much should anyone actually worry about it, and who’s on the hook to fix it.” That second question is the one a security hire gets asked in an interview a lot more than “can you write a boto3 script,” and it turns out the honest answer depends on judgment a pipeline can’t supply on its own.

References

• SOC 2 as Code: Automating Evidence Collection for a SaaS Startup — the pipeline and findings this register is built from
• NIST SP 800-30 — Guide for Conducting Risk Assessments