Post

Watching the Inside: CIP-015 INSM with Zeek and Suricata

Internal Network Security Monitoring under CIP-015 — a passive sensor watching a SPAN of east-west OT traffic, catching the rogue DNP3 breaker command that the perimeter from post 4 can't see, decoded by Zeek and alerted by Suricata.

Watching the Inside: CIP-015 INSM with Zeek and Suricata

Post 4 put a wall around the grid equipment and one guarded door in it, and then had to admit the uncomfortable part: the wall does nothing about the attack from post 3. That rogue breaker command travelled from 10.10.20.12 to 10.10.20.11, both inside the perimeter, over a network segment the firewall never touches.

A firewall inspects what crosses a boundary. But the control-room software and the substation equipment inside the protected zone talk to each other all day, and that conversation never crosses the boundary, so the firewall never sees any of it. And that internal space is exactly where an attacker already is by the time they do anything: inside the ESP, speaking the normal protocol, and looking just like an operator.

If you can’t keep the attacker out, and you can’t tell their command from a real one, you’re left with one option: watch the inside of the network and notice when something happens that shouldn’t. In the regulations that’s CIP-015 — Internal Network Security Monitoring (INSM), the big new 2026 control, and this post builds it: a passive sensor, a mirror of the east-west traffic, and two tools that catch the exact command post 4 couldn’t see.

Where this fits. Post 5 of the series, and the one the whole thing has been building toward. Post 4 drew the perimeter; this post watches the space the perimeter can’t.

What is CIP-015?

CIP-015 (Internal Network Security Monitoring) requires that high- and medium-impact BES Cyber Systems monitor network traffic inside the Electronic Security Perimeter — east-west, not just at the boundary. It exists because the perimeter model has a blind spot the size of the entire internal network, and the regulators finally wrote a standard for it.

Three capabilities are required, and they map cleanly onto tools:

  • Collect — get visibility into internal traffic. That’s a network mirror (SPAN/TAP) feeding a passive sensor.
  • Detect — identify anomalous or unauthorized activity. That’s Zeek turning traffic into protocol-aware logs, and Suricata matching signatures.
  • Evaluate and retain — keep the data long enough to investigate. That’s log retention, and in a fuller build a SIEM.

The key word is passive. An INSM sensor never sits in the traffic path; it can’t, or it becomes a point of failure in the middle of a control system. It just gets a copy of the traffic and is otherwise invisible: no IP address on the monitored segment, no packets sent, nothing for an attacker to hit.

The design

The sensor is mgmt-sensor, a VM in the management zone, outside the ESP, which is where an INSM sensor belongs (it’s an EACMS, so it monitors the BES Cyber System without being part of it). It has two network interfaces doing very different jobs:

  • A management interface on the management zone (10.10.10.20) — how I administer it.
  • A monitoring interface on the OT bridge with no IP address at all — a passive port that only ever receives a mirror of the OT traffic.
flowchart TB
    subgraph esp["ESP — 10.10.20.0/24 (firewall is blind to traffic in here)"]
        grid["ot-grid .10"]
        plc["ot-plc .11<br/>DNP3 outstation"]
        scada["ot-scada .12<br/>SCADA master"]
        scada -.->|"DNP3 CROB — east-west"| plc
        grid -.->|"Modbus poll"| plc
    end

    esp ==>|"SPAN: mirrored copy<br/>(tc, one-way)"| mon

    subgraph mgmt["Management zone — outside the ESP"]
        mon["mgmt-sensor monitoring NIC<br/>(no IP, promiscuous)"]
        zeek["Zeek<br/>(protocol decode → dnp3.log)"]
        suri["Suricata<br/>(signature → alert)"]
        mon --> zeek
        mon --> suri
    end

The double arrow is the key part: a one-way copy of the internal traffic leaves the ESP and arrives at a sensor that can never send anything back into it.

Setting it up

1. The mirror (SPAN)

The OT hosts sit on a Linux bridge (virbr-ot). To copy their traffic to the sensor without sitting in the path, I use tc to mirror each OT host’s tap interface to the sensor’s monitoring tap. Only VM-originated frames are mirrored, so the full conversation is captured once with no duplication:

1
2
3
4
# for each OT tap, mirror VM-originated frames to the sensor's monitoring port
tc qdisc add dev "$src" ingress
tc filter add dev "$src" parent ffff: matchall \
    action mirred egress mirror dev "$sensor_tap"

The sensor’s monitoring interface comes up with no address and in promiscuous mode:

1
ip link set enp8s0 up promisc on   # no 'ip addr' — it never has one

The proof it works is a tcpdump on that passive interface, showing OT traffic the sensor is not a party to:

tcpdump on the passive monitoring interface showing mirrored east-west OT traffic The sensor has no IP on this segment, yet sees 10.10.20.11 ↔ 10.10.20.10 Modbus polling and the DNP3 conversation. This is exactly the traffic the EAP in post 4 could not filter — because it never reaches the EAP.

2. Zeek — turn traffic into protocol-aware logs

Zeek watches the monitoring interface and writes structured logs. The one that matters is conn.log, whose service field names the application protocol it identified:

1
2
3
$ grep -E ':20000' conn.log | awk '{print $3, $4"->"$5":"$6, $8}'
10.10.20.12 53375->10.10.20.11:20000 dnp3_tcp
10.10.20.12 49193->10.10.20.11:20000 dnp3_tcp

dnp3_tcp — Zeek recognised industrial control traffic east-west inside the ESP, with no help from me beyond pointing it at the mirror. And because Zeek 8 ships a DNP3 analyzer, it decodes the function codes into a dedicated dnp3.log:

Zeek dnp3.log decoding the DNP3 function codes, including DIRECT_OPERATE Zeek names the exact function. The polling reads are READ; the setup exchange is DISABLE_UNSOLICITED / ENABLE_UNSOLICITED; and the breaker command is DIRECT_OPERATE — DNP3 function code 5, the CROB from post 3.

This is the whole point of the series landing. The command post 3 showed was byte-for-byte identical to a legitimate one, the one post 4’s firewall couldn’t see, is now sitting in a log labelled by function, on a sensor the attacker doesn’t even know is there.

3. Suricata — alert on the command that matters

Where Zeek just describes the traffic, Suricata makes a call on it. Here’s a signature that fires specifically on a DNP3 breaker operation:

1
alert dnp3 any any -> any 20000 (msg:"OT INSM CIP-015: DNP3 DIRECT_OPERATE breaker command (func 5)"; dnp3_func:5; sid:9000001; rev:1;)

Fire the rogue CROB from ot-scada, and the sensor catches it:

Suricata alert firing on the DNP3 DIRECT_OPERATE breaker command 10.10.20.12 → 10.10.20.11:20000 — signature 9000001, DIRECT_OPERATE. The same command, now an alert with a timestamp, a source, and a CIP-015 label.

1
2
$ jq -r 'select(.event_type=="alert") | "\(.timestamp[11:19])  \(.src_ip) -> \(.dest_ip):\(.dest_port)  [\(.alert.signature_id)] \(.alert.signature)"' eve.json
22:36:51  10.10.20.12 -> 10.10.20.11:20000  [9000001] OT INSM CIP-015: DNP3 DIRECT_OPERATE breaker command (func 5)

Two things that will waste your afternoon

Neither is in any tutorial, and I lost real time to both.

Zeek and Suricata both ignored the traffic at first, for different reasons.

Zeek saw the TCP connection but left service empty and produced no dnp3.log. The cause is checksum offloading: virtual NICs hand checksum calculation to the hardware, so frames captured off a virtual mirror carry checksums that look wrong, and Zeek silently drops them before any analyzer runs. One line fixes it, and it is mandatory on any virtual SPAN:

1
redef ignore_checksums = T;

Suricata refused to even load the DNP3 rules — protocol "dnp3" cannot be used in a signature. The reason is that Suricata ships with the DNP3 app-layer parser disabled by default:

1
2
3
4
app-layer:
  protocols:
    dnp3:
      enabled: no      # <- flip to yes

Both defaults make sense for a generic IT deployment and are exactly wrong for OT monitoring. If your sensor is quiet when it should be alarming, suspect the defaults before you suspect the mirror.

The CROB versus the SELECT: detection is a design choice

Real DNP3 controls often use select-before-operate: a SELECT (function 3) to arm the point, then an OPERATE (function 4) to fire it. My lab uses DIRECT_OPERATE (function 5), which does both in one message. A signature written only for function 5 would miss an attacker who used the two-step form — so the ruleset covers 3, 4, and 5:

1
2
3
dnp3_func:5   # DIRECT_OPERATE
dnp3_func:4   # OPERATE
dnp3_func:3   # SELECT — stage one; catching this is catching the attack a step earlier

That last rule is the interesting one. A SELECT with no matching OPERATE from a source you don’t expect is an attacker fumbling the sequence, which is about the earliest warning you can get. INSM isn’t only about catching the command itself; it’s about knowing the protocol well enough to catch the setup for it.

What INSM catches that the perimeter can’t

The point of building this next to post 4 is the contrast, so here it is directly:

Event Post 4 firewall (EAP) Post 5 sensor (INSM)
Outsider connects to DNP3 port Blocked and logged (never reaches the sensor)
Jump host → OT over SSH Allowed and logged Seen as a session
Rogue CROB, ot-scada → ot-plc Invisible — never crosses the EAP Detected — Zeek decodes it, Suricata alerts
Modbus polling ot-grid ↔ ot-plc Invisible Baselined as normal

The firewall and the sensor aren’t redundant. They each see a different half of the network, and only together do they cover all of it. That’s the whole reason CIP-015 was added on top of the perimeter standards instead of folded into them.

Verifying it worked

INSM is real if all four hold, and all four do:

  1. The sensor is passiveip addr on the monitoring interface shows no address, and it never transmits. It cannot be a path into the ESP.
  2. It sees east-west traffictcpdump on that interface shows the OT hosts talking to each other, traffic the EAP never receives.
  3. Zeek decodes the protocolconn.log reports dnp3_tcp, and dnp3.log names the function codes.
  4. Suricata alerts on the breaker command — a CROB produces signature 9000001 with source, destination, and timestamp.

The CIP-015 assessment

Capability Current state Finding Risk Remediation
Collect (INSM-01) Passive SPAN via tc to a no-IP sensor port Mirror is host-config, not persisted as a service Reboot loses monitoring silently Systemd unit that re-establishes the mirror on boot
Detect (INSM-02) Zeek decode + Suricata signatures for DNP3 func 3/4/5 Signature-only; no baseline/anomaly detection Novel technique with no signature slips through Add Zeek behavioural scripts; alert on new talker pairs
Evaluate & retain (INSM-03) Logs on the sensor No central collection or retention window Evidence ages out; no correlation Forward Zeek/Suricata to a SIEM (post 6 territory)

The honest gap is INSM-03. Detection without retention and correlation catches the event but loses the investigation. A single sensor with local logs is fine for a demo, but a real program needs that data going somewhere it’s kept and searchable.

Pros and cons

Pros Cons
Catches the exact attack the perimeter cannot see Signatures only catch what you thought to write a rule for
Sensor is passive — no new attack surface inside the ESP A mirror on a busy segment is a lot of data to store and search
Zeek’s protocol decode turns raw traffic into readable evidence Virtual SPAN gotchas (checksums, disabled parsers) are silent failures
Maps directly onto the three CIP-015 capabilities One sensor, one segment — real networks need many

Wrap-up

The blind spot is covered now. A command that crosses no boundary, carries no exploit, and looks identical to a legitimate one is now decoded, named, and alerted on, by a sensor the attacker can’t see and can’t reach. None of the tooling is exotic: a SPAN port, Zeek, and Suricata are the same stack IT defenders have run for twenty years, just pointed at OT traffic. What’s new is the regulator finally requiring it inside the perimeter, because that’s where the grid attacks actually happen.

Two posts left. The findings have been piling up: broad egress rules, an undocumented anti-lockout rule, SSH on the perimeter device, a mirror that isn’t a real service, logs that go nowhere. Post 6 stops writing those up as prose and turns them into code: CIP-010 configuration baselines and evidence collection you can actually run and diff, so “we’re compliant” becomes something you prove on demand instead of just asserting.

References

This post is licensed under CC BY-NC-ND 4.0 by the author.