From Log to Incident, and the Budget That Decides What You See

I have taught this material before as two sessions, and I usually open each session the same way: this is not a tool class. Tools get named below, plenty of them, but every one is an implementation of the same pipeline, and the pipeline is what transfers to whatever product your employer bought. A SOC turns telemetry into decisions under time pressure. Machines generate records at a volume no human can read, dwell time favors the attacker, and the triage queue grows while you think. The SOC's product is not alerts and it is certainly not dashboards. It is decisions: contain or watch, escalate or close, declare or keep investigating. A SOC that only produces alerts is a very expensive smoke detector nobody answers.

Everything between the raw record and the decision is one pipeline, and every stage discards volume to add meaning. Billions of log lines become parsed events, thousands of rule matches, hundreds of alerts, and a handful of incidents in a normal week. Students assume the funnel narrows on its own. It does not. Every narrowing is a filter someone designed: a parser, a rule, a triage decision. When it narrows in the wrong place, a source never collected or a rule never written, that is not less noise. That is a blind spot, and nobody notices until an incident asks for the data.

This post converts my teaching deck on security operations into prose, slides and speaker notes both, the same way the names, addresses, and time post converted its companion deck. It also pairs with the security operations and monitoring architecture material in my Cybersecurity Architect's Handbook, Second Edition, which carries the enterprise-depth treatment of what this post covers at teaching depth. One warning up front: the budget argument in Part 1 is not a digression. It is the design conversation you will be in within a year of taking a security job.

Four words the whole post depends on

Vendors blur these daily, so we fix them first. A log is a record that something happened: raw, as the source wrote it. Existence, not judgment. An event is a normalized log the platform understood: parsed into fields, the same fact, now queryable and comparable across sources. An alert is an event, or a correlation of events, that crossed a detection rule: a machine's opinion that something deserves human attention, and opinions can be wrong, which is the tuning problem. An incident is one or more alerts a human or a playbook decided matter: a declaration with consequences, response effort, notification clocks, paperwork. Machines propose; someone decides.

The teaching cue that earns its keep for years: most SIEM arguments are a disagreement about which of these four words the speaker meant. "We get a million alerts a day" almost always means a million events. "The SIEM missed the incident" usually means a log was never collected, or an alert fired and nobody triaged it: different failures, different fixes, and you cannot diagnose which until everyone uses the same four words. Try it on vendor marketing: "processes 10 billion security events" usually means logs.

Part 1: The Pipeline

The pipeline: sources, collection and transport, parsing and normalization, correlation and detection, alert and triage, incident. Volume drops at every stage as meaning is added.

Each stage does one job: collection moves the records, parsing turns them into events, correlation decides when to raise a hand, triage forms a human opinion, an incident is declared. Along the wedge, volume falls by orders of magnitude, and those are honest magnitudes: a mid-size enterprise really does produce billions of log lines a day once flow records and endpoint telemetry are counted. If a stage adds no meaning, it is cost. If it discards the wrong thing, it is a blind spot. The rest of this post is those arrows.

The telemetry floor: what devices emit, and only if configured to

Every device class emits something different, none of it by accident. Network devices speak syslog with a facility and a severity from 0 through 7; most shops ship 0 through 6 and drop debug, because shipping severity 7 from a busy firewall is how you learn what your ingest bill is made of. SNMP traps tell you when something happened; polling tells you the state you forgot to ask about; both are thin on security value next to syslog and flow. TACACS+ command accounting is the unsung source: who typed what on the router matters enormously during an incident. Firewalls produce session logs (allow and deny, bytes, NAT), threat events, and VPN authentications, and the allow logs are the volume monster we will meet again. Flow exporters produce NetFlow or IPFIX: who talked to whom, when, how much. Hold on to what flow omits, because it defines the job: no payload, no URLs, no filenames. Flow answers "did that host talk to that address and how much," which is exactly the exfiltration and C2 question, and nothing else. Endpoints contribute OS event logs, Sysmon-class telemetry, and EDR data. Identity providers contribute auth successes and failures, MFA outcomes, token grants, and admin changes: the highest detection value per line of anything here. Cloud and SaaS contribute control-plane audit records, and the delivery changes: these arrive by API pull, not by wire.

Now the honest note behind half of all "the SIEM never saw it" tickets: the pipeline begins in device configuration, not in the SIEM. A wrong severity threshold, a wrong facility, a wrong destination, or logging to a buffer that rolled over, and the record never existed as far as your SOC is concerned. My version: a firewall whose configured log destination was a collector decommissioned eleven months earlier. It sent every log into the void for most of a year, and nobody noticed, because absence looks exactly like health until you check.

Visibility is a placement decision, made before any tool is bought

Where the sensors sit decides what any tool can ever show you. The perimeter is the easy tap: everyone has it, it sees ingress and egress, and it sees nothing that stays inside. Lateral movement lives east-west, and east-west stays dark unless a sensor was designed in, because attackers know which zones nobody watches. Every zone boundary from your segmentation work is a candidate collection point; the zones with no boundary stay dark. NDR sensors follow the same logic as any tap: perimeter placement is cheap and half blind, east-west placement is where the interesting traffic is and where nobody budgeted. At orientation depth, know the SPAN versus TAP trade: SPAN ports drop packets under load and lose their configuration in changes; TAPs cost money and never lie. Budget for taps at the choke points that matter. And cloud gets its own sentence, because people raised on packet captures assume visibility means wire access, then meet a SaaS platform where the only telemetry is the audit API the vendor chose to offer, at the license tier that includes it. The dark zones stay dark by default. You choose your blind spots when you place collectors; the only question is whether you choose them on purpose.

Time: the precondition, in one paragraph

Every correlation rule in Part 2 is secretly a temporal join. "Failed logins followed by a success within five minutes" assumes the IdP and the VPN concentrator agree what five minutes is. A device four minutes slow produces logs that are individually true and collectively misleading: sequences reorder, effects precede causes, the join window misses entirely. Forensics assumes a defensible timeline, and "the firewall was four minutes fast" is a cross-examination gift. SP 800-53 AU-8 is the control: clocks synchronized to an authoritative source, which is what makes AU-12 generation and AU-6 review mean anything. The NTP hierarchy that satisfies it, an internal stratum pair with every log source pointed at it and drift monitored, is built in the names, addresses, and time post, so I will not re-teach it. One misconception to kill in passing: "the SIEM timestamps on arrival, so it doesn't matter." Arrival time tells you when the log traveled, not when the event happened. You need both fields, and detections must run on event time. Logs with bad timestamps corrupt the evidence this entire post is about.

Normalization, and the failure mode nobody alerts on

Parsing splits raw lines into fields and maps vendor dialects onto one schema, so src_ip, SourceIp, and c-ip become the same field. The result is one query language over forty vendors' formats, which is the whole point of a SIEM. A Cisco ASA line reading %ASA-6-302013: Built outbound TCP connection... becomes action=allow direction=outbound proto=tcp dst_port=443. A log became an event. Nothing was learned yet, but now it can be.

This stage earns its section because of the silent failure. A vendor changes log format in a firmware update, the parser mismatches, fields come out empty, and the rule referencing the now-empty field simply never fires again. No error, no alert about the missing alerts: the rule looks healthy and matches nothing, because the engine cannot distinguish "field empty because the parser broke" from "field empty because nothing matched." I have watched a detection stay dark for a month after a firmware update reshuffled a message format. The fix was not cleverness; it was monitoring parse-failure rates and per-source volumes so a source going quiet or garbled pages someone. Nothing is what an outage looks like and nothing is what healthy looks like, and only baselines tell them apart.

Correlation and enrichment

Correlation climbs a ladder. A threshold rule counts N failures from one source in T minutes. A sequence rule notices many failures and then a success on the same account: order matters now. A multi-source join notices an IdP success from a new country, plus EDR seeing a new process, plus the firewall seeing new egress, joined on identity and time. Each rung needs more sources onboarded, normalized, and time-synced. The pipeline earns the rule, which is why I taught it in this order.

Enrichment is the same stage from the other side: correlation decides when to raise a hand, enrichment decides what the hand is holding. Attach asset context (crown jewel or lab box, and whose), identity context (privileged, on leave, usual geography), and threat intel before a human looks. "Login from new country" is an alert. "Domain admin, on PTO, from an address on an active feed, touching the payment server" is an answerable alert, and its severity is now obvious. Better detection does not always mean smarter rules; often it means the same rule with context attached, and triage time collapses. Remember this paragraph when we reach automation: it is the first SOAR conversation in the post, made before the acronym appears.

Why everything should be logged

This is the argument the post was written for. Both sides get their full weight, because neither one is wrong.

You cannot detect what you did not record. No rule, however clever, fires on a log that was never generated or never shipped. And detection is only half the reason. Forensics is retroactive: the attacker chooses, after the fact, which log source mattered, and they did not consult your ingest budget. First contact routinely lands in a source nobody rated interesting: a print server, a badge system, a forgotten jump box. Every seasoned responder has this story. Mine was a device log everyone had written off as noise, right up until it was the only record of first contact we had. On top of the forensic argument sit the compliance floors, which apply regardless of taste: PCI DSS Requirement 10.5.1 wants twelve months of audit history with three months immediately available, and other regimes carry their own clocks.

The control anchors are AU-12 and AU-11 from SP 800-53 Rev. 5. AU-12, audit record generation: the capability to produce the records must exist on the components you selected, configured and verified, not assumed. AU-11, retention: records kept long enough to support after-the-fact investigation and meet the floors, as a stated period per source, not a storage default.

So we log everything. Right?

"Everything" meets a budget, and the budget always wins

For most readers of this site, this section is not an aside. It is the job. You will design under ingest pricing, not unlimited licenses, and a post that pretends otherwise teaches you to fail your first real design conversation. Three budgets constrain "everything," and only two are denominated in dollars.

License economics: ingest-priced platforms charge per gigabyte per day, making every log line a line item; workload-priced platforms charge when you search and correlate; others tier by node, user, or entity. Categories rather than quotes, because the models move yearly, but the shape is stable: on the dominant models, volume is the bill. Storage economics: hot, indexed, searchable storage costs an order of magnitude more than object storage, and retention floors multiplied by verbose sources compound monthly. The quiet scandal of most deployments is that the majority of ingested data is never queried. You are paying premium rates to warehouse silence.

The third budget is the one no purchase order can raise: analyst attention. Alert volume spends human minutes, the scarcest resource in the room, and the numbers are the argument. Five hundred alerts a day at five minutes each is roughly 41 analyst-hours a day: five full-time humans doing nothing but triage before a single investigation happens. One noisy rule paging forty times a night at four minutes a dismissal eats close to three hours of a shift, roughly a thousand hours a year. Half an analyst, spent on one bad rule. More sources means more rules firing means longer queues, and real alerts age in line.

The control that makes this teachable instead of depressing is AU-2, event logging, and hear its register exactly: selecting which event types to log is an explicit, documented, periodically reviewed business decision, coordinated with the people who investigate. Selection is what the framework expects. Budget is a design input, not a failure to apologize for. The shame is not "we don't log everything." The shame is an undocumented default. Ask who decided the current logging scope at your workplace. The honest answer is usually "the default config," which means nobody decided, which is the actual finding.

The resolution is architecture: tier it

The tiered logging architecture. All sources feed a routing layer that filters, normalizes, and forks: the high-detection-value subset to the correlation tier, everything to the archive tier, with search and re-hydration on demand.

The trick is realizing that "log everything" and "send everything to the SIEM" are different claims. Generation and retention, AU-12 and AU-11, are satisfied in an archive tier at object-storage prices: everything, compressed and cheap, searchable when the incident demands it. Slower is fine; absent is not. Months to years live here, and so do the compliance floors. The correlation tier, the expensive one, gets only the high-detection-value subset: identity, endpoint, egress, and crown-jewel applications first, hot and indexed, detections running against it, priced per line so every line earns its seat. Between them sits the routing layer, forwarders, collectors, or an observability pipeline, the most underrated component in the whole build: it filters, normalizes, and decides each line's destination. Re-hydration mechanics vary by product; the pattern is what transfers, and the fuller monitoring architecture this pipeline feeds is the kind of thing I treat at design depth in [chapter ref] of the Handbook.

The war story that goes with this diagram is one I tell in first person because I lived it. A firewall allow-log was sixty percent of our ingest and zero percent of our detections. Nobody could name a rule that used it. The compliance need it served was retention, not correlation. Re-routing it to the archive tier paid, by itself, for the identity sources missing from the correlation tier. Every environment has this log. Go find yours.

Rank sources by detection value per gigabyte

Tiering becomes executable when you rank the sources. This table, from slide 14 of the deck, is Part 1's practical takeaway: an onboarding order someone can start Monday.

Identity sits first because most techniques touch an identity somewhere, the volume is tiny, and the detections per gigabyte embarrass everything below it. The inverted build I see constantly is a SIEM full of firewall permit logs and empty of IdP events: high bill, low signal. And row 8 does not say "stop collecting." Bulk allow-logs and flow have real forensic and scoping value after the fact, which is exactly what the archive tier is for. When the ingest budget runs out, the line you draw with this table is defensible, and documented per AU-2.

Work it through: the sixty percent question

Your license is priced per gigabyte per day. Firewall allow-logs are sixty percent of your ingest. What stays in the correlation tier, what tiers to cheap storage, what stops being collected, and what do you tell the auditor about each?

Work it before reading on. A good answer keeps identity, EDR, DNS, egress denies, and crown-jewel allows in the correlation tier; routes the bulk allow-logs to the archive tier, still searchable; and stops collecting almost nothing, reducing verbosity and deduplicating instead. To the auditor: retention obligations are met in the archive tier per AU-11, and the correlation-tier selection is a documented AU-2 decision reviewed on a schedule.

Two partial answers fail differently. The first stops collecting the allow-logs entirely. Push back with the forensics argument: the attacker picks which log mattered, after the fact, and you just deleted the scoping evidence to save disk that object storage sells by the pallet. The second keeps everything hot "to be safe." Price that out loud: it costs real money and buries the queue, and "safe" was doing unexamined work in that sentence. The auditor question is the one people skip and the one that matters. Auditors accept tiering readily when retention is provable and the selection is documented. What they write up is the undocumented default.

Part 2: SIEM in Detail

A SIEM is not magic. It is the Part 1 pipeline with a search engine and a rule engine bolted on. Agents, forwarders, syslog receivers, and API pulls feed vendor parsers and a normalization schema. The event stream then splits, and the split matters: events flow to the correlation engine and to storage at the same time, which is why you can detect in near real time and still investigate last month. The storage tiers underneath (hot, warm, cold) are the tiering argument living inside the product, and the search path runs against them: a query over the hot tier returns in seconds, over cold storage in minutes or hours, and the pricing model you chose decides which tier your incident's data landed in.

Detection engineering is a discipline, not a settings page

A detection is a claim: if an adversary does X here, this rule fires. Claims get tested, not assumed. The shift to make is from "the SIEM detects things" to "we wrote and maintain a set of claims about what we can see." Vendor content packs are a starting inventory, not a coverage plan; half their rules reference sources you did not onboard. Coverage is planned against ATT&CK: build the technique shortlist for your environment, map every rule to it, and work the gaps by priority rather than by content-pack order. The framework meets you halfway: since v18, ATT&CK expresses detection guidance as Detection Strategies and Analytics tied to named log sources, which means the Part 1 priority table is literally the input. No source, no analytic, no coverage, and now you can prove it. Every rule carries metrics (fire rate, true-positive rate, time to triage), because a rule nobody measures is folklore. And the rule set is a living inventory: owned, reviewed, retired. This is AU-6 in practice, said plainly: review and analysis is the control. A log nobody reads is not a control, and a rule nobody maintains is not a detection. The question is never "how many rules do we have." It is "which techniques can we currently see, and which can we not."

A schematic ATT&CK coverage map. Green cells fired correctly in an exercise, amber cells are rules that exist but were never validated, gray cells are gaps: no rule, or no telemetry to feed one.

The coverage map is the most useful artifact a detection program produces, and the discipline is in the shading. Green means a detection fired correctly in a purple-team run or simulation, not "a rule exists." Amber is the uncomfortable middle where most real coverage lives: rules imported, never validated, possibly killed months ago by a parser change. Gray splits into two findings, no rule written versus no telemetry to feed one, and the second routes straight back to the priority table, which is how detection engineering and ingest budgeting become one conversation. The gray cells are the deliverable: an honest map shows leadership what the organization cannot currently see, and that is what funds the next log source. A real map covers the current Enterprise matrix at technique level, usually in ATT&CK Navigator, and as of this writing that means fifteen tactics: v19 split Defense Evasion into Stealth (TA0005) and Defense Impairment (TA0112), so pre-2026 example layers mis-map.

Detection as code, and the rent every rule pays

Detections are software and deserve software practice: the rule as text in a repo with logic, metadata, ATT&CK mapping, and owner; test cases with sample events that must fire it and, just as important, samples that must not; a second engineer reading the logic before production; CI pushing to the SIEM so the repo, not the console, is the source of truth. The argument that lands is one this post already taught you: detections fail silently, so tests are not bureaucracy, they are the only witness. Version history has a quieter payoff: during an incident review, "what could we see on March 3rd" is answerable from a repo and unanswerable from a console someone has been editing live for two years. At orientation depth, know Sigma as the portable rule format with converters to vendor query languages, with the honest caveat that complex multi-source joins usually end up vendor-native. Ask who reviews detection changes at your workplace before they go live. "Whoever wrote it clicks save" is the common answer, and now you know what it costs.

Because every rule pays rent in analyst minutes. A rule that fires forty times a night and is dismissed forty times is not working; it is training the SOC to ignore it, and past the triage capacity line, every additional alert makes you less likely to see the one that matters. Tuning options in order of preference: enrich first, then narrow the logic, then raise the threshold, then suppress known-good. Deletion is last, and genuinely on the list: a rule whose rent permanently exceeds its value should die, and detection-as-code means its logic survives in history. False-positive economics rank the worklist: minutes per dismissal times fires per day, summed per rule. Against the Part 1 numbers, an afternoon spent tuning is the best-paying work in the SOC. Several public breach post-mortems turn on an alert that fired correctly and aged in a queue while the intrusion proceeded. The root cause was volume, not blindness. Alert fatigue is not a morale problem. It is a designed-in defect with a named owner: whoever owns the rule.

Tools: SIEM platforms

One category, one job: collect, normalize, correlate, retain, search, at someone's price. The table gives the market shape as of this writing; the paragraph after it gives the judgment. Representative examples, not endorsements, in no ranked order, and the honest limitation column is exactly that: the thing each platform's own admirers would concede.

What the money buys, honestly: scale you do not have to engineer, parsers maintained by someone else, correlation content out of the box, support with an SLA, and SaaS operations you never staff. Maintained parsers alone justify enterprise spend at scale. But nothing in the commercial column changes the pipeline, and the free half of the table teaches the discipline the commercial half sells. The market consolidates yearly, so verify every name before relying on it.

Part 3: XDR and the Category Map

The acronyms are not competing products. They are different answers to one question: whose telemetry, correlated where?

The category map. Telemetry domains along the bottom (endpoint, network, identity, cloud, email, SaaS), with EDR and NDR as single-domain depth plays, XDR as one vendor's sensors correlated natively, and SIEM as the bring-your-own-everything breadth layer.

EDR and NDR are depth plays: one domain each, done deeply, with response verbs attached (isolate the host, kill the process). SIEM is the breadth play: any source you can ship, from any vendor's tools, correlated centrally, with you doing the assembly and keeping the flexibility and the data. XDR is the vendor's bet in between: their own sensors, correlated natively, integration pre-done, promising SIEM outcomes with less assembly, bounded by the vendor's product line. The dashed edges on the XDR band are deliberate: its boundary is the vendor's catalog, not your environment. Acronym inflation is real: EDR vendors rebrand as XDR, XDR vendors grow SIEM features, SIEM vendors bolt on agents. The letters converge while the sensor lists stay different, so read the sensor list, not the acronym. "Single pane of glass" usually comes up here, and my standing line is that a single pane is only a virtue if the pane shows everything you own. Otherwise it is a single pane over part of the window.

XDR's strengths are real, and I say that from observation: I have watched a mid-size shop get to useful endpoint-plus-identity correlation in a week on XDR after a SIEM project had idled for a year. The costs are equally real, and they are architectural. Coverage is bounded by the vendor's sensors, so your firewalls, homegrown applications, and legacy estate may sit outside the X. The single-vendor trust decision concentrates detection, response, and the evidence itself in one party's cloud. And exit costs are the quiet one: leaving means re-platforming detections and losing history, so portability is a contract clause, not a feature, and the time to ask is procurement, not renewal.

You will meet XDR three ways. Alongside a SIEM, XDR for endpoint-centric speed and the SIEM for everything else and retention: the common enterprise shape. As the small-org SIEM substitute, defensible when the estate mostly is the vendor's own sensors plus M365-style SaaS. And under a SIEM as the integration layer above multiple vendors' tools, where large and regulated shops land, because auditors and forensics both want vendor-neutral retention. The right answer is an inventory question: list your telemetry sources, then check which sit inside the X. Something always lives outside it. The question is whether that something matters.

Tools: EDR, NDR, XDR

Per domain: who watches the endpoint, who watches the wire, who correlates their own. Commercial representatives as of this writing: CrowdStrike Falcon, Microsoft Defender for Endpoint, and SentinelOne for EDR; Corelight (commercial Zeek), Vectra, Darktrace, and ExtraHop for NDR; Microsoft Defender XDR, CrowdStrike, SentinelOne, Palo Alto Cortex, and Trend Vision One for XDR. Open and free: Wazuh for agent telemetry and rules, Velociraptor for DFIR-grade hunting and collection, osquery for the SQL-over-endpoints model; on the wire, Zeek (metadata done right; its logs read like a wire-level textbook), Suricata (signatures, IDS and IPS), and Arkime (full-packet capture, searchable). Open XDR, honestly: there isn't one. The free path is EDR and NDR telemetry into a SIEM you assemble, which you should notice is the SIEM pattern, which is why the category map holds together. What the money buys: managed sensor engineering, native cross-sensor correlation, response actions with vendor backing, and cloud analytics at telemetry volumes a lab cannot generate. The open stack buys you the understanding. Same caveat: representative names in the most consolidation-prone corner of the market.

Part 4: Automation and Incident Response

These share a part because they share a failure mode. The sentence to carry: automating a process you have not run manually just makes the wrong thing happen faster.

So SOAR gets staged by risk-if-wrong, not by vendor demo order. Step one is enrichment: auto-attach asset owner, identity risk, intel hits, and prior cases to every alert. This is where most of the payoff lives and it is nearly free of downside, because enrichment cannot contain the wrong host. It answers the analyst's first five questions before they ask, and it fixes the Part 2 tuning economics, since answerable alerts triage in a fraction of the time. Step two is containment with human approval: one click to isolate a host, disable an account, or block an indicator, machine proposing, analyst deciding. This removes the ticket queue from the 3 a.m. containment path without surrendering judgment; the design artifact is the approval flow, not the automation. Step three, full playbooks, is legitimate and narrow: end-to-end automation only for alert classes you genuinely understand, high confidence, low blast radius, run manually many times first. Known-bad hash on a non-critical endpoint, yes. Anything touching identity or revenue systems, not until the manual runbook has been executed enough times that the edge cases are documented rather than discovered. One more misconception: SOAR is not a maturity shortcut. The playbook library is the asset and the platform is plumbing. Buying the platform before the severity model and runbooks exist automates chaos.

The phishing-triage playbook. Rectangles run unattended: detonation, IOC extraction, the recipient hunt, notification, case recording. Diamonds are human decisions: declaring it malicious, and approving the containment scope.

Trace it with a clock. Accounting reports an invoice-themed email at 9:40. Sandbox detonation shows credential harvesting at 9:42. IOC extraction and the mailbox hunt find 34 recipients and 3 clicks by 9:45. Everything to that point ran unattended and produced an answerable case. Then the diamonds. A human declares it malicious, and notice what just happened: that is the alert-to-incident boundary from the vocabulary section, live. A human approves quarantining 34 mailboxes, because pulling mail from executives' inboxes on a false positive is a career event for somebody. Post-approval, the machine executes in seconds what a manual response does in an hour of clicking. The close step matters most for this post: every run should ask whether a detection improved. Could the SIEM have caught the wave before a user reported it? That feeds the coverage map. The exercise I give students: after fifty clean runs, which boxes would you fully automate? Detonation and the hunt, usually yes; the quarantine approval, most keep human. Defending that split is exactly the step-three judgment the ladder exists to build.

Tools: SOAR and automation

Playbook execution, case management, and integrations: the plumbing under the maturity ladder. Commercial representatives as of this writing: Palo Alto Cortex XSOAR as the category's reference point, Splunk SOAR coupled to the Splunk stack, and Tines and Torq at the SaaS low-code end, with the honest market note that SOAR features increasingly ship inside SIEM and XDR platforms; the standalone category is dissolving into its neighbors. Open and free: Shuffle (workflow automation built for security, and the SOAR in my companion lab guides), TheHive with Cortex (case management plus an analyzer and responder engine, the open IR bench), and n8n or any general-purpose automation platform, not security-branded and entirely capable of ladder steps one and two. What the money buys: maintained integrations, the real cost of DIY automation, because a connector library across forty products is a full-time job that never finishes; plus case management with audit trails, vendor content, and support. The playbook logic itself, the actual asset, you write either way.

Incident response, as SP 800-61r3 actually teaches it now

Teach the current revision, because the internet will happily teach the old one. NIST SP 800-61 Revision 3, finalized April 2025, retired the four-phase lifecycle (preparation; detection and analysis; containment, eradication, and recovery; post-incident) that a decade of study guides and some cert objectives still print. If you learned IR from those guides, this paragraph is your update. The new frame maps incident response onto CSF 2.0's six functions. Govern, Identify, and Protect are the preparation side: roles, policy, and authority to act; assets, risk, and the log-source map; controls that shrink the blast radius. Detect, Respond, and Recover are the incident-handling side, with the clock running. Improvement, the ID.IM category, is drawn as feedback into every function rather than a phase you schedule after the fire. The substance survives the reshuffle: containment and eradication live inside Respond, so if you learned the old model you are relabeling, not relearning. And notice what Identify contains: the log-source map. Part 1 of this post was preparation in 800-61r3's terms all along.

Severity is a table you agree on before the incident, and its value is not the exact rows but that the rows exist, argued by people who were calm. Two design features matter. Criteria are objective, so a tired analyst looks up the level instead of judging it: SEV 1 for active compromise of a crown-jewel system, ongoing exfiltration, or safety impact; SEV 2 for confirmed malicious activity with contained scope; SEV 3 for suspicious activity that could be benign; SEV 4 for policy violations and confirmed-benign triggers that feed tuning. And each level pre-grants notification and authority: SEV 1 means executives and legal now, regulatory clocks checked (GDPR Article 33's 72 hours is the one everyone has heard of, and every regime and many contracts carry their own), and containment authority pre-granted, including permission to take a revenue system down, decided by executives in a conference room months earlier rather than by a shift lead at 2 a.m. with a career on the line. Objective criteria end the 2 a.m. argument. Severity is looked up, not negotiated under adrenaline.

Roles get decided in advance too, because during the incident is a casting call under fire. The incident commander owns the incident, sequences the work, and is allowed to say no to executives: one throat, one timeline, and deliberately not necessarily the best technologist in the building. Technical leads run containment and forensics and report facts to the commander, not to the room. The communications lead is the only voice outward, with an agreed cadence and legal-reviewed wording, because uncoordinated outbound is how minor incidents become news stories. Legal comes in at declaration, not at disclosure: the 72-hour clocks start whether counsel knew or not, and privilege over the investigation is established early or not at all. The executive sponsor holds the decisions that genuinely belong upstairs: pay or don't, disclose or don't, keep running or shut down. Engineers should never be holding those, and executives should know in advance they will be asked. And the communication plan is names and phone numbers on paper, reachable when the phone system, the IdP, and the wiki holding the contact list are the things that are down.

Evidence survives scrutiny only if you handled it like evidence, and most readers will be first on scene rather than the forensic examiner, which is exactly where evidence gets ruined. Working depth means three habits. Hash at acquisition: image, memory capture, log export, hashed at collection and recorded, because the hash is how you prove months later that what you present is what you took. Chain of custody: who collected it, when, from where, who has touched it since, in a written unbroken record, boring on purpose, because a gap in it is opposing counsel's whole afternoon. Time integrity: timelines are the deliverable, and they inherit every clock error from Part 1; AU-8 synchronized time and recorded timezone context make a timeline defensible rather than a story. AU-9, protection of audit information, closes the loop: editing the audit trail is standard attacker tradecraft (Defense Impairment has its own tactic column in current ATT&CK for a reason), so integrity controls on the log path are part of evidence handling, not an ops nicety. Decide acquisition capability in advance: imaging, memory capture, cloud snapshot procedure. That is preparation in 800-61r3 terms. And handle everything as if it will be read aloud to you in a deposition, because at hour one you rarely know whether the incident will end up legal, regulatory, or an insurance claim.

The post-incident review is the stage everyone skips and the only one that compounds. Skipped for an understandable reason: by the time the incident closes, the team is exhausted and the review feels like homework about a test you already failed. It compounds because it is the only mechanism that converts one incident's pain into the next incident's speed. Blameless in format, specific in output: findings with owners and dates, tracked to closure like any other finding. Blameless matters mechanically, not morally; people who expect blame edit the timeline, and an edited timeline teaches you nothing. The questions that compound route backward through this entire post: which log source was missing (Part 1), which detection should have fired (Part 2), where did the timeline stall (this part). Tabletops are the practice discipline: scenario-driven from your threat model rather than generic ransomware every year, on a calendar, with executives in the room for the decisions that are theirs, and with findings that get owners and dates too. A tabletop with no tracked findings was a table read. Every other stage spends capability. This one builds it.

Part 5: SOC Design

Everything so far was the machine. This part is the humans, and the arithmetic is less forgiving than the technology.

Four operating models, argued by trade-off, no favorites. In-house 24x7 buys full context and full control, detections that know your quarter-close from your compromise, at the cost of the headcount arithmetic below; rational for large or regulated shops where security is existential. Co-managed has the provider cover nights, weekends, and commodity triage while your team keeps detection engineering and escalations; the most common defensible mid-size pattern, and it works exactly as well as the handoff seam is designed: shared case management, a jointly agreed severity model, joint runbooks. Without them, co-managed is two SOCs ignoring each other. MSSP is outsourced monitoring and triage against your tooling; you keep response and give up context, because the provider does not know that server, that admin, that pattern. MDR brings the provider's stack, the provider's analysts, and response actions as a service, and it is honestly where most organizations people actually work at will land. The market blurs the last two constantly, so keep the definitions crisp: MSSP watches your tooling and hands you alerts; MDR brings its own stack and takes actions. For MDR especially, the contract clauses are the architecture. Can they isolate a host? What telemetry leaves with you when the contract ends? Can your team see and tune the detections? Ask in procurement, not at renewal.

The 24x7 staffing arithmetic: 168 hours of chair time per week, divided by 40 productive analyst hours, times a 1.5 to 1.7 coverage factor for PTO, sick time, training, and attrition, is roughly 7 analysts for one always-occupied chair.

Walk the arithmetic slowly; it is deliberately un-fancy and it makes the operating-model decision for most rooms. There are 168 hours of chair time in a week whether you staff them or not. Forty productive hours per analyst is generous once meetings and training are counted. The coverage factor is the part managers forget, and it is why every "we'll do 24x7 with five people" plan dies within two quarters: someone takes PTO, someone quits, and now it is forced overtime, which accelerates the quitting. And nobody triages alone: solo overnight coverage means no second opinion on the 3 a.m. containment call and a single point of failure with car trouble. Two chairs minimum lands you at 8 to 12 analysts before you have hired a single detection engineer, threat-intel person, or manager. At loaded cost that is a seven-figure annual line, and that number, next to an MDR quote, is the actual build-versus-buy conversation. Below it, "24x7 in-house" is two exhausted people and a pager, and burnout does the SOC design for you. This arithmetic is why MDR is a rational answer for small organizations, not a concession. Buying coverage you cannot staff is the grown-up move.

Tier the queue or tier the people, and pick deliberately. The tiered model (T1 triages against runbooks, T2 investigates, T3 hunts and engineers) scaled the industry's SOCs for twenty years, and its pathology is well documented: tier 1 becomes a burnout queue of dismissals, context sheds at every handoff, and your best people never touch the alert until it is cold. The tierless model, whoever catches an alert owns it end to end, is not utopia. It is what becomes possible once enrichment and approved-containment automation have eaten the work tier 1 existed to do, and it presumes people capable of owning an investigation; a tierless SOC of juniors with raw alerts is chaos with a flat org chart. The trend line points tierless as automation absorbs triage, but the honest prerequisite is the Part 4 ladder, climbed.

Measure with metrics that drive decisions, and learn to spot the ones that decorate slides. The test is one sentence: if the number went up, what decision would change? If none, decoration. Driving decisions: MTTD and MTTR as trends (is the pipeline getting faster, and where is the drag, usually the triage queue rather than the detection), the coverage map moving from gray toward green over time, false-positive rate per rule as the ranked tuning worklist, and log-source coverage against the priority table as a standing report leadership can fund against. Decorating slides: raw alert counts (measures rule noise, rewards the wrong thing), "billions of events ingested" (measures your bill, dressed as capability), tickets closed per analyst (a speed-running incentive on judgment work), and "99.9% of alerts resolved," where resolved and investigated are not the same word and dismissal at scale scores the same. Every vanity metric was once someone's budget justification, which is exactly why it survives.

The build-versus-buy line falls exactly where context stops compounding. Build and keep in-house: detection engineering for your environment, asset and identity context, the severity model and runbooks, and relationships with the business, because every month in-house makes these better and no vendor can know your quarter-close from your compromise. Buy what commoditizes: 24x7 eyes on glass, commodity triage, sensor engineering, platform operations, because providers do these across hundreds of clients and your doing them in-house adds cost, not context. The sequencing for a growing org is the non-obvious part and I will defend it: MDR first; then bring detection engineering in-house first, because context compounds fastest there and a detection tuned to your environment transfers to any platform; then insource triage last, if ever. The reverse mistake, insourcing triage while outsourcing detection content, buys the burnout queue and none of the compounding. Nobody sells detection content for software only you run; your homegrown application's detections land in the build column every time.

And the slide I refuse to cut, resized to prose: what a two-person IT shop can actually run, because most working readers live at this scale, not at a bank, and material that pretends everyone has a SOC floor teaches them to fail their first design conversation. The defensible small-org stack: an MDR contract, because the staffing math just told you what you cannot build; EDR everywhere plus identity provider logging, because the priority table says those two sources buy the most detection per dollar at any scale; the free stack (Wazuh or Security Onion) beside it, not as production redundancy but as the learning instrument and the local eyes the provider does not sell; and the severity table, the contact list, and one tabletop a year, because paper is free. The reframe that matters: outsourcing the SOC does not outsource the decisions. AU-2 selection is still a documented decision even when the correlation runs in someone else's cloud, and the contract clauses are your architecture review. Your advantage is context: the provider watches a thousand networks and knows none of them. You know one, completely. You know which server matters. The provider never will.

Work it through: zero budget, one afternoon

Zero budget. One afternoon. One log source into your new free SIEM. Which goes in first, and defend the choice by detection value: what rules can you actually write against it by dinner?

The strong answer is identity: domain controller or IdP authentication logs. Top of the priority table, tiny volume, and by dinner you can genuinely write brute-force, password-spray, and impossible-travel detections that fire on real attacker behavior. Equally defensible with argument: endpoint telemetry, if an agent rollout counts as one source, and challenge yourself on whether it does in a single afternoon; or DNS at the resolver, one choke point, C2 and tunneling visibility, cheap. The partial answer to catch yourself giving is the firewall, chosen because it is the easiest device to point at syslog. Ask what detection you will write against permit logs tonight, and watch the answer become "well, dashboards," which is the Part 1 lesson resurfacing: ease of collection is not detection value. The other dodge is "all of them," and it dodges the exercise, and the exercise is the job: ranking under constraint. It is also the first hour of the homelab build below, in disguise.

How and Why a SOC Homelab Moves a Cybersecurity Career

Security operations is a reps discipline, and the homelab is where reps are free. Reading about detection engineering is not writing a rule. Watching your own alert fire is not the same as being told that alerts fire. The pipeline stops being a diagram the first time you personally trace one log from a firewall to an incident ticket you opened on yourself. I have watched that single moment do more for a student than three lectures, and it is available to anyone with an evening and a spare VM.

What transfers from the lab to the job is nearly everything in this post. Pipeline plumbing: you onboard a source, you fix the parser, and you discover the timestamp problem yourself, which teaches Part 1's preconditions in a way no paragraph can. Detection writing and tuning: your first rule will be noisy, I promise you this, and tuning your own noise teaches false-positive economics faster than any lecture, because the analyst-minutes being wasted are yours. Triage reps: you learn what an answerable alert feels like by triaging unanswerable ones. And vocabulary fluency: after a week of watching your own logs become events become alerts, the four words stop being definitions you memorized and become things you can point at.

Then there is the career signal, and I will be direct about it because I have sat on the hiring side. A candidate who can say "here is the coverage map of my own lab, here is a rule I wrote, here is the alert it caught and the one it missed" walks into an analyst interview with evidence instead of adjectives. Hiring managers read homelab stories two ways at once: as initiative, and as proof the fundamentals are load-bearing rather than recited. "The one it missed" is not a weakness in that sentence. It is the strongest part, because it demonstrates you test your claims, and testing claims is the entire discipline of Part 2.

The concrete build. Anchor on Security Onion or Wazuh; either teaches the discipline, and the honest difference fits in two sentences. Wazuh gives you the agent-and-rules SIEM experience and a straight line to writing your own decoders. Security Onion gives you Suricata and Zeek on the wire alongside the Elastic stack, so the network telemetry from Part 1 is visible from day one. Point three sources at it: your lab firewall over syslog, one Windows host with Sysmon, one Linux host with the agent and auditd. Three device classes, three parsers, three rows of the priority table. Write one detection rule by your own hand, failed-then-successful SSH or N failed Windows logons: your first entry on your own coverage map. Then trigger it yourself. Be your own adversary: fail the logins, then succeed, and watch the log become an event become an alert. Triage your own alert and write the three-line case note. You have now run the entire pipeline, end to end, alone.

Then the progression that turns a weekend project into a curriculum: map what you can see to ATT&CK, find the gap, close it with a new source or a new rule, and test your own detections with an open adversary-emulation toolkit, at orientation depth, to make your greens honest. Expectations, set honestly: an evening to stand it up, a weekend to make it interesting, and months to make it teach you. The failures are the curriculum. The parser that will not parse, the alert that will not fire, the clock that drifted: every one is a production incident you got to have for free. My existing homelab posts cover the infrastructure side of this build, and the lab-to-enterprise progression, from a free stack you assembled yourself to the platforms it prepared you for, is the same open-source-to-enterprise discipline I work through in [chapter ref] of the Handbook. The free stack is not the consolation prize. It is the classroom.

The Same Pipeline, Now Carrying Everything

The pipeline reprise, annotated. Placement and time sync at collection, tiering and detection-value onboarding at the fork, coverage maps and detection-as-code at correlation, the SOAR ladder and 800-61r3 discipline at response, and a deliberately chosen staffing model around all of it.

The post keeps the promise the opening diagram made. It is the same chain, logs become events become alerts become incidents, and every arrow is now a design you can name: a placement decision, a documented AU-2 selection, a tested detection, an approval flow, a staffing model chosen on purpose. Cover the callouts and reconstruct them from memory against the bare chain. What you can attach unprompted is what you actually learned.

The Operational Rule

Know your log-source map, your detection coverage, and your escalation path before the incident. During it is too late.

Each clause is a document that either exists at your workplace or does not, and checking costs nothing. The log-source map: what is collected, where it lands, what tier holds it. Part 1's deliverable. The coverage map: which techniques you can see, honestly shaded. Part 2's deliverable. The escalation path: the severity table, the names, the declared authority. Part 4's deliverable. Ask for all three tomorrow. The answers, including the awkward silences, are a maturity assessment nobody had to buy. And notice the rule's shape: it is 800-61r3's preparation functions, Identify and Govern, restated in a sentence an engineer will remember. The framework is what you cite. The sentence is what you use.

Further Study: The Primary Sources, by Number

NIST SP 800-61 Rev. 3 (April 2025): Incident Response Recommendations and Considerations for Cybersecurity Risk Management. Supersedes Rev. 2 and its four-phase model; the IR frame this post taught.

NIST CSF 2.0 (February 2024): the organizing frame. Detect and Respond structured the operations material; Govern, Identify, and Protect carried the preparation argument.

NIST SP 800-92 (2006), with Rev. 1 in draft (the Cybersecurity Log Management Planning Guide): log management planning. Rev. 1 remained in draft at publish; verify its status before citing.

NIST SP 800-53 Rev. 5, AU and IR families: AU-2 selection, AU-6 review, AU-8 time, AU-9 protection, AU-11 retention, AU-12 generation; IR-4, IR-6, and IR-8 for response.

MITRE ATT&CK (v19.2 at publish): the coverage map's vocabulary, at attack.mitre.org with Navigator for the layers. v18 restructured detection guidance into Detection Strategies and Analytics tied to log sources; v19 split Defense Evasion into Stealth (TA0005) and Defense Impairment (TA0112), for fifteen Enterprise tactics.

The standing caveat, which is itself part of the discipline: every document and version above is a currency check before you cite it or build on it. "NIST says" without a number is how bad guidance propagates. The platforms in the tool sections will be renamed, acquired, and re-positioned before some readers change jobs. The pipeline, the four words, the AU family, and the staffing arithmetic will not. Learn the parts that do not move.