Operate

Responding to Incidents

Name: InnerWarden
Author: InnerWarden

How to read an InnerWarden incident, the response actions available, how automatic containment works, and the guardrails you can trust.

Responding to Incidents

For: operators handling a live alert on a host InnerWarden is protecting.

An incident is InnerWarden telling you "I saw something on this machine that looks like an attack, here is what it is and here is what I did or want to do about it." This page is how to read one, what your options are, and how automatic response works so you trust what it does on its own.

If you are tuning out noise rather than handling a real attack, that belongs in Everyday Operations (quieting) or Safe Observe and Allowlist (trusting a specific benign source). This page is for when something might actually be wrong.

Read the incident first

Start by looking at what fired:

innerwarden get incidents --days 7
innerwarden get incidents --days 1 --severity high

Every incident carries a plain-language explained alert: a "what happened and why it matters" line plus the MITRE ATT&CK technique it maps to. You read it as understanding ("InnerWarden saw this, knows what it is, and is handling it"), not as a raw detector name. That explanation comes through on the dashboard and in every notification too, so you get the same story on your phone as in the terminal.

To pivot from an incident to the full story of one attacker:

innerwarden get entity <ip-or-user>       # full timeline: every event, decision, and enrichment for that entity

The entity view stitches together everything InnerWarden knows about that IP or user: when it first appeared, what it tried, its reputation (AbuseIPDB, DShield, GeoIP), and how it was handled. For a visual, journey-style version of the same thing, the dashboard's investigation view is richer, see Dashboard and Notifications.

To see what the agent decided and whether the action actually landed:

innerwarden get decisions --days 7
innerwarden get responses                 # the outcome of each response action

Your response options

Whether the agent acted automatically or left it to you, these are the responses available. The most common manual ones are simple commands:

innerwarden action block <ip> --reason "<why>"      # block an attacking IP
innerwarden action unblock <ip> --reason "<why>"    # reverse a block (false positive, or you are done)

The full set of responses InnerWarden can take, automatically or on your say-so:

Response	What it does
Block IP	Drops the attacker's traffic at the firewall. Six backends (ufw, iptables, nftables, pf on macOS, firewalld, and wire-speed XDP), auto-selected for your host.
Unblock IP	Reverses a block through the response lifecycle. The one path the reconciler will not silently re-apply.
Kill process	Terminates the compromised process tree. Time-bounded.
Suspend user	Drops a sudoers deny file with a TTL, auto-cleans on expiry.
Block container	Pauses a Docker container, auto-unpauses after its TTL.
Rate-limit (nginx)	An HTTP 403 deny at the web layer, with TTL and auto-cleanup.
Monitor	Captures limited traffic for the IP (tcpdump plus metadata) without blocking, so you can watch instead of act.
Honeypot redirect	Sends the attacker into an SSH/HTTP decoy with a fake shell, captures their credentials, commands, and the IOCs they reveal (C2 servers, malware drop points).

Every response is bounded, audited, and reversible. None of them is fire-and-forget: each has a TTL where that makes sense, each writes to the audit trail, and each can be undone. In dry-run mode (the default at install) they log exactly what they would do without doing it. The exact flags for each command live in CLI Reference.

You can also act from your phone (Telegram inline buttons: Block / Ignore / Dismiss, plus /unblock <ip>) or from the dashboard. Both surfaces are covered in Dashboard and Notifications.

Automatic vs. manual response

By default, a fresh install is observe-only and dry-run: it detects and explains, it does not block or kill anything until you arm it. That is intentional, you decide when it is allowed to act. Arming enforcement safely (after a clean baseline and a verified trust list) is the workflow in Safe Observe and Allowlist.

Once armed, the agent can respond on its own for clear-cut cases. The decision path runs each incident through enrichment (IP reputation, GeoIP), then an optional AI provider or the on-device Local Warden model, then a confidence and threshold check, before any skill fires. The detail of that pipeline is in Architecture; what matters as an operator is the guardrails around it:

Ambiguous alerts wait for your decision. When an incident is not clearly an attack and not clearly benign, InnerWarden does not guess and it does not silently drop it. It marks it as needing review and escalates it to you (a Telegram alert with Block / Ignore / Dismiss buttons). Low and Medium reviews time out honestly; High and Critical re-notify until you decide. Nothing important is ever quietly dismissed.
A second opinion, never the only gate. When an AI provider is configured, it can ask for human review rather than auto-acting. Its role is advisory. When no AI is configured, the full review path still runs deterministically, the AI is an enhancement, never a dependency.
Your decisions teach it. When you Block / Ignore / Dismiss, that decision feeds back as a label to the Local Warden model, so it gets better at your host over time.

When the kernel acts instantly

For the most dangerous patterns, the eBPF layer can block at the kernel before the agent even triages. When the kill-chain detection in the kernel blocks an exec, the full response pipeline fires automatically: the sensor raises a Critical incident, the agent kills the process tree, blocks the C2 IP at wire speed (XDP), captures forensics (a /proc snapshot plus socket state), propagates the block to any mesh peers, and notifies you. This is the one path where containment happens first and triage follows, because the alternative is letting the exploit run.

High and Critical incidents also trigger automatic forensics capture (a packet capture plus a /proc snapshot) so the evidence exists even if the attacker cleans up.

The circuit breaker

InnerWarden will not let an alert storm turn into a response storm. Two limits protect both you and the box:

AI rate limiting and a circuit breaker. The agent caps AI calls per tick and, when incident volume spikes, suspends AI triage for that tick and auto-resets after a cooldown. A flood cannot run up your AI bill or stampede the decision path.
Decision cooldown. The same action against the same detector and entity will not re-fire within a cooldown window, so one persistent attacker does not generate a thousand duplicate blocks.
Burst summary. When 50 or more auto-blocks happen in an hour, you get a single "heavy attack, all contained" message that names the server and breaks down what kind of attack it was, instead of 50 separate alerts. That is normal internet background noise being handled, not 50 emergencies.

Containment you can trust

A block is only worth something if it is real. InnerWarden verifies containment against the live firewall, not an internal record:

A High or Critical case whose IP the firewall is actually dropping reads as Contained, verified by checking the live ufw/iptables/nftables state, not by trusting a row in a database. If the firewall and the record disagree, the live firewall wins and the block is re-applied. An attacker cannot get a free pass because some internal record claimed it was already blocked.
This truthful-containment behavior is scoped to recon-class signals, so anything showing active harm stays surfaced in "needs your attention" even when its IP is blocked. You never lose sight of an attacker that is still doing damage just because their traffic is being dropped.

When you reverse a containment, use innerwarden action unblock (or /unblock <ip> from Telegram). That is the one path the reconciler treats as a deliberate operator decision and will not re-apply on the next sweep.

The audit trail

Every decision and every action is written to a hash-chained local audit trail on the box. It records what was detected, what the agent (or you) decided, the confidence, the target, and the result, chained so tampering is detectable. It is yours and it stays local, there is no cloud control plane.

You can verify the chain and review admin actions from the dashboard's Compliance view. How long each record is kept (and how to export or erase it) is owned by Privacy and GDPR; how the audit trail maps to compliance controls is in ISO 27001 Mapping.