Skip to content
Start here

Architecture

How InnerWarden fits together: a deterministic sensor and an interpretive agent that watch the host from the kernel up, then decide and act.

Architecture

For: anyone who wants the mental model of how InnerWarden is put together, before diving into any one part.

The shape of InnerWarden is simple to hold in your head: one machine watches itself from the kernel up, decides what matters, and acts. Two Rust services split that work. A sensor does nothing but collect, deterministically, with no AI and no outbound network. An agent reads what the sensor produced, makes sense of it, decides, and acts. Everything lives on the box. There is no cloud control plane, and the data and the audit trail never leave.

                                   the host
  ┌──────────────────────────────────────────────────────────────────────┐
  │  Ring -2  firmware / UEFI / SMM                                        │
  │  Ring -1  hypervisor                                                   │
  │  Ring  0  kernel        <── eBPF programs hook here                    │
  │  Ring  3  userspace     <── processes, files, network, AI agents      │
  └──────────────────────────────────────────────────────────────────────┘
                    │ activity flows up through every ring
                    v
   SENSOR  (deterministic, no AI, no outbound network)
   ┌────────────────────────────────────────────────────────────┐
   │  eBPF programs + collectors  ──>  detectors  ──>  events    │
   │       (watch the rings)         (find patterns)  + incidents│
   └────────────────────────────────────────────────────────────┘
                    │ events-*.jsonl / incidents-*.jsonl + SQLite mirror
                    v
   AGENT  (interpretation, may call out)
   ┌────────────────────────────────────────────────────────────┐
   │  correlation  ──>  decision  ──>  skills (block / kill /    │
   │  (stitch into        (triage)       suspend / honeypot)     │
   │   attack chains)                    + notifications         │
   └────────────────────────────────────────────────────────────┘
                    │
                    v
       dashboard · Telegram / Slack / Discord · CLI · hash-chained audit trail

The split is the whole point. The sensor is boring and trustworthy on purpose: it has no judgement to be fooled and nothing to phone home with. All the interpretation, all the "is this actually bad", all the calls to AI or threat intelligence, live in the agent. If you want the full inventory of what it watches and detects, that lives on What It Detects; this page is just the map.

The two services

Sensor (innerwarden-sensor). Deterministic collection, and only that. It loads the eBPF programs into the kernel, runs the collectors that read logs and inventory the host, captures raw network traffic, runs every event past the stateful detectors, and writes the results to disk. It never calls an LLM, never makes an outbound connection, and its collectors are fail-open: one collector dying never crashes the process. This is the part you can reason about as "it just reports facts".

Agent (innerwarden-agent). The interpretive layer. It reads what the sensor produced, correlates single events into attack chains, decides what to do, executes response skills, and sends notifications. It is allowed to call external APIs (AI providers, IP reputation, GeoIP) because deciding sometimes needs context the box alone does not have. It also hosts the dashboard and the loopback API that AI agents consult.

You drive both through one CLI binary, innerwarden. Day-to-day operation of the two services, the supervisor that keeps them alive, the systemd units, and the permissions they need are all owned by Everyday Operations.

The rings: why it watches so low

Most host security tooling lives in userspace (Ring 3) and watches other userspace. InnerWarden watches the whole stack, because the most serious compromises happen below where userspace tools can see:

  • Ring 3, userspace: processes, files, network connections, and the AI agents running on the box. The bulk of everyday detection.
  • Ring 0, the kernel: where the eBPF programs attach. Syscalls, process lifecycle, file opens, and network events are seen as the kernel itself sees them, which is much harder for malware to hide from than reading a log after the fact.
  • Ring -1, the hypervisor: integrity checks for VM detection and KVM-level tampering, run on a timer inside the agent.
  • Ring -2, firmware / UEFI / SMM: periodic firmware and System Management Mode audits, also on a timer inside the agent.

The firmware (Ring -2), hypervisor (Ring -1), and DDoS-shield checks all run inline inside the agent on their own timers, not as separate daemons. So "does it really cover firmware?" has a simple answer: yes, and it is deployed, just folded into the agent rather than running as its own service.

The pipeline, end to end

Activity flows in one direction, from the kernel to a decision:

  1. eBPF programs + collectors watch the rings. In the kernel, eBPF programs hook syscalls and process, file, and network events. In userspace, collectors read auth and system logs, inventory Docker, watch the filesystem in real time, snapshot process memory, and capture DNS, HTTP, and TLS straight off the wire. Native capture replaced the old approach of bolting on external tools (see Native collectors below).
  2. Detectors find patterns. Every event runs through the stateful detectors: SSH brute force, credential theft, port scans, reverse shells, ransomware bursts, C2 beacons, container escape, and many more. A detector that fires produces an incident.
  3. The sensor writes it down. Events and incidents are written as dated JSONL (events-*.jsonl, incidents-*.jsonl) and mirrored into a local SQLite store. The exact on-disk schema is owned by Data Formats. For very high-throughput deployments there is an optional Redis Streams sink.
  4. The agent correlates. Single events are often innocent on their own. The cross-layer correlation engine stitches them into attack chains, pivoting on the same IP, user, container, or process across the firmware, hypervisor, kernel, userspace, network, and honeypot layers. A privilege escalation followed by a sensitive read followed by an outbound connection is a chain, not three coincidences.
  5. The agent decides. Triage runs through a gate, an allowlist check, enrichment, and then an AI provider, which may be the on-device Local Warden model (no API key, runs locally) or a cloud LLM. The output is a decision with a MITRE technique attached and a plain-language explanation.
  6. The agent acts, or would. When enforcement is armed, the decision drives a skill: block the IP (via UFW, iptables, nftables, pf, firewalld, or wire-speed XDP), kill the process, suspend the user, isolate the container, or redirect to a honeypot. High and critical incidents also trigger forensic capture and notifications. In the default observe-only posture, the decision is logged as what it would have done, and nothing executes.
  7. Everything is recorded. Decisions land in a hash-chained local audit trail you can verify, plus the dashboard and your chat channels. Tamper with the chain and verification fails.

The two-loop rhythm inside the agent: a fast loop (every couple of seconds) handles new incidents through gate → allowlist → enrich → decide → act → notify. A slow loop (every half-minute or so) does the background work: correlation, baseline learning, attacker-intelligence consolidation, data retention, and the firmware/hypervisor timers.

Baseline learning: detection without a rule

Alongside the rule-based detectors, the agent learns this host's normal over an initial training window: event rates by source and hour, which processes spawn which children, when users normally log in, and where each process normally connects out. After it has a baseline, it flags the deviations a static rule would miss: an unexpected nginx → sh lineage, a login at 3 a.m. that never happens, a brand-new outbound destination, or the eerie silence of a sensor that suddenly stops reporting (a classic sign something disabled it). This is why arming enforcement before a clean baseline is a bad idea, covered in Safe Observe and Allowlist.

The local store

There is no database server to run. The sensor writes dated JSONL files and mirrors them into a single SQLite store under the data directory; the agent reads incrementally using byte-offset cursors (or Redis Streams consumer groups in the high-throughput setup). The attacker-intelligence profiles use an embedded key-value store with a JSON snapshot. Field names and the JSONL schema are documented on Data Formats; how long things are kept and how to export or erase them is owned by Privacy and GDPR.

Native collectors, not bolt-ons

Earlier versions of InnerWarden leaned on an "integrated setup" that wired in external tools (Falco, Suricata, osquery) as data sources. That approach was dropped in favour of native collectors. DNS, HTTP, and TLS are now captured directly off the wire via raw sockets (with JA3/JA4 TLS fingerprinting), kernel visibility comes from InnerWarden's own eBPF programs, and host inventory comes from native collectors. The result is one install with no external dependencies to keep running, version-match, or parse. (Suricata remains optional for deep packet inspection in compliance-driven environments, but nothing requires it.)

Where to go next