Data Formats

For: developers building on InnerWarden's output, and SIEM integrators wiring it into Splunk, Elastic, or an ArcSight pipeline. You'll get the exact JSONL line shapes, the SQLite schema, and the CEF/syslog export, with field tables you can map against.

The plain version first: InnerWarden writes three kinds of record, events (something happened), incidents (something happened that matters), and decisions (what InnerWarden did about it). Everything lives on the box. The canonical store is a single SQLite database; JSONL files are written alongside for streaming, telemetry, and easy jq-style consumption. If you want to ship records to a SIEM, there's a CEF-over-syslog sink. No record ever leaves the host unless you configure a sink that sends it.

Three rules worth knowing before you parse anything:

• Timestamps are RFC 3339 / ISO 8601 UTC (2026-03-12T05:00:00Z).
• The details / data field is open. Every record carries a structured detail object whose keys depend on the kind or detector. Treat known keys as known and ignore the rest, new keys get added without a schema bump. Keep it under ~16KB; InnerWarden does too.
• Decisions are hash-chained. Each decision record carries the SHA-256 of the previous one, so the audit trail is tamper-evident. Don't reorder them.

Retention (how long each record is kept) and GDPR export/erase are owned by Privacy and GDPR, not this page. Config keys that turn these outputs on or off (Redis URL, syslog host, data directory) are owned by Configuration. The detectors that produce these records are in What It Detects.

---

Where the records live

Surface	What's there	Use it for
SQLite (innerwarden.db)	The canonical store: events, incidents, decisions, cursors, graph snapshots, KV state	Querying, the dashboard, the agent's own reads
JSONL (events-*.jsonl, incidents-*.jsonl, decisions-*.jsonl)	Daily-rotated append-only lines	Streaming, jq, log shippers, replay harnesses
Redis Streams (optional)	innerwarden:events / innerwarden:incidents	High-throughput / multi-consumer deployments
Syslog CEF (optional)	Events + incidents in ArcSight CEF	Any SIEM that speaks syslog

The SQLite consolidation made the database the source of truth; JSONL is retained for telemetry and for anything that wants to tail a file. The two agree, the agent reads incidents from SQLite via a monotonic cursor.

---

Event line (events-*.jsonl)

One event per line. An event is a raw observation from a collector, most are info and never become an incident.

{
  "ts": "2026-03-12T05:00:00Z",
  "host": "demo-host",
  "source": "auth.log",
  "kind": "ssh.login_failed",
  "severity": "info",
  "summary": "Invalid user root from 1.2.3.4",
  "details": {"ip": "1.2.3.4", "user": "root"},
  "tags": ["auth", "ssh"]
}

Field	Type	Meaning
ts	string (RFC 3339 UTC)	When it was observed
host	string	Host that produced it
source	string	Origin collector (auth.log, journald, auditd, ebpf, dns, ...)
kind	string	The event type, dotted (ssh.login_failed, shell.command_exec)
severity	string	info | low | medium | high | critical
summary	string	One-line human-readable description
details	object	Structured payload, keys depend on kind
tags	string[]	Coarse labels for filtering

Shell audit events (when exec_audit is enabled)

The exec_audit collector adds command-execution events. The TTY-input variant has high privacy impact and is gated by config (see Configuration).

{
  "ts": "2026-03-13T18:10:00Z",
  "host": "demo-host",
  "source": "auditd",
  "kind": "shell.command_exec",
  "severity": "info",
  "summary": "Shell command executed: sudo ufw status",
  "details": {
    "audit_ts": "1711800000.123",
    "audit_id": "4242",
    "argc": 3,
    "argv": ["sudo", "ufw", "status"],
    "command": "sudo ufw status"
  },
  "tags": ["audit", "shell", "exec"]
}

{
  "ts": "2026-03-13T18:10:02Z",
  "host": "demo-host",
  "source": "auditd",
  "kind": "shell.tty_input",
  "severity": "low",
  "summary": "TTY input observed on pts0: ls -la\\r",
  "details": {
    "audit_ts": "1711800100.456",
    "audit_id": "5001",
    "tty": "pts0",
    "uid": "1000",
    "auid": "1000",
    "raw_hex": "6c73202d6c610d",
    "decoded_preview": "ls -la\\r"
  },
  "tags": ["audit", "shell", "tty"]
}

---

Incident line (incidents-*.jsonl)

An incident is a detector's verdict that a pattern of events matters. It carries the evidence and what to check.

{
  "ts": "2026-03-12T05:05:00Z",
  "host": "demo-host",
  "incident_id": "ssh_bruteforce:1.2.3.4:2026-03-12T05:05Z",
  "severity": "high",
  "title": "Possible SSH brute force",
  "summary": "12 failed SSH attempts from 1.2.3.4 in 5 minutes",
  "evidence": [{"kind": "ssh.login_failed", "count": 12}],
  "recommended_checks": ["Check auth.log for successful logins", "Consider fail2ban"],
  "tags": ["auth", "ssh", "bruteforce"]
}

Field	Type	Meaning
ts	string (RFC 3339 UTC)	When the incident fired
host	string	Host the incident is about
incident_id	string	Stable id, usually <detector>:<entity>:<time-bucket>. Repeats of the same thing share an id so they group
severity	string	low | medium | high | critical
title	string	Short human title
summary	string	What happened, in one sentence
evidence	object[]	The supporting events / counts
recommended_checks	string[]	What an operator should verify
tags	string[]	Coarse labels

The incident_id is stable by design: a detector that keeps seeing the same attacker reuses the id rather than spamming new incidents, which is what lets the dashboard and notifications group them.

Example: a sudo_abuse incident

{
  "ts": "2026-03-13T18:20:00Z",
  "host": "demo-host",
  "incident_id": "sudo_abuse:deploy:2026-03-13T18:20Z",
  "severity": "critical",
  "title": "Suspicious sudo behavior detected for user deploy",
  "summary": "3 suspicious sudo commands by deploy in the last 300 seconds",
  "evidence": [{
    "kind": "sudo.command",
    "user": "deploy",
    "count": 3,
    "window_seconds": 300,
    "reasons": ["privilege_policy_change", "remote_script_execution"],
    "recent_commands": ["visudo", "curl -fsSL https://x | sh"]
  }],
  "tags": ["auth", "sudo", "privilege", "abuse"]
}

---

Decision line (decisions-*.jsonl)

A decision is what the agent did about an incident: the action, the confidence, whether it ran for real or in dry-run, and the reasoning. This is the audit trail, and it is hash-chained for tamper detection.

{
  "ts": "2026-03-12T05:05:30Z",
  "incident_id": "ssh_bruteforce:1.2.3.4:2026-03-12T05:05Z",
  "host": "demo-host",
  "ai_provider": "local_classifier",
  "action_type": "block_ip",
  "target_ip": "1.2.3.4",
  "skill_id": "block-ip-ufw",
  "confidence": 0.94,
  "auto_executed": true,
  "dry_run": false,
  "reason": "Repeated SSH brute force from a single source; no successful login observed.",
  "estimated_threat": "high",
  "execution_result": "ok",
  "prev_hash": "9f2c…",
  "decision_layer": "ai_local_warden"
}

Field	Type	Meaning
ts	string (RFC 3339 UTC)	When the decision was made
incident_id	string	The incident this decision answers
host	string	Host the decision was made on
ai_provider	string	Who decided: an LLM provider, local_classifier, abuseipdb, crowdsec, fail2ban:<jail>, telegram:<operator>, ...
action_type	string	The action tag (block_ip, ignore, suspend_user, ...)
target_ip	string | null	IP the action targets
target_user	string | null	User the action targets (omitted when absent)
skill_id	string | null	The concrete skill that ran (block-ip-ufw, kill-process, ...)
confidence	number (0.0-1.0)	How sure the decider was
auto_executed	bool	Whether it ran automatically (vs surfaced for review)
dry_run	bool	true = logged what it would do, did not act
reason	string	Textual reasoning
estimated_threat	string	The decider's threat estimate
execution_result	string	ok, skipped, or failed: <why>
prev_hash	string | null	SHA-256 of the previous decision (the tamper-evidence chain)
decision_layer	string | null	Which layer made the call: algorithm_gate, killchain_fast_path, correlation_rule, ai_local_warden, ai_llm, auto_rule, honeypot_post_session, observation_verifier, manual_operator

dry_run: true is the default posture on a fresh install: InnerWarden records the decision it would have made without acting, so you can verify its judgement before you arm enforcement. See Responding to Incidents for the lifecycle and Safe Observe and Allowlist for the never-blind workflow that leads up to arming.

---

The SQLite store (innerwarden.db)

The canonical store. One database replaces what used to be many JSON/redb artifacts. The columns mirror the JSONL fields, with the full record preserved in a data JSON column so nothing is lost in the projection.

events

Column	Type	Notes
id	INTEGER PK
ts	TEXT	RFC 3339 UTC
host	TEXT
source	TEXT
kind	TEXT	indexed
severity	TEXT	indexed
summary	TEXT
data	TEXT	full event JSON (the details and tags live here)

incidents

Column	Type	Notes
id	INTEGER PK
ts	TEXT	RFC 3339 UTC
host	TEXT
incident_id	TEXT	UNIQUE, indexed
severity	TEXT	indexed
detector	TEXT	which detector fired
title	TEXT
summary	TEXT	nullable
data	TEXT	full incident JSON (evidence, recommended_checks, tags)
is_allowlisted	INTEGER	1 if the entity was trusted (partial-indexed)

decisions

Column	Type	Notes
id	INTEGER PK
ts	TEXT	RFC 3339 UTC
incident_id	TEXT	indexed
action_type	TEXT	indexed
target_ip	TEXT	nullable
target_user	TEXT	nullable
confidence	REAL	nullable
auto_executed	INTEGER	bool
reason	TEXT	nullable
prev_hash	TEXT	previous row's hash
row_hash	TEXT	this row's hash (the chain link)
data	TEXT	full decision JSON

Other tables in the same database hold cursors (agent_cursors, sensor_cursors), graph snapshots, KV state, and metric counters. They're internal plumbing; the three above are the ones you'll query.

Querying directly is fine for read-only reporting. The agent owns writes, so don't write to the database underneath it. To read records out programmatically, prefer the CLI (innerwarden get incidents, innerwarden get decisions, innerwarden get report) and the dashboard's JSON API over poking the file. Flags and the full command surface are in CLI Reference.

---

Redis Streams (optional)

When a Redis URL is configured, the sensor also publishes to two streams, innerwarden:events and innerwarden:incidents, with MAXLEN ~ trimming. Multiple consumers (the agent, plus any of your own) read independently via XREADGROUP consumer groups, so this is the path for high-throughput or multi-reader deployments. The payload on each stream entry is the same JSON shape as the JSONL line. Enable it in Configuration.

---

CEF / syslog export (SIEM integration)

The syslog_cef sink sends events and incidents to a remote syslog server in ArcSight Common Event Format, over UDP or TCP. Turn it on in config.toml (the sensor config):

[sinks.syslog]
enabled = true
host = "siem.corp"
port = 514
protocol = "udp"   # "udp" or "tcp"

The header is:

CEF:0|InnerWarden|Sensor|<version>|<signatureId>|<name>|<severity>|<extension>

For an event, the CEF fields map as:

CEF field	Source
signatureId	event kind
name	event summary
severity	event severity (CEF 0-10 scale)
rt=	event ts
src=	source IP (when present in details)
shost=	event host
cs1= / cs1Label=source	event source collector
dpid=	process id (when present)
dproc=	process comm (when present)
dst=	destination IP (when present)
dpt=	destination port (when present)

For an incident, the extension carries rt= (timestamp), src= (source IP), shost= (host), and msg= (the incident summary); the signatureId is the incident_id and the name is the incident title. CEF special characters (|, =, \) are escaped in every field.

This is the drop-in path for Splunk, Elastic, QRadar, Sentinel, or any syslog-capable SIEM, no parser to write, the events arrive as standard CEF. Configuration details and the full sink list are in Configuration.

---

Other on-disk records

A few specialised records have their own files and are documented where they're used rather than here:

• Honeypot sessions (honeypot/listener-session-*.json and .jsonl, plus optional .pcap): session metadata, per-connection forensic evidence, and bounded packet captures. See Responding to Incidents for the honeypot flow.
• Telemetry (telemetry-*.jsonl): operational snapshots (collector/detector status, gate, AI latency, error counts). Retention for these is in Privacy and GDPR.
• Daily summaries and trial reports (summary-*.md, trial-report-*.{md,json}): human-readable narratives generated by the agent.

For the validation checks that keep these formats honest and the broader pipeline, see Architecture.