Skip to content
Kernel Security

How InnerWarden Uses eBPF to Watch the Linux Kernel

11 min read

Most security tools read logs. They wait for sshd to write a "Failed password" line, parse it, and react. By the time the log exists, the event already happened. The attacker already tried the password. The process already ran. The file was already opened.

eBPF changes this. It lets you run small, verified programs inside the Linux kernel itself. You see every syscall, every process, every network connection the moment it happens, before any log is written. InnerWarden's sensor defines 74 eBPF programs in its source to watch a Linux host at the kernel level; how many of them load on a given machine depends on its kernel.

What is eBPF

eBPF (extended Berkeley Packet Filter) is a technology that lets you run sandboxed programs inside the Linux kernel without modifying kernel source code or loading kernel modules. Every eBPF program is verified by the BPF verifier before it runs. The verifier checks that the program terminates, does not access invalid memory, and cannot crash the kernel. If verification fails, the program is rejected.

This means eBPF programs are safe by construction. They cannot cause kernel panics. They cannot corrupt memory. They cannot enter infinite loops. The verifier guarantees it. This makes eBPF ideal for production security monitoring: you get kernel-level visibility with kernel-level safety.

Why eBPF beats log parsing

Traditional security tools work by tailing log files. auth.log, syslog, journald, nginx access logs. This approach has three fundamental problems:

  • Timing: logs are written after the event. By the time you parse "Accepted publickey for root," the attacker is already logged in. eBPF sees the syscall before the log line exists.
  • Evasion: attackers can clear logs, redirect output, or kill the logging daemon. You cannot clear a tracepoint. eBPF hooks into the kernel itself, not into userspace log writers.
  • Overhead: log parsing means reading files, applying regex, and maintaining file watchers. eBPF runs in kernel space and hands events to the sensor through a ring buffer: no file tailing, no regex over log lines, no polling.

eBPF does not replace logs. InnerWarden still monitors auth.log, journald, nginx, and other sources. But eBPF adds a layer that an attacker in userspace cannot switch off by editing or deleting a log file.

The eBPF program families InnerWarden defines

The sensor's source defines 74 eBPF programs: kprobes and kretprobes, tracepoints and raw tracepoints, LSM hooks, and one XDP program. That is a count of programs defined in source, not of programs loaded: what loads depends on the kernel, and the sensor keeps running with whatever the verifier accepts. Together they cover process execution, network activity, file access, privilege escalation, execution policy, and network-level blocking. The main hook points:

Process, network and file probes
execve: every process execution

Fires every time a process is created on the system. Captures the binary path, arguments, UID, GID, and parent PID, and a refused or failed attempt as well as the exec that ran. This is how InnerWarden detects reverse shells, crypto miners, and suspicious binaries running from /tmp.

connect: every network connection

Fires every time a process opens a network connection. Captures the destination IP, port, protocol, and the PID making the connection. This detects C2 callbacks, data exfiltration, and unexpected outbound connections from server processes that should never initiate network traffic.

openat: sensitive file access

Fires every time a file is opened. Captures the file path, flags (read/write/create), and the process doing the access. This detects attempts to read /etc/shadow, modify SSH authorized_keys, tamper with system binaries, or access sensitive configuration files.

Kprobe
commit_creds: privilege escalation detection

This kernel function is called whenever a process changes its credentials. InnerWarden watches for UID transitions to 0 (root) from non-root processes. When a normal user process suddenly becomes root outside of sudo or su, it is almost certainly a privilege escalation exploit: a kernel vulnerability, a SUID binary abuse, or a container escape. This kprobe catches it at the exact moment it happens.

LSM hooks
bprm_check_security: execution blocking

This Linux Security Module hook fires before a binary is executed, and an LSM program attached to it can refuse the exec: the binary never runs and the call returns EPERM. InnerWarden attaches two programs here. One can refuse execution from world-writable directories such as /tmp and /dev/shm, where attackers drop payloads, when that policy is switched on. The other is the Execution Gate: once armed after a clean rehearsal, it refuses any program that is not on the agent's allowlist. Both need a kernel with bpf in its active LSM stack. Other LSM hooks (file_open for the Secret Read Guard, mmap, ptrace, bpf and user namespaces) work the same way.

1 XDP Program
XDP blocklist: dropping blocked IPs early

XDP (eXpress Data Path) runs at the network driver level, before the kernel networking stack even sees the packet. InnerWarden maintains a BPF HashMap of blocked IPs. Every incoming packet is checked against the map. Blocked packets are dropped immediately. No TCP handshake, no SYN-ACK, no response. The attacker gets silence. Dropping a packet here is cheaper than letting it reach a userspace firewall; we have not published a packets-per-second figure for it.

Container awareness through cgroup tracking

Every eBPF event includes the cgroup_id of the process that triggered it. On a container host, each container runs in its own cgroup. This means InnerWarden knows exactly which container each event belongs to: which container spawned the suspicious process, which container opened the network connection, which container tried to escalate privileges.

This is critical for container security. A process executing /bin/sh inside a production web container is very different from the same command on the host. Inner Warden can correlate eBPF events with Docker container metadata to provide context-aware detection and response.

Filtering the noise before it leaves the kernel

Hooking syscalls on a busy server produces a lot of events: a web server can make thousands of connect, openat and clone calls a second. Most of that is filtered inside the eBPF programs themselves, before an event reaches the ring buffer and user space, with three maps:

  • COMM_ALLOWLIST: known-safe, high-volume process names (system daemons, package managers), filtered per handler, so a name allowed for one kind of event is still seen for the others.
  • CGROUP_ALLOWLIST: trusted container cgroups, such as your monitoring stack, whose syscall patterns are expected.
  • PID_RATE_LIMIT: a per-process rate limit, so one process opening files thousands of times a second sends a bounded number of events.

The hooks still fire for those processes; the events are simply dropped in the kernel instead of being shipped to user space and thrown away there.

Writing Rust for the BPF verifier

InnerWarden's eBPF programs are written in Rust, compiled as a #![no_std] crate targeting the bpfel-unknown-none architecture. This is not normal Rust. There is no standard library, no heap allocation, no Vec, no String, no println. The BPF verifier enforces a 512-byte stack limit per function. Every variable must be on the stack or in a BPF map.

eBPF Rust constraints
// No heap: no Vec, String, Box, Arc
// No standard library: #![no_std]
// 512-byte stack limit per function
// All loops must have bounded iteration counts
// No function pointers or dynamic dispatch
// No floating point arithmetic
// Every memory access must be bounds-checked

// Shared types live in a separate crate:
// crates/sensor-ebpf-types/
// Used by both the eBPF programs and userspace loader

InnerWarden uses the Aya framework to load and manage eBPF programs from userspace. Aya is a pure-Rust eBPF library with no dependency on libbpf, clang, or BCC. The eBPF programs are compiled at build time and embedded in the sensor binary. Shared types between kernel and userspace live in a dedicated crate (sensor-ebpf-types) to ensure type safety across the boundary.

Loaded at start, attached until stop

The programs are compiled at build time, embedded in the sensor binary, and loaded through Aya when the sensor starts. Each one the kernel's verifier accepts stays attached until the sensor stops; a kernel that lacks a hook or refuses a program does not stop the others.

There is no polling, no file watching, no periodic scan: events flow from the kernel to userspace through ring buffers. We do not quote an overhead figure here, because it depends on the workload and on which programs your kernel loads; measure it on your own hosts.

What eBPF catches that logs miss

There is an entire class of attacks designed to evade log-based detection. eBPF catches them because it operates at a layer that attackers cannot manipulate from userspace.

  • Fileless malware: malware that runs entirely in memory, never touching the filesystem. No file means no file-integrity alert. But the exec probe sees the process start.
  • In-memory attacks: an attacker who gains code execution inside an existing process (e.g., through a buffer overflow) never creates a new log entry. But the connect and openat probes see the unexpected network connections and file accesses from that process.
  • Log-clearing attackers: the first thing a skilled attacker does after gaining access is clear logs. They truncate auth.log, delete bash_history, and stop syslog. None of this affects eBPF tracepoints. The kernel does not care about your log files.
  • Container escapes: when a process escapes its container namespace, it typically involves credential changes (caught by commit_creds kprobe) and unexpected process execution (caught by the exec probe). The cgroup_id tracking reveals that the process originated in a container but is now operating at the host level.
  • Privilege escalation exploits: kernel exploits that escalate privileges do not write log lines. They modify in-kernel credential structures. The commit_creds kprobe fires at the exact moment this happens, before the attacker can take any action with their new privileges.

How it fits into the pipeline

eBPF events flow through the same sensor pipeline as every other collector. The sensor receives events from the kernel via ring buffers, normalizes them into the standard Event format, and passes them through stateful detectors. The detectors correlate eBPF events with other sources. A privilege escalation caught by the kprobe is correlated with the SSH session that preceded it, the docker container that hosted it, and the IP address that initiated it.

Event flow
kernel tracepoint/kprobe/LSM/XDP
  → BPF ring buffer
    → sensor (eBPF collector)
      → detectors (correlation + state)
        → incidents-*.jsonl
          → agent (AI triage + enrichment + response)

The agent then enriches the incident with AbuseIPDB and GeoIP data, runs it through the AI confidence scorer, and executes the appropriate response: block the IP, suspend the user, or kill the process.

Enable eBPF monitoring

eBPF monitoring is part of the paid host sensor (Pro and Enterprise) and needs a Linux kernel with BTF (BPF Type Format), which most current distributions ship. The kernel guards (LSM hooks, including the Execution Gate) also need bpf in the active LSM stack; see Safe observe and allowlist.

The programs your kernel accepts load when the sensor starts. If none can load, the sensor continues with its other collectors. eBPF is additive, not required.

What to do next