Surviving a Zero-Day React2Shell Attack: How We Recovered Root & 100% CPU Under Peak Live Traffic

1. The Incident: Complete Resource Starvation at 100% CPU

Under standard operational conditions, our production multi-core Linux host runs at a stable 15–20% CPU utilization and ~30% RAM utilization. This server concurrently powered the live LMS video delivery for Pro Trainer IT students while handling continuous subscription transactions on SubsDrop.

At approximately 03:40 AM, automated health check probes reported 504 Gateway Timeouts. Within two minutes, SSH daemon connection attempts began hanging with TCP reset timeouts. Out-of-band hypervisor metrics showed all 8 vCPUs pinned at 100.0%, RAM pegged at 99%, and disk I/O throughput maxed out.

Dozens of customer tickets poured in: students couldn't access ongoing cohorts and buyers were stuck on payment verification. The host operating system was completely suffocating.

2. Out-of-Band VNC Rescue & Process Inspection

Because standard SSH connection handshakes were denied by process thread pool starvation, I opened a low-level out-of-band VNC emergency console provided by our cloud hypervisor. This accessed the TTY console directly without needing network socket allocation.

Executing 'top' showed multiple worker processes disguised as legitimate Linux kernel threads like '[kworker/u16:3]' and '[systemd-journald-helper]', each consuming 99.8% CPU.

Inspecting '/proc/<pid>/exe' revealed that these processes were actually executing compiled XMRig binaries hidden under '/tmp/.sysd' and '/dev/shm/.cache'.

# Step 1: Freeze rogue process tree immediately
kill -STOP 41922 41923

# Step 2: Uncover real binary path and working directory
ls -l /proc/41922/exe
# Output: /proc/41922/exe -> /tmp/.sysd/.xmrig (deleted)

ls -l /proc/41922/cwd
# Output: /proc/41922/cwd -> /dev/shm/.cache

# Step 3: Analyze outbound mining sockets
ss -tunap | grep -E "3333|14444|pool"
# tcp SYN_SENT 10.0.1.4:48122 -> 198.51.100.22:3333 (users:(("kworker",pid=41922,fd=3)))

3. Root Cause Analysis: CVE-2025-55182 Exploitation

Parsing Nginx reverse-proxy access logs and Node.js process logs revealed an unauthenticated POST request containing serialized JSON payloads aimed at an upstream React SSR handler.

This matched the signature for CVE-2025-55182 (dubbed React2Shell), where improper deserialization of component server actions permitted arbitrary command injection. The attacker injected a base64-encoded curl script that downloaded a payload into '/tmp', executed it, and immediately deleted the source file on disk to evade basic signature scanners.

4. Scrubbing Malware Persistence (Cron, Sockets, Immutable Flags)

Modern cryptominers drop multiple layers of stealth persistence so that if you terminate the process, it resurrects automatically within minutes. We methodically scrubbed every persistence hook:

1. Checked and cleaned root and www-data crontabs, '/etc/crontab', and '/etc/cron.d/'. Found a stealth entry triggering a base64 curl command every 5 minutes.

2. Inspected '/etc/systemd/system/' for unauthorized timer units.

3. Checked '/root/.ssh/authorized_keys' and verified no foreign public keys had been appended.

4. Removed immutable flags (chattr -i) on files under '/tmp' and completely emptied rogue directories.

# Remove immutable attributes if malware locked files
chattr -i -a /tmp/.sysd/* 2>/dev/null
rm -rf /tmp/.sysd /dev/shm/.cache

# Clean all cron directories
rm -f /etc/cron.d/sync-system-time
sed -i '/pastebin\|curl\|wget/d' /var/spool/cron/crontabs/* 2>/dev/null

# Terminate rogue processes permanently
pkill -9 -f "xmrig"
pkill -9 -f ".sysd"

# Flush DNS cache and block mining pool IPs at kernel level
sudo ufw deny out to 198.51.100.0/24 comment "Rogue Mining Pool Subnet"

5. Architectural Hardening & Container Resource Capping

Recovering root access was only half the battle. To ensure such an incident could never bring down both platforms again, we executed a structural re-architecture:

1. Container Decoupling: We isolated SubsDrop and Pro Trainer IT into independent Docker containers with strict 'cpu_quota' limits. No single service can ever consume more than 250% CPU or 4GB RAM.

2. Filesystem Mount Hardening: Mounted '/tmp' and '/dev/shm' with 'noexec,nosuid,nodev' options in '/etc/fstab'. Even if an RCE downloads a binary into '/tmp', Linux kernel refuses to execute it.

3. Private Database VPC Isolation: Verified MongoDB is bound strictly to internal private IP with zero 0.0.0.0 binding.

services:
  subsdrop-api:
    image: subsdrop/core:latest
    restart: unless-stopped
    deploy:
      resources:
        limits:
          cpus: "2.50"      # Hard ceiling: max 2.5 vCPUs
          memory: 3500M     # Hard ceiling: max 3.5GB RAM
        reservations:
          cpus: "0.50"
          memory: 1024M
    read_only: true         # Root filesystem is read-only
    tmpfs:
      - /tmp:noexec,nosuid,size=128m
    security_opt:
      - no-new-privileges:true
    networks:
      - internal_vpc

networks:
  internal_vpc:
    internal: true

6. Production Checklist & Post-Mortem Lessons

The host was fully restored to service in under 45 minutes with 0 bytes of customer data lost. Post-patch CPU immediately dropped back to a baseline of 14%.

Lessons cemented into our team's Standard Operating Procedure (SOP):

• Never co-locate multi-tenant apps on raw bare-metal or single PM2 instances without container resource boundaries.

• Always mount '/tmp' as 'noexec'. This blocks 90% of automated zero-day malware drops from running.

• Keep out-of-band VNC console access verified and ready before emergencies occur.