EDR Monitor — Behavioral Malware Detection System

A university diploma project demonstrating real-time Endpoint Detection & Response (EDR) using behavioral analysis and machine learning on a Linux host.

---

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        Linux Host / VM                          │
│                                                                 │
│  ┌──────────────┐   HTTP POST /api/metrics   ┌──────────────┐  │
│  │              │ ─────────────────────────▶ │              │  │
│  │  agent/      │                            │  backend/    │  │
│  │  (psutil +   │                            │  (FastAPI +  │  │
│  │   watchdog + │                            │   SQLite +   │  │
│  │   IsoForest) │                            │   WebSocket) │  │
│  │              │                            │              │  │
│  └──────────────┘                            └──────┬───────┘  │
│        │                                           │           │
│  Filesystem watcher                        WS /ws  │           │
│  /etc /tmp /var/log                                │           │
│                                            ┌───────▼───────┐   │
│                                            │  frontend/    │   │
│                                            │  (React +     │   │
│                                            │   Recharts +  │   │
│                                            │   shadcn/ui)  │   │
│                                            │  :3000        │   │
│                                            └───────────────┘   │
└─────────────────────────────────────────────────────────────────┘

Components:

Directory	Tech	Purpose
agent/	Python 3.11, psutil, watchdog, scikit-learn	Collects process metrics, runs ML inference
backend/	FastAPI, aiosqlite, uvicorn	REST API, SQLite storage, WebSocket broadcast
frontend/	React 18, Vite, shadcn/ui, Recharts	Real-time SOC dashboard

---

Quick Start (local, no Docker)

1. Backend

cd backend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn main:app --reload --port 8000

2. Agent

cd agent
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Train the ML model (runs once, saves data/model.pkl)
python train.py

# Start the agent
python main.py

Note: The agent reads process metadata — run with sufficient privileges (or as root) to see all processes and network connections.

3. Frontend

cd frontend
npm install
npm run dev        # http://localhost:3000

---

Quick Start (Docker Compose)

docker compose up --build

Services:

• Backend → http://localhost:8000 (API docs: http://localhost:8000/docs)
• Dashboard → http://localhost:3000

---

ML Model

The agent uses Isolation Forest (scikit-learn) for unsupervised anomaly detection.

Feature	Description
cpu_percent	Process CPU utilisation
memory_mb	RSS memory in megabytes
open_files	Number of open file descriptors
connections	Number of open TCP connections

Training data priority:

1. data/UNSW_NB15_training-set.csv — real network intrusion dataset (download separately)
2. Synthetic normal-behaviour data (fallback, auto-generated if CSV absent)

Contamination factor: 5% (5 % of training data treated as outliers).

---

REST API

Method	Endpoint	Description
GET	/api/stats	Dashboard statistics
GET	/api/metrics?limit=200	Recent process metrics
GET	/api/alerts?limit=50	Security alerts
GET	/api/file-events?limit=100	Filesystem events
POST	/api/metrics	Ingest batch from agent
WS	/ws	Real-time alert stream

Interactive docs: http://localhost:8000/docs

---

Simulating Anomalies (demo)

CPU spike (Linux)

# Requires stress-ng
stress-ng --cpu 4 --timeout 30s

# Alternative — pure Python
python3 -c "
import multiprocessing, time
def spin(): 
    while True: pass
procs = [multiprocessing.Process(target=spin) for _ in range(4)]
[p.start() for p in procs]
time.sleep(30)
[p.terminate() for p in procs]
"

Memory spike

python3 -c "
x = bytearray(2 * 1024 * 1024 * 1024)  # allocate 2 GB
import time; time.sleep(30)
"

Filesystem trigger (write to /tmp)

for i in $(seq 1 20); do echo "test" > /tmp/edr_test_$i.txt; sleep 1; done

Within ~10 seconds the dashboard will show the anomalous process highlighted in red and a toast notification will appear bottom-right.

---

Screenshots

Add screenshots here after first run

Overview	Processes	Alerts

---

Dataset

The system optionally uses the UNSW-NB15 network intrusion dataset for training.

Download: https://research.unsw.edu.au/projects/unsw-nb15-dataset

Place the file at: data/UNSW_NB15_training-set.csv

Then retrain:

cd agent && python train.py

---

Project Structure

dipl-edr/
├── agent/
│   ├── main.py          # monitoring loop + fs watcher
│   ├── model.py         # IsolationForest wrapper
│   ├── train.py         # standalone training script
│   └── requirements.txt
├── backend/
│   ├── main.py          # FastAPI app + WebSocket
│   └── requirements.txt
├── frontend/
│   ├── src/
│   │   ├── App.tsx              # root layout + WebSocket client
│   │   ├── components/
│   │   │   ├── Overview.tsx     # stats cards + chart
│   │   │   ├── Processes.tsx    # process table
│   │   │   ├── Alerts.tsx       # alerts table
│   │   │   ├── FileEvents.tsx   # fs events table
│   │   │   ├── SystemChart.tsx  # Recharts line chart
│   │   │   └── ui/              # shadcn-style primitives
│   │   ├── hooks/use-toast.ts
│   │   └── lib/
│   │       ├── api.ts           # typed fetch helpers
│   │       └── utils.ts
│   ├── package.json
│   └── vite.config.ts
├── data/                # .gitignored — holds model.pkl + edr.db
├── docker-compose.yml
└── README.md