# EDR Monitor — Behavioral Malware Detection System

A university diploma project demonstrating real-time **Endpoint Detection & Response (EDR)**
using behavioral analysis and machine learning on a Linux host.

---

## Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│                        Linux Host / VM                          │
│                                                                 │
│  ┌──────────────┐   HTTP POST /api/metrics   ┌──────────────┐  │
│  │              │ ─────────────────────────▶ │              │  │
│  │  agent/      │                            │  backend/    │  │
│  │  (psutil +   │                            │  (FastAPI +  │  │
│  │   watchdog + │                            │   SQLite +   │  │
│  │   IsoForest) │                            │   WebSocket) │  │
│  │              │                            │              │  │
│  └──────────────┘                            └──────┬───────┘  │
│        │                                           │           │
│  Filesystem watcher                        WS /ws  │           │
│  /etc /tmp /var/log                                │           │
│                                            ┌───────▼───────┐   │
│                                            │  frontend/    │   │
│                                            │  (React +     │   │
│                                            │   Recharts +  │   │
│                                            │   shadcn/ui)  │   │
│                                            │  :3000        │   │
│                                            └───────────────┘   │
└─────────────────────────────────────────────────────────────────┘
```

**Components:**

| Directory | Tech | Purpose |
|-----------|------|---------|
| `agent/`   | Python 3.11, psutil, watchdog, scikit-learn | Collects process metrics, runs ML inference |
| `backend/` | FastAPI, aiosqlite, uvicorn | REST API, SQLite storage, WebSocket broadcast |
| `frontend/`| React 18, Vite, shadcn/ui, Recharts | Real-time SOC dashboard |

---

## Quick Start (local, no Docker)

### 1. Backend

```bash
cd backend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn main:app --reload --port 8000
```

### 2. Agent

```bash
cd agent
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Train the ML model (runs once, saves data/model.pkl)
python train.py

# Start the agent
python main.py
```

> **Note:** The agent reads process metadata — run with sufficient privileges
> (or as root) to see all processes and network connections.

### 3. Frontend

```bash
cd frontend
npm install
npm run dev        # http://localhost:3000
```

---

## Quick Start (Docker Compose)

```bash
docker compose up --build
```

Services:
- Backend → http://localhost:8000  (API docs: http://localhost:8000/docs)
- Dashboard → http://localhost:3000

---

## ML Model

The agent uses **Isolation Forest** (scikit-learn) for unsupervised anomaly detection.

| Feature | Description |
|---------|-------------|
| `cpu_percent` | Process CPU utilisation |
| `memory_mb`   | RSS memory in megabytes |
| `open_files`  | Number of open file descriptors |
| `connections` | Number of open TCP connections |

Training data priority:
1. `data/UNSW_NB15_training-set.csv` — real network intrusion dataset (download separately)
2. Synthetic normal-behaviour data (fallback, auto-generated if CSV absent)

Contamination factor: **5%** (5 % of training data treated as outliers).

---

## REST API

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET`  | `/api/stats` | Dashboard statistics |
| `GET`  | `/api/metrics?limit=200` | Recent process metrics |
| `GET`  | `/api/alerts?limit=50` | Security alerts |
| `GET`  | `/api/file-events?limit=100` | Filesystem events |
| `POST` | `/api/metrics` | Ingest batch from agent |
| `WS`   | `/ws` | Real-time alert stream |

Interactive docs: http://localhost:8000/docs

---

## Simulating Anomalies (demo)

### CPU spike (Linux)
```bash
# Requires stress-ng
stress-ng --cpu 4 --timeout 30s

# Alternative — pure Python
python3 -c "
import multiprocessing, time
def spin(): 
    while True: pass
procs = [multiprocessing.Process(target=spin) for _ in range(4)]
[p.start() for p in procs]
time.sleep(30)
[p.terminate() for p in procs]
"
```

### Memory spike
```bash
python3 -c "
x = bytearray(2 * 1024 * 1024 * 1024)  # allocate 2 GB
import time; time.sleep(30)
"
```

### Filesystem trigger (write to /tmp)
```bash
for i in $(seq 1 20); do echo "test" > /tmp/edr_test_$i.txt; sleep 1; done
```

Within ~10 seconds the dashboard will show the anomalous process highlighted in red
and a toast notification will appear bottom-right.

---

## Screenshots

> *Add screenshots here after first run*

| Overview | Processes | Alerts |
|----------|-----------|--------|
| ![overview](docs/overview.png) | ![processes](docs/processes.png) | ![alerts](docs/alerts.png) |

---

## Dataset

The system optionally uses the **UNSW-NB15** network intrusion dataset for training.

Download: https://research.unsw.edu.au/projects/unsw-nb15-dataset

Place the file at: `data/UNSW_NB15_training-set.csv`

Then retrain:
```bash
cd agent && python train.py
```

---

## Project Structure

```
dipl-edr/
├── agent/
│   ├── main.py          # monitoring loop + fs watcher
│   ├── model.py         # IsolationForest wrapper
│   ├── train.py         # standalone training script
│   └── requirements.txt
├── backend/
│   ├── main.py          # FastAPI app + WebSocket
│   └── requirements.txt
├── frontend/
│   ├── src/
│   │   ├── App.tsx              # root layout + WebSocket client
│   │   ├── components/
│   │   │   ├── Overview.tsx     # stats cards + chart
│   │   │   ├── Processes.tsx    # process table
│   │   │   ├── Alerts.tsx       # alerts table
│   │   │   ├── FileEvents.tsx   # fs events table
│   │   │   ├── SystemChart.tsx  # Recharts line chart
│   │   │   └── ui/              # shadcn-style primitives
│   │   ├── hooks/use-toast.ts
│   │   └── lib/
│   │       ├── api.ts           # typed fetch helpers
│   │       └── utils.ts
│   ├── package.json
│   └── vite.config.ts
├── data/                # .gitignored — holds model.pkl + edr.db
├── docker-compose.yml
└── README.md
```