VisionRoad-Frontend
This commit is contained in:
sumona-banerjeee
584
Readme.md
Normal file
584
Readme.md
Normal file
@@ -0,0 +1,584 @@
|
||||
# YOLOPOTHOLE - Pothole Detection System
|
||||
|
||||
Real-time pothole detection system using YOLO with adaptive ROI, ByteTrack tracking, and WebSocket-based progress monitoring.
|
||||
|
||||
---
|
||||
|
||||
## 📋 Table of Contents
|
||||
|
||||
- [Features](#features)
|
||||
- [Architecture](#architecture)
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Installation](#installation)
|
||||
- [Configuration](#configuration)
|
||||
- [Usage](#usage)
|
||||
- [API Reference](#api-reference)
|
||||
- [WebSocket Protocol](#websocket-protocol)
|
||||
- [Sample Data](#sample-data)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
|
||||
---
|
||||
|
||||
## ✨ Features
|
||||
|
||||
- **Adaptive ROI Detection**: Adjusts detection region based on vehicle speed
|
||||
- **ByteTrack Multi-Object Tracking**: Prevents duplicate pothole counting
|
||||
- **Real-time WebSocket Updates**: Live progress monitoring during processing
|
||||
- **Frame-by-Frame Analysis**: Detailed detection logs with bounding box coordinates
|
||||
- **Video Playback with Overlays**: Interactive video player with detection visualization
|
||||
- **RESTful API**: Complete CRUD operations for video processing
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
```
|
||||
YOLOPOTHOLE/
|
||||
├── app/
|
||||
│ ├── core/
|
||||
│ │ └── storage.py # In-memory storage
|
||||
│ ├── routes/
|
||||
│ │ └── upload_process_routes.py # API endpoints
|
||||
│ ├── services/
|
||||
│ │ ├── upload_service.py # File upload handling
|
||||
│ │ └── video_processor.py # Core detection logic
|
||||
│ └── ws/
|
||||
│ └── websocket_manager.py # WebSocket management
|
||||
├── frontend/
|
||||
│ ├── app/
|
||||
│ │ └── page.tsx # Main page
|
||||
│ └── components/
|
||||
│ ├── upload-section.tsx # Upload UI
|
||||
│ ├── summary-section.tsx # Results summary
|
||||
│ └── video-player-section.tsx # Video player
|
||||
├── models/
|
||||
│ └── pothole-detector.pt # YOLO model
|
||||
├── uploads/ # Uploaded videos
|
||||
├── results/ # JSON results
|
||||
└── main.py # FastAPI entry
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📦 Prerequisites
|
||||
|
||||
### Backend
|
||||
- Python 3.9+
|
||||
- CUDA-compatible GPU (optional, recommended)
|
||||
- FFmpeg
|
||||
|
||||
### Frontend
|
||||
- Node.js 18+
|
||||
- npm/yarn/pnpm
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Installation
|
||||
|
||||
### Backend Setup
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone <repository-url>
|
||||
cd YOLOPOTHOLE
|
||||
|
||||
# Create virtual environment
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # Windows: .venv\Scripts\activate
|
||||
|
||||
# Install dependencies
|
||||
pip install fastapi uvicorn python-multipart
|
||||
pip install opencv-python ultralytics
|
||||
pip install websockets
|
||||
|
||||
# Create required directories
|
||||
mkdir -p uploads results models
|
||||
```
|
||||
|
||||
### Frontend Setup
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
|
||||
# Install dependencies
|
||||
npm install
|
||||
|
||||
# Required packages
|
||||
npm install lucide-react
|
||||
npm install @radix-ui/react-progress
|
||||
npm install @radix-ui/react-scroll-area
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ Configuration
|
||||
|
||||
### Backend Configuration
|
||||
|
||||
**`app/core/storage.py`**
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
# Storage directories
|
||||
UPLOAD_DIR = Path("uploads")
|
||||
RESULTS_DIR = Path("results")
|
||||
MODELS_DIR = Path("models")
|
||||
|
||||
# In-memory storage
|
||||
processing_status = {}
|
||||
detection_results = {}
|
||||
|
||||
# Ensure directories exist
|
||||
UPLOAD_DIR.mkdir(exist_ok=True)
|
||||
RESULTS_DIR.mkdir(exist_ok=True)
|
||||
MODELS_DIR.mkdir(exist_ok=True)
|
||||
```
|
||||
|
||||
**`main.py`**
|
||||
```python
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from app.routes.upload_process_routes import router
|
||||
|
||||
app = FastAPI(title="YOLOPOTHOLE API", version="1.0.0")
|
||||
|
||||
# CORS configuration
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["http://localhost:3000"], # Frontend URL
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
# Include routes
|
||||
app.include_router(router, prefix="/api/v1", tags=["detection"])
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
uvicorn.run(app, host="0.0.0.0", port=8000)
|
||||
```
|
||||
|
||||
### Frontend Configuration
|
||||
|
||||
**`.env.local`**
|
||||
```bash
|
||||
NEXT_PUBLIC_API_URL=http://localhost:8000/api/v1
|
||||
NEXT_PUBLIC_WS_URL=ws://localhost:8000/api/v1
|
||||
```
|
||||
|
||||
### Model Configuration
|
||||
|
||||
**Adaptive Parameters** (in `video_processor.py`):
|
||||
```python
|
||||
# Speed < 30 km/h: ROI 50%, Confidence 0.35
|
||||
# Speed 30-60 km/h: ROI 65%, Confidence 0.28
|
||||
# Speed > 60 km/h: ROI 75%, Confidence 0.22
|
||||
|
||||
MIN_DETECTION_FRAMES = 3 # Frames needed to confirm pothole
|
||||
DETECTION_TIME_WINDOW = 1.0 # Time window in seconds
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📖 Usage
|
||||
|
||||
### Starting the Backend
|
||||
|
||||
```bash
|
||||
# Development
|
||||
python main.py
|
||||
|
||||
# Production
|
||||
uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4
|
||||
```
|
||||
|
||||
### Starting the Frontend
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
|
||||
# Development
|
||||
npm run dev
|
||||
|
||||
# Production
|
||||
npm run build
|
||||
npm start
|
||||
```
|
||||
|
||||
### Basic Workflow
|
||||
|
||||
1. **Upload Video**: Select video file and set vehicle speed
|
||||
2. **Monitor Progress**: Real-time WebSocket updates
|
||||
3. **View Results**: Summary statistics and video playback
|
||||
4. **Analyze Detections**: Frame-by-frame detection logs
|
||||
|
||||
---
|
||||
|
||||
## 🔌 API Reference
|
||||
|
||||
### REST Endpoints
|
||||
|
||||
#### 1. Upload Video
|
||||
```http
|
||||
POST /api/v1/upload
|
||||
Content-Type: multipart/form-data
|
||||
|
||||
Parameters:
|
||||
- file: Video file (mp4, avi, mov, mkv)
|
||||
- speed_kmh: Vehicle speed (integer, default: 30)
|
||||
|
||||
Response:
|
||||
{
|
||||
"video_id": "uuid-string",
|
||||
"filename": "video.mp4",
|
||||
"message": "Video uploaded successfully. Processing started.",
|
||||
"status": "queued"
|
||||
}
|
||||
```
|
||||
|
||||
#### 2. Get Processing Status
|
||||
```http
|
||||
GET /api/v1/status/{video_id}
|
||||
|
||||
Response:
|
||||
{
|
||||
"status": "processing",
|
||||
"progress": 45,
|
||||
"message": "Processing frame 450/1000"
|
||||
}
|
||||
```
|
||||
|
||||
#### 3. Get Detection Results
|
||||
```http
|
||||
GET /api/v1/results/{video_id}
|
||||
|
||||
Response: See "Sample Detection Results" below
|
||||
```
|
||||
|
||||
#### 4. List All Videos
|
||||
```http
|
||||
GET /api/v1/videos
|
||||
|
||||
Response:
|
||||
{
|
||||
"videos": [
|
||||
{
|
||||
"video_id": "uuid-string",
|
||||
"status": "completed",
|
||||
"progress": 100,
|
||||
"summary": { ... }
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Internal API Calls (Frontend)
|
||||
|
||||
**Upload Request**:
|
||||
```typescript
|
||||
const formData = new FormData()
|
||||
formData.append("file", file)
|
||||
formData.append("speed_kmh", "30")
|
||||
|
||||
const response = await fetch(`${API_URL}/upload`, {
|
||||
method: "POST",
|
||||
body: formData
|
||||
})
|
||||
|
||||
const result = await response.json()
|
||||
// Returns: { video_id, filename, message, status }
|
||||
```
|
||||
|
||||
**Results Request**:
|
||||
```typescript
|
||||
const response = await fetch(`${API_URL}/results/${videoId}`)
|
||||
const detectionData: DetectionData = await response.json()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 WebSocket Protocol
|
||||
|
||||
### Connection
|
||||
```javascript
|
||||
const ws = new WebSocket(`ws://localhost:8000/api/v1/ws/${videoId}`)
|
||||
```
|
||||
|
||||
### Message Types
|
||||
|
||||
#### 1. Status Update
|
||||
```json
|
||||
{
|
||||
"type": "status",
|
||||
"status": "processing",
|
||||
"progress": 0,
|
||||
"message": "Loading model..."
|
||||
}
|
||||
```
|
||||
|
||||
#### 2. Progress Update
|
||||
```json
|
||||
{
|
||||
"type": "progress",
|
||||
"progress": 45,
|
||||
"message": "Processing frame 450/1000",
|
||||
"unique_potholes": 12,
|
||||
"total_detections": 87
|
||||
}
|
||||
```
|
||||
|
||||
#### 3. Completion
|
||||
```json
|
||||
{
|
||||
"type": "complete",
|
||||
"status": "completed",
|
||||
"progress": 100,
|
||||
"message": "Processing completed successfully",
|
||||
"summary": {
|
||||
"unique_potholes": 25,
|
||||
"total_detections": 143,
|
||||
"total_frames": 1000,
|
||||
"detection_rate": 35.2
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 4. Error
|
||||
```json
|
||||
{
|
||||
"type": "error",
|
||||
"status": "error",
|
||||
"message": "Processing failed: Model not found"
|
||||
}
|
||||
```
|
||||
|
||||
#### 5. Heartbeat
|
||||
```json
|
||||
{
|
||||
"type": "heartbeat"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Sample Data
|
||||
|
||||
### Sample Detection Results
|
||||
|
||||
```json
|
||||
{
|
||||
"video_id": "550e8400-e29b-41d4-a716-446655440000",
|
||||
"video_path": "uploads/550e8400-e29b-41d4-a716-446655440000.mp4",
|
||||
"speed_kmh": 45,
|
||||
"processed_at": "2025-12-18T10:30:45.123456",
|
||||
"video_info": {
|
||||
"total_frames": 1200,
|
||||
"fps": 30.0,
|
||||
"duration": 40.0,
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"resolution": "1920x1080"
|
||||
},
|
||||
"summary": {
|
||||
"total_frames": 1200,
|
||||
"unique_potholes": 18,
|
||||
"total_detections": 245,
|
||||
"frames_with_detections": 147,
|
||||
"detection_rate": 12.25
|
||||
},
|
||||
"pothole_list": [
|
||||
{
|
||||
"pothole_id": 1,
|
||||
"first_detected_frame": 45,
|
||||
"first_detected_time": 1.5,
|
||||
"confidence": 0.876
|
||||
},
|
||||
{
|
||||
"pothole_id": 2,
|
||||
"first_detected_frame": 128,
|
||||
"first_detected_time": 4.27,
|
||||
"confidence": 0.923
|
||||
}
|
||||
],
|
||||
"frames": [
|
||||
{
|
||||
"frame_id": 45,
|
||||
"speed_kmh": 45,
|
||||
"roi_ratio": 0.65,
|
||||
"potholes": [
|
||||
{
|
||||
"frame_id": 45,
|
||||
"pothole_id": 1,
|
||||
"type": "pothole",
|
||||
"confidence": 0.876,
|
||||
"bbox": {
|
||||
"x1": 450,
|
||||
"y1": 720,
|
||||
"x2": 580,
|
||||
"y2": 820
|
||||
},
|
||||
"center": {
|
||||
"x": 515,
|
||||
"y": 770
|
||||
},
|
||||
"area": 13000
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Sample WebSocket Messages (Sequential)
|
||||
|
||||
```javascript
|
||||
// 1. Initial connection
|
||||
{ "type": "status", "status": "queued", "progress": 0, "message": "Video uploaded, starting processing..." }
|
||||
|
||||
// 2. Model loading
|
||||
{ "type": "status", "status": "processing", "progress": 0, "message": "Loading model..." }
|
||||
|
||||
// 3. Processing started
|
||||
{ "type": "status", "status": "processing", "progress": 5, "message": "Model loaded, processing video..." }
|
||||
|
||||
// 4. Progress updates (every 5%)
|
||||
{ "type": "progress", "progress": 10, "message": "Processing frame 120/1200", "unique_potholes": 3, "total_detections": 18 }
|
||||
{ "type": "progress", "progress": 25, "message": "Processing frame 300/1200", "unique_potholes": 7, "total_detections": 52 }
|
||||
{ "type": "progress", "progress": 50, "message": "Processing frame 600/1200", "unique_potholes": 12, "total_detections": 134 }
|
||||
|
||||
// 5. Completion
|
||||
{
|
||||
"type": "complete",
|
||||
"status": "completed",
|
||||
"progress": 100,
|
||||
"message": "Processing completed successfully",
|
||||
"summary": {
|
||||
"unique_potholes": 18,
|
||||
"total_detections": 245,
|
||||
"total_frames": 1200,
|
||||
"detection_rate": 12.25
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Sample Frontend State
|
||||
|
||||
```typescript
|
||||
type DetectionData = {
|
||||
video_id: string
|
||||
video_info: {
|
||||
fps: number // 30.0
|
||||
width: number // 1920
|
||||
height: number // 1080
|
||||
total_frames: number // 1200
|
||||
}
|
||||
summary: {
|
||||
unique_potholes: number // 18
|
||||
total_detections: number // 245
|
||||
total_frames: number // 1200
|
||||
detection_rate: number // 12.25
|
||||
}
|
||||
frames: Array<{
|
||||
frame_id: number
|
||||
potholes: Array<{
|
||||
pothole_id: number
|
||||
bbox: { x1: number; y1: number; x2: number; y2: number }
|
||||
confidence: number
|
||||
}>
|
||||
}>
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### Backend Issues
|
||||
|
||||
**Model Not Loading**
|
||||
```bash
|
||||
# Check model path
|
||||
ls models/pothole-detector.pt
|
||||
|
||||
# Test YOLO installation
|
||||
python -c "from ultralytics import YOLO; print('OK')"
|
||||
```
|
||||
|
||||
**CUDA/GPU Issues**
|
||||
```bash
|
||||
# Check CUDA availability
|
||||
python -c "import torch; print(torch.cuda.is_available())"
|
||||
|
||||
# Force CPU mode in video_processor.py
|
||||
self.pothole_model = YOLO("models/pothole-detector.pt", device='cpu')
|
||||
```
|
||||
|
||||
**WebSocket Connection Failed**
|
||||
- Ensure CORS is properly configured
|
||||
- Check firewall settings for port 8000
|
||||
- Verify WebSocket URL matches backend
|
||||
|
||||
### Frontend Issues
|
||||
|
||||
**Video Not Playing**
|
||||
```typescript
|
||||
// Check browser console for errors
|
||||
// Ensure video MIME type is supported
|
||||
// Verify video file is accessible via ObjectURL
|
||||
```
|
||||
|
||||
**Bounding Boxes Not Showing**
|
||||
```typescript
|
||||
// Check canvas dimensions match video
|
||||
// Verify detection data structure
|
||||
// Inspect frameDetectionMap in DevTools
|
||||
```
|
||||
|
||||
**Progress Not Updating**
|
||||
```typescript
|
||||
// Check WebSocket connection status
|
||||
// Verify video_id matches between upload and WS
|
||||
// Look for network errors in browser DevTools
|
||||
```
|
||||
|
||||
### Performance Optimization
|
||||
|
||||
**Slow Processing**
|
||||
- Use GPU acceleration (CUDA)
|
||||
- Reduce video resolution
|
||||
- Lower frame rate
|
||||
- Adjust confidence thresholds
|
||||
|
||||
**High Memory Usage**
|
||||
```python
|
||||
# Limit thread pool workers
|
||||
executor = ThreadPoolExecutor(max_workers=2)
|
||||
|
||||
# Reduce detection history
|
||||
pothole_tracker = defaultdict(lambda: deque(maxlen=10))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 Notes
|
||||
|
||||
- **Tracking**: Requires consistent object IDs from ByteTrack
|
||||
- **Frame Calculation**: Uses `Math.round(currentTime * fps)` for accuracy
|
||||
- **Logging**: Limited to last 50 entries to prevent memory issues
|
||||
- **Storage**: Results saved to both memory and JSON files
|
||||
- **Cleanup**: Implement periodic cleanup for old videos/results
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security Considerations
|
||||
|
||||
- Validate file types and sizes on upload
|
||||
- Sanitize video_id to prevent path traversal
|
||||
- Implement rate limiting for API endpoints
|
||||
- Add authentication for production deployments
|
||||
- Use HTTPS/WSS in production
|
||||
|
||||
|
||||
|
||||
**Built with FastAPI, YOLO, React, and shadcn/ui**
|
||||
Reference in New Issue
Block a user