Distributed Service Performance Monitor

A real-time performance monitoring server to test distributed systems, featuring live metrics visualization and robust data collection.

Features

Metrics Collection

• Real-time service performance monitoring
• Database operation timing
• Cache performance tracking
• Automatic data aggregation and processing

Storage & Processing

• Distributed SQLite storage (Turso)
• Redis caching layer
• Asynchronous processing queue
• Retry mechanisms with exponential backoff
• Connection pooling and transaction management

Dashboard

• Real-time metrics visualization
• Customizable time ranges (30m, 1h, 24h, 7d, custom)
• Performance statistics (avg, P50, P95, P99)
• Database and cache activity monitoring
• CSV export functionality
• Interactive time series charts

Architecture

The system uses a multi-layered architecture:

1. Frontend: React-based dashboard with Chart.js
2. Storage: Turso Database (distributed SQLite) + Redis cache
3. Processing: Async queue with multiple workers
4. Collection: Distributed metrics collection with retry logic

Technical Stack

• Frontend: React, Chart.js, Tailwind CSS
• Database: Turso (distributed SQLite)
• Cache: Redis
• Language: Go 1.23
• Deployment: Docker + Fly.io

Setup

1. Deploy using fly.io:

   fly launch
   fly deploy
   

Development

For local development:

1. Install dependencies:

   go mod download
   

2. Start the service:

   go run main.go
   

3. Access the dashboard at http://localhost:8080

Architecture Notes

• The system uses a queue-based architecture for processing metrics
• Implements automatic retries for failed operations
• Features connection pooling for database operations
• Supports distributed deployment through Fly.io
• Uses websockets for real-time metric updates

Performance Considerations

• Metrics are processed asynchronously to prevent blocking
• Connection pooling optimizes database access
• Redis caching reduces database load
• Configurable retry mechanisms ensure reliability
• Dashboard uses data bucketing for better visualization