Storage and Retention¶

Manage disk space and data retention. The cleanup service automatically prunes old records and files based on configurable retention periods.

Overview¶

The system stores several types of data that grow over time:

• Events - Security event records with risk scores
• Detections - Individual object detections with metadata
• GPU Stats - Historical GPU metrics for monitoring
• Logs - Application log entries in the database
• Thumbnails - Generated preview images for detections
• Original Images - Camera uploads (optionally cleaned)

The CleanupService runs daily to enforce retention policies and reclaim disk space.

Retention Configuration¶

Database Records¶

Source: backend/core/config.py

Cleanup Schedule¶

The cleanup service runs daily at a configurable time (default: 03:00). Configure in the service initialization:

# backend/services/cleanup_service.py
cleanup_service = CleanupService(
    cleanup_time="03:00",        # HH:MM (24-hour format)
    retention_days=30,           # Override config default
    thumbnail_dir="data/thumbnails",
    delete_images=False          # Keep original camera images
)

What Gets Cleaned¶

Automatic Cleanup¶

Optional Cleanup¶

Warning: Original camera images are not deleted by default. If you need automated deletion of originals, it currently requires changing how CleanupService is initialized (advanced/operator customization).

Cleanup Statistics¶

Each cleanup operation returns detailed statistics:

class CleanupStats:
    events_deleted: int = 0
    detections_deleted: int = 0
    gpu_stats_deleted: int = 0
    logs_deleted: int = 0
    thumbnails_deleted: int = 0
    images_deleted: int = 0        # Only if delete_images=True
    space_reclaimed: int = 0       # Estimated bytes freed

Statistics are logged after each cleanup:

INFO: Cleanup completed: <CleanupStats(events=45, detections=328, gpu_stats=8640, logs=1024, files=328, space=52428800 bytes)>

Storage Dashboard¶

The frontend StorageDashboard component provides real-time visibility into storage usage.

Features¶

• Disk Usage Overview - Progress bar showing total disk usage percentage
• Storage Breakdown - Size and file count for thumbnails, images, and clips
• Database Records - Count of events, detections, GPU stats, and logs
• Cleanup Preview - Dry run showing what would be deleted

Accessing the Dashboard¶

Navigate to Settings in the web interface to view the Storage Dashboard.

Dashboard Metrics¶

Cleanup Preview (Dry Run)¶

Before running actual cleanup, preview what would be deleted:

Via API¶

# Dry run cleanup preview
# Note: this endpoint requires an API key only when `API_KEY_ENABLED=true`.
curl -X POST "http://localhost:8000/api/system/cleanup?dry_run=true"

Response:

{
  "events_deleted": 45,
  "detections_deleted": 328,
  "gpu_stats_deleted": 8640,
  "logs_deleted": 1024,
  "thumbnails_deleted": 328,
  "images_deleted": 0,
  "space_reclaimed": 52428800,
  "retention_days": 30,
  "dry_run": true,
  "timestamp": "2025-12-27T10:30:00Z"
}

Via Dashboard¶

Click Preview Cleanup in the Storage Dashboard to see what would be deleted without actually deleting anything.

Manual Cleanup¶

Trigger Immediate Cleanup¶

# Via API (requires API key only when `API_KEY_ENABLED=true`)
curl -X POST "http://localhost:8000/api/system/cleanup"

Cleanup Job Status¶

To view the latest cleanup status (and whether a cleanup is currently running):

curl http://localhost:8000/api/system/cleanup/status

Storage Directories¶

Disk Space Estimation¶

Total estimate: ~2 GB for 30 days with moderate activity.

Monitoring Storage¶

API Endpoints¶

# Storage statistics
curl http://localhost:8000/api/system/storage

# Database record counts
curl http://localhost:8000/api/system/stats

Alerts¶

Configure disk space alerts by monitoring disk_usage_percent:

Best Practices¶

Sizing Recommendations¶

Optimization Tips¶

1. Reduce thumbnail size - Thumbnails are the largest storage consumer after images
2. Increase polling interval - Reduce GPU stat records with higher GPU_POLL_INTERVAL_SECONDS
3. Enable image cleanup - If you have backups, enable delete_images=True
4. Shorter retention - Reduce RETENTION_DAYS if disk space is limited
5. Compress logs - Log files are already rotated; consider offsite backup

Troubleshooting¶

Cleanup Not Running¶

1. Check cleanup service is started:

curl http://localhost:8000/api/system/health

1. Check logs for cleanup activity:

grep "Cleanup" data/logs/security.log

1. Verify cleanup time configuration

Disk Space Not Decreasing¶

1. Check if images are being deleted (delete_images setting)
2. Verify thumbnail paths exist and are accessible
3. Check for orphaned files not tracked in database

Cleanup Errors¶

Check logs for transaction failures:

grep "Cleanup failed" data/logs/security.log

See Also¶

• Environment Variable Reference - All configuration options
• Monitoring - Health checks and diagnostics
• Troubleshooting - Common issues
• Backup - Backup strategies