Deployment Topology¶

This document describes the container architecture, network configuration, GPU passthrough, and resource allocation for the Home Security Intelligence system.

Container Architecture¶

All services run as containers on a single Docker/Podman network (security-net), enabling service discovery by container name.

Deployment Mode Options¶

flowchart TB
    subgraph Host["Host Machine (GPU Server)"]
        subgraph Network["security-net (bridge)"]
            subgraph Core["Core Services"]
                FE["frontend<br/>:8080"]
                BE["backend<br/>:8000"]
            end

            subgraph Data["Data Layer"]
                PG["postgres<br/>:5432"]
                RD["redis<br/>:6379"]
            end

            subgraph AI["AI Services (GPU)"]
                DET["ai-yolo26<br/>YOLO26<br/>:8095"]
                LLM["ai-llm<br/>Nemotron<br/>:8091"]
                FLO["ai-florence<br/>Florence-2<br/>:8092"]
                CLIP["ai-clip<br/>CLIP ViT-L<br/>:8093"]
                ENR["ai-enrichment<br/>Multi-Model<br/>:8094"]
            end

            subgraph Mon["Monitoring"]
                PROM["prometheus<br/>:9090"]
                GRAF["grafana<br/>:3000"]
                JAEG["jaeger<br/>:16686"]
                ALERT["alertmanager<br/>:9093"]
                LOKI["loki<br/>:3100"]
                PYRO["pyroscope<br/>:4040"]
                ALLOY["alloy<br/>:12345"]
            end
        end

        GPU["NVIDIA GPU<br/>(via CDI)"]
    end

    FE <--> BE
    BE <--> PG
    BE <--> RD
    BE --> DET
    BE --> LLM
    BE --> FLO
    BE --> CLIP
    BE --> ENR
    DET --> GPU
    LLM --> GPU
    FLO --> GPU
    CLIP --> GPU
    ENR --> GPU
    PROM --> BE
    GRAF --> PROM

Network Configuration¶

Source: docker-compose.prod.yml:834-836

networks:
  security-net:
    driver: bridge

GPU Passthrough Configuration¶

All AI services use NVIDIA Container Toolkit (CDI) for GPU access.

Source: docker-compose.prod.yml:137-143 (ai-yolo26 example)

deploy:
  resources:
    reservations:
      devices:
        - driver: nvidia
          count: 1
          capabilities: [gpu]

GPU Requirements¶

• NVIDIA GPU: RTX A5500 (24GB) or similar with 24GB+ VRAM
• CUDA: 12.0+ support required
• Container Toolkit: nvidia-container-toolkit must be installed

Installation Commands¶

# Install NVIDIA Container Toolkit (Ubuntu/Debian)
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
    sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt update && sudo apt install -y nvidia-container-toolkit

# Configure Docker
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

VRAM Allocation¶

Total GPU VRAM: ~24GB (RTX A5500)

+------------------------------------------------------------------+
|                        GPU VRAM (24GB)                            |
+------------------------------------------------------------------+
| Nemotron LLM (Q4_K_M)                          | ~14,700 MB       |
| ================================================                  |
+------------------------------------------------------------------+
| YOLO26                                      | ~650 MB          |
| ======                                                            |
+------------------------------------------------------------------+
| Enrichment Service Budget                      | ~6,800 MB        |
| (on-demand: vehicle, pet, pose, threat, etc.)                     |
| ==================                                                |
+------------------------------------------------------------------+
| Florence-2 (optional)                          | ~1,500 MB        |
| =====                                                             |
+------------------------------------------------------------------+
| CLIP ViT-L (optional)                          | ~400 MB          |
| ==                                                                |
+------------------------------------------------------------------+
| System/Driver Overhead                         | ~500 MB          |
| ==                                                                |
+------------------------------------------------------------------+

VRAM by Service¶

Enrichment VRAM Budget Source: docker-compose.prod.yml:284

- VRAM_BUDGET_GB=6.8

Volume Mounts¶

Source: docker-compose.prod.yml:818-832

volumes:
  postgres_data:
    driver: local
  redis_data:
    driver: local
  prometheus_data:
    driver: local
  grafana_data:
    driver: local
  alertmanager_data:
    driver: local
  loki_data:
    driver: local
  pyroscope_data:
    driver: local

Health Checks¶

All services include Docker health checks for orchestration and monitoring.

Source: docker-compose.prod.yml (various services)

Start Period Notes¶

AI services have longer start periods due to model loading:

• ai-yolo26: 60s for YOLO26 model loading
• ai-llm: 120s for Nemotron LLM loading (largest model)
• ai-enrichment: 180s for multiple model initialization

Resource Limits¶

Source: docker-compose.prod.yml (various services)

Note: AI services (ai-yolo26, ai-llm, ai-florence, ai-clip, ai-enrichment) do not have CPU/memory limits to allow full GPU utilization.

Service Dependencies¶

Backend Initialization Sequence¶

Source: docker-compose.prod.yml:353-376

# Backend startup order
depends_on:
  postgres:
    condition: service_healthy
  redis:
    condition: service_healthy
  ai-yolo26:
    condition: service_healthy
  ai-llm:
    condition: service_healthy

flowchart LR
    PG["postgres"] --> BE["backend"]
    RD["redis"] --> BE
    DET["ai-yolo26"] --> BE
    LLM["ai-llm"] --> BE
    BE --> FE["frontend"]
    PROM["prometheus"] --> GRAF["grafana"]
    LOKI["loki"] --> ALLOY["alloy"]
    PYRO["pyroscope"] --> ALLOY

Deployment Commands¶

# Start all services
docker compose -f docker-compose.prod.yml up -d

# Or with Podman
podman-compose -f docker-compose.prod.yml up -d

# Check container status
docker compose -f docker-compose.prod.yml ps

# View logs
docker compose -f docker-compose.prod.yml logs -f backend

# Check GPU usage
nvidia-smi --query-compute-apps=pid,name,used_memory --format=csv

# Rebuild without cache
docker compose -f docker-compose.prod.yml build --no-cache backend

Related Documentation¶

• Configuration - Environment variables and settings
• Design Decisions - Why containerized deployment
• Architecture Overview - System design