Enrichment Light (/enrich-lt) — inside ai-gateway¶

[!IMPORTANT] > ai-enrichment-light is no longer a standalone container. The lightweight models (pose, threat, person ReID, pet, depth) run as Triton models inside the single ai-gateway container and are served by the /enrich-lt router on the gateway's port (${AI_GATEWAY_PORT:-8090}). The old service and its port 8096 no longer exist; the ai/enrichment-light/ directory survives only as the legacy server image that CI still builds. The heavy counterparts live on the /enrichment router in the same container.

Overview¶

Models Hosted¶

Triton model names in parentheses — this is what health checks and metrics use:

These run inside the gateway's shared VRAM budget (~4GB base, ~6GB peak) — there is no separate 4GB-GPU budget any more. Triton runs with --model-control-mode=none (ai/gateway/entrypoint.sh), so every model in the repository is loaded at startup.

API Endpoints¶

All paths are prefixed with /enrich-lt on the gateway: POST http://<gateway>:8090/enrich-lt/<endpoint>.

The image-carrying endpoints (/pose-analyze, /threat-detect, /person-reid, /pet-classify) take a JSON body with a base64 image plus a bbox (crop region) — {"image_base64": "...", "bbox": {"x": 0, "y": 0, "width": 100, "height": 200}} (a [x1, y1, x2, y2] list also works). Only /depth-estimate takes the full image ({"image_base64": "..."}).

Health Check¶

curl -s http://localhost:8090/enrich-lt/health | jq

Response (per-model Triton readiness):

{
  "status": "healthy",
  "models": {
    "pose": true,
    "threat": true,
    "reid": true,
    "pet": true,
    "depth": true
  }
}

status is "degraded" when any model is not ready.

Pose Analysis¶

POST /enrich-lt/pose-analyze
Content-Type: application/json

{ "image_base64": "<base64-crop>", "bbox": { "x": 100, "y": 80, "width": 120, "height": 300 } }

Returns keypoints, a derived posture (e.g. standing, crouching), alerts, and inference_time_ms.

Threat Detection¶

POST /enrich-lt/threat-detect

Returns threats_detected[] (class, confidence, bbox), is_threat, max_confidence.

Person Re-identification¶

POST /enrich-lt/person-reid

Returns the embedding vector and inference_time_ms.

Pet Classification¶

POST /enrich-lt/pet-classify

Returns pet_type, confidence, and household-pet matching.

Depth Estimation¶

POST /enrich-lt/depth-estimate
Content-Type: application/json

{ "image_base64": "<base64-full-image>" }

Returns normalized depth statistics (min/max/mean) for the frame.

Prometheus Metrics¶

curl -s http://localhost:8090/metrics      # gateway + merged Triton metrics
curl -s http://localhost:8002/metrics      # Triton native (per-model)

Gateway-level metrics emitted per call: hsi_ai_inference_duration_seconds{service, endpoint} (histogram) and hsi_ai_inference_errors_total{service, endpoint} (counter); Triton adds nv_inference_* per model. Prometheus scrapes both via the triton-metrics job. The retired enrichment_light_* metrics no longer exist.

Configuration¶

Tier Routing (which service handles which model)¶

# .env — route a model to the light or heavy router
ENRICHMENT_POSE_SERVICE=light      # Default: light
ENRICHMENT_THREAT_SERVICE=light    # Default: light
ENRICHMENT_REID_SERVICE=light      # Default: light
ENRICHMENT_PET_SERVICE=light       # Default: light
ENRICHMENT_DEPTH_SERVICE=light     # Default: light

The backend's enrichment_client.py resolves each model's URL from these values against ENRICHMENT_LIGHT_URL (http://ai-gateway:8090/enrich-lt in compose) or ENRICHMENT_URL (…/enrichment).

Preload¶

# Models loaded eagerly at gateway startup instead of on demand (comma-separated)
ENRICHMENT_LIGHT_PRELOAD_MODELS=pose_estimator,threat_detector,person_reid,pet_classifier,depth_estimator

Quantization / device overrides¶

Per-model quantization toggles (VEHICLE_QUANTIZED, DEMOGRAPHICS_QUANTIZED) and the device_env_var entries in models.yml (e.g. POSE_DEVICE) are applied by ai/gateway/entrypoint.sh and patch_triton_configs.py at container start — see GPU Setup.

Backend Integration¶

# backend/services/enrichment_client.py (simplified)
light_url = settings.enrichment_light_url          # ENRICHMENT_LIGHT_URL
if settings.enrichment_pose_service == "light":
    await client.post(f"{light_url}/pose-analyze", json={...})
else:
    await client.post(f"{settings.enrichment_url}/pose-analyze", json={...})

Health Checks (compose)¶

The router has no separate healthcheck — container health is the gateway's aggregate check:

# ai-gateway (docker-compose.prod.yml)
healthcheck:
  test: ['CMD', 'curl', '-f', 'http://localhost:8090/health']
  start_period: 180s # Triton initialises 14 models (matches the compose comment, corrected from 13 in the gateway-consolidation follow-up)

/health reports healthy only when Triton's server is ready and every model is ready.

Light vs Heavy Enrichment (both on :8090)¶

Troubleshooting¶

# Which models are ready?
curl -s http://localhost:8090/enrich-lt/health | jq

# Triton's view of one model
podman exec ai-gateway curl -s http://localhost:8002/v2/models/pose | jq '.state,.ready'

# Model-load failures show up first in the gateway log
podman logs ai-gateway 2>&1 | grep -iE "pose|threat|reid|pet|depth|error"

# Weights present on the host?
ls /export/ai_models/model-zoo/   # yolov8n-pose/, osnet-ain-x1-0/, threat-detection*/, pet-classifier/, depth-anything-v2-tiny/

If a model reports degraded, run ./ai/download_models.sh (manifest models.yml) and restart the gateway: podman compose -f docker-compose.prod.yml restart ai-gateway.

Related Documentation¶

• AI Orchestration Overview
• Model Zoo
• GPU Memory Limits
• Enrichment Pipeline