CI/CD Pipeline Documentation¶

This document describes the GitHub Actions CI/CD pipeline for Home Security Intelligence.

Overview¶

The CI/CD pipeline consists of multiple workflows that run on different triggers:

Main CI Workflow¶

File: .github/workflows/ci.yml

The main CI workflow is the most comprehensive, running on every push to main and all pull requests.

Workflow Triggers¶

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  workflow_dispatch: # Manual trigger

Environment Configuration¶

env:
  PYTHON_VERSION: '3.14'
  NODE_VERSION: '24'
  UV_VERSION: '0.9.18'

Change Detection¶

The workflow uses intelligent change detection to only run relevant tests:

Detected file patterns:

• Backend: backend/**, pyproject.toml, uv.lock, ai/**, scripts/**/*.py
• Frontend: frontend/** (excluding .md files)
• Docs only: docs/**, **/*.md

Backend Jobs¶

1. Build Backend Dependencies¶

• Purpose: Shared dependency installation for all backend jobs
• Caching: Uses uv with cache suffix for optimization

2. Backend Lint (Ruff)¶

• Purpose: Code style and formatting checks
• Tools:
• ruff check - Linting
• ruff format --check - Format verification
• radon cc/mi - Complexity analysis

3. Backend Type Check (Mypy)¶

• Purpose: Static type checking
• Command: uv run mypy backend/ --ignore-missing-imports

4. Backend Unit Tests (4 Shards)¶

• Purpose: Fast, isolated unit tests
• Parallelization: 4 shards with pytest-split, xdist parallelization within shards
• Coverage: Collected per-shard, merged in unit-tests-coverage-merge
• Services: PostgreSQL 16
• Threshold: absolute floor 84, enforced in the merge job only when every unit shard passed (measured value, R-1). Shards run with --cov-fail-under=0; the executed absolute backend floor is 80% combined (validate.sh / nightly). pyproject.toml fail_under = 85 is the PR diff gate's RELATIVE baseline, not an absolute floor (owner ruling A7.1).

5. Integration Tests (4 Parallel Jobs)¶

Each integration test job:

• Uses PostgreSQL and Redis services
• Runs with retry logic (3 attempts)
• Collects coverage separately
• Has 20-minute timeout

6. Contract Tests (Main Branch Only)¶

• Purpose: Validate API schema consistency
• Trigger: Only on main branch pushes
• Action on Failure: Creates Linear issue automatically

7. Dead Code Detection (Main Branch Only)¶

• Purpose: Identify unused code
• Tool: Vulture
• Action on Failure: Creates Linear issue automatically

Frontend Jobs¶

1. API Types Contract Check¶

• Purpose: Ensure frontend types match backend API
• Validates:
• OpenAPI spec is current
• WebSocket types are current
• Generated types match API

2. Frontend Lint (ESLint)¶

• Purpose: Code style and quality
• Additional: Knip dead code detection (non-blocking)

3. Frontend Type Check (TypeScript)¶

• Purpose: Static type checking
• Command: npm run typecheck

4. Frontend Tests (Vitest - 16 Shards)¶

• Purpose: Component and hook unit tests
• Parallelization: 16 shards for memory management
• Memory: 6GB heap per shard
• Special Handling: Early process termination to prevent OOM during cleanup

5. E2E Tests (Playwright - 6 Shards)¶

• Purpose: End-to-end browser testing
• Browser: Chromium (primary, required for merge)
• Parallelization: 6 shards
• Retry: 3 attempts per shard

Branch Protection¶

Summary jobs aggregate shard results for branch protection rules:

Deploy Workflow¶

File: .github/workflows/deploy.yml

Triggered on push to main branch. Builds and publishes container images.

Build Stage¶

Builds images for multiple architectures:

ai-yolo26 (./ai/yolo26/Dockerfile) was in this matrix until 2026-09-23, when the standalone GPU image was retired fully (owner ruling — Triton on ai-gateway serves yolo26 among the 14 models); the recipe lives in archive/ai-yolo26-image/.

Platforms:

• linux/amd64 (ubuntu-latest runner)
• linux/arm64 (ubuntu-24.04-arm runner)

Multi-Arch Manifest Merge¶

After architecture-specific builds complete, creates multi-arch manifests for:

• ghcr.io/{repo}/backend:latest
• ghcr.io/{repo}/frontend:latest

Smoke Tests¶

Validates deployed images:

1. Pulls latest images from GHCR
2. Starts services with docker-compose.ci.yml
3. Waits for health checks (backend: 120s, frontend: 60s)
4. Runs smoke test script
5. Validates health endpoints

Supply Chain Security¶

SBOM Generation¶

• Tool: Anchore SBOM Action
• Format: SPDX JSON
• Retention: 90 days

Image Signing¶

• Tool: Sigstore Cosign
• Method: OIDC keyless signing
• Signs: Both latest and sha tags

SLSA Provenance¶

• Level: SLSA Level 3
• Tool: GitHub attestation API
• Purpose: Cryptographic build provenance

Deployment Stages¶

build -> merge -> smoke-test -> sbom-and-sign -> staging-deployment
                      |
                      v
            post-deployment-validation
                      |
                      v
              slsa-provenance

Release Workflow¶

File: .github/workflows/release.yml

Triggered on tag pushes matching v* pattern.

Changelog Generation¶

Automatically categorizes commits:

Linear Integration¶

Extracts and links Linear issue references (NEM-XXX pattern) in release notes.

Pre-release Detection¶

Automatically marks as pre-release if tag contains:

• -alpha
• -beta
• -rc

SAST Workflow¶

File: .github/workflows/sast.yml

Static Application Security Testing.

Bandit (Python Security)¶

bandit -r backend/ \
  -c .bandit.yml \
  -b .bandit.baseline \
  -x "backend/tests/*" \
  --severity-level medium

• Fails on HIGH/CRITICAL issues
• Uses baseline file for known false positives

Semgrep¶

Scans with rulesets:

• p/python
• p/typescript
• p/react
• p/owasp-top-ten
• Custom semgrep.yml

Test Coverage Gate¶

File: .github/workflows/test-coverage-gate.yml

Enforces test coverage requirements on PRs.

Coverage Requirements¶

The gate is a diff gate, not a set of per-layer thresholds: test-coverage-gate.yml runs scripts/check-test-coverage-gate.py --base-branch … --strict, which compares the PR's coverage against coverage-baseline.json published by main (pyproject.toml fail_under = 85 is that gate's RELATIVE baseline, not an absolute floor — owner ruling A7.1). The layer labels below are the expected test types the gate's PR-failure comment template prints; no per-layer percentage is evaluated:

The enforced floors — combined 80% via validate.sh/nightly, unit tier 84, integration tier 37 in the CI merge steps — are in Coverage Requirements.

Checks¶

1. Test Coverage Gate: Verifies new files have corresponding tests
2. Integration Test Check: Ensures API/service changes have integration tests
3. API Test Generation: Validates OpenAPI spec for test generation

Other Workflows¶

Security Scanning¶

Quality & Testing¶

Documentation & Maintenance¶

Integrations¶

Scheduled¶

Running Locally¶

Simulate CI Checks¶

# Full validation (recommended before PRs)
./scripts/validate.sh

# Backend checks
uv run ruff check backend/
uv run ruff format --check backend/
uv run mypy backend/
uv run pytest backend/tests/unit/ -n auto

# Frontend checks
cd frontend
npm run lint
npm run typecheck
npm test

Simulate Deploy Build¶

# Build images locally
podman-compose -f docker-compose.prod.yml build --no-cache backend
podman-compose -f docker-compose.prod.yml build --no-cache frontend

# Run smoke tests (frontend host ports: FRONTEND_HTTP_PORT=8080,
# FRONTEND_HTTPS_PORT=8444 per .env.example)
./scripts/ci-smoke-test.sh \
  --backend-url http://localhost:8000 \
  --frontend-url http://localhost:8080

Troubleshooting¶

Common CI Failures¶

Viewing Artifacts¶

Test results and coverage reports are uploaded as artifacts:

1. Go to Actions tab
2. Select workflow run
3. Scroll to "Artifacts" section
4. Download relevant artifacts

Re-running Failed Jobs¶

1. Go to the failed workflow run
2. Click "Re-run jobs" dropdown
3. Select "Re-run failed jobs" to save time

Related Documentation¶

• Testing Guide - Test infrastructure and patterns
• Testing Workflow - TDD practices
• Git Workflow - Branch and commit conventions
• Code Quality - Linting and formatting tools