Middleware Hub¶

Request processing pipeline and cross-cutting concerns for the Home Security Intelligence API

Overview¶

The middleware stack in this application processes all incoming HTTP requests before they reach route handlers, and all outgoing responses before they're sent to clients. This hub documents each middleware component, their execution order, and how they work together to provide logging, error handling, security, and request validation.

Middleware components in FastAPI/Starlette are executed in a specific order determined by their registration sequence in backend/main.py. The order matters because each middleware can modify the request before passing it to the next middleware, and modify the response on the way back. Understanding this execution order is critical for debugging and extending the middleware stack.

The middleware architecture follows defense-in-depth principles, with multiple layers of validation, security headers, and error handling. Each middleware is designed to be independent and testable, with clear responsibilities and minimal coupling to other components.

Documents¶

Architecture Diagram¶

graph TD
    subgraph "Incoming Request"
        REQ[HTTP Request] --> AUTH
    end

    subgraph "Middleware Stack (Execution Order)"
        AUTH[AuthMiddleware<br/>backend/api/middleware/auth.py] --> CT
        CT[ContentTypeValidationMiddleware<br/>backend/api/middleware/content_type_validator.py] --> RID
        RID[RequestIDMiddleware<br/>backend/api/middleware/request_id.py] --> BAG
        BAG[BaggageMiddleware<br/>backend/api/middleware/baggage.py] --> RT
        RT[RequestTimingMiddleware<br/>backend/api/middleware/request_timing.py] --> RL
        RL[RequestLoggingMiddleware<br/>backend/api/middleware/request_logging.py] --> RR
        RR[RequestRecorderMiddleware<br/>backend/api/middleware/request_recorder.py] --> DEP
        DEP[DeprecationMiddleware<br/>backend/api/middleware/deprecation.py] --> DEPL
        DEPL[DeprecationLoggerMiddleware<br/>backend/api/middleware/deprecation_logger.py] --> CORS
        CORS[CORSMiddleware<br/>FastAPI builtin] --> SEC
        SEC[SecurityHeadersMiddleware<br/>backend/api/middleware/security_headers.py] --> BL
        BL[BodySizeLimitMiddleware<br/>backend/api/middleware/body_limit.py] --> GZIP
        GZIP[GZipMiddleware<br/>FastAPI builtin] --> IDEM
        IDEM[IdempotencyMiddleware<br/>backend/api/middleware/idempotency.py] --> ROUTE
    end

    subgraph "Route Processing"
        ROUTE[Route Handler] --> EH
        EH[Exception Handlers<br/>backend/api/exception_handlers.py]
    end

    subgraph "Response"
        EH --> RESP[HTTP Response]
    end

    style AUTH fill:#f9f,stroke:#333
    style CORS fill:#bbf,stroke:#333
    style SEC fill:#bfb,stroke:#333
    style EH fill:#fbb,stroke:#333

Quick Reference¶

Key Concepts¶

Middleware Execution Order¶

Middleware in Starlette/FastAPI executes in reverse registration order for requests and registration order for responses. The middleware registered first wraps all subsequent middleware:

Request:  Client -> Last Registered -> ... -> First Registered -> Route
Response: Route -> First Registered -> ... -> Last Registered -> Client

This means AuthMiddleware (registered first in backend/main.py:1084) processes requests first and responses last.

Exception Handlers vs Middleware¶

Exception handlers (backend/api/exception_handlers.py) are distinct from middleware. They catch exceptions raised during request processing and convert them to HTTP responses. They're registered separately via app.add_exception_handler() and process exceptions based on type hierarchy.

Context Variables¶

Several middleware components use Python context variables to share state across the request lifecycle:

• request_id - Request correlation ID
• correlation_id - Distributed tracing ID
• trace_id / span_id - OpenTelemetry context

Configuration¶

Middleware Registration¶

Middleware is registered in backend/main.py:1083-1162:

# From backend/main.py:1083-1162
# Add authentication middleware (if enabled in settings)
app.add_middleware(AuthMiddleware)

# Add Content-Type validation middleware for request body validation (NEM-1617)
app.add_middleware(ContentTypeValidationMiddleware)

# Add request ID middleware for log correlation
app.add_middleware(RequestIDMiddleware)

# Add OpenTelemetry Baggage middleware for cross-service context propagation (NEM-3796)
app.add_middleware(BaggageMiddleware)

# Add request timing middleware for API latency tracking (NEM-1469)
app.add_middleware(RequestTimingMiddleware)

# Add request logging middleware for structured observability (NEM-1963)
if get_settings().request_logging_enabled:
    app.add_middleware(RequestLoggingMiddleware)

# Add request recording middleware for debugging production issues (NEM-1964)
if get_settings().request_recording_enabled:
    app.add_middleware(RequestRecorderMiddleware)

# Add RFC 8594 deprecation headers middleware (NEM-2089)
app.add_middleware(DeprecationMiddleware, config=_get_deprecation_config())

# Add deprecation logger middleware for tracking deprecated endpoint usage (NEM-2090)
app.add_middleware(DeprecationLoggerMiddleware)

# Security: Restrict CORS methods to only what's needed
_cors_origins = get_settings().cors_origins
_allow_credentials = "*" not in _cors_origins
app.add_middleware(
    CORSMiddleware,
    allow_origins=_cors_origins,
    allow_credentials=_allow_credentials,
    allow_methods=["GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"],
    allow_headers=["*"],
)

# Add security headers middleware for defense-in-depth
app.add_middleware(SecurityHeadersMiddleware, hsts_preload=get_settings().hsts_preload)

# Add body size limit middleware to prevent DoS attacks (NEM-1614)
app.add_middleware(BodySizeLimitMiddleware, max_body_size=10 * 1024 * 1024)

# Add GZip compression middleware for response compression (NEM-3741)
app.add_middleware(GZipMiddleware, minimum_size=1000, compresslevel=5)

# Add idempotency middleware for mutation endpoints (NEM-1999)
if get_settings().idempotency_enabled:
    app.add_middleware(IdempotencyMiddleware)

Additional Middleware¶

The middleware directory contains 25 modules. Key middleware not detailed here include: setup_guard.py (SetupGuardMiddleware - blocks API with 503 until admin setup is complete), etag.py (ETag caching), profiling.py (request profiling), prometheus.py (metrics), baggage.py (context propagation).

SetupGuardMiddleware¶

The SetupGuardMiddleware (backend/api/middleware/setup_guard.py) is a critical middleware that returns 503 for all API requests until the first admin user has been registered. This ensures the system is properly initialized before accepting traffic. It should be considered part of the execution order, running before AuthMiddleware.

Related Hubs¶

• API Reference - Endpoint documentation
• Observability - Logging and tracing integration
• Security - Security middleware and authentication
• Resilience Patterns - Circuit breakers and retry logic

Last updated: 2025-01-24 - Complete middleware hub documentation for NEM-3461