Hardware Profiles: The Tiering Strategy¶

Errata (2026-09-23): E1, E3, E4, E6, E9, E10, E17, E18 in 11-errata-2026-09-23.md correct claims in this document. The original text is kept as the record; read those entries before relying on it. Current design: 2026-09-23-vss-gaming-gpu-profile-design.md.

Proposed 2026-09-19. This reframes the product thesis from "port VSS to consumer GPUs" to "contribute a consumer-GPU profile tier, for which a reference implementation already exists."

The gap is confirmed unowned [O, 2026-09-19]¶

Source: 1:1 with the VSS Product Manager, 2026-09-19. Consumer GPUs are NOT on the VSS roadmap.

This is the strongest evidence in this directory and it outranks every inference drawn from the repo. Everything below was originally argued from absence — no GeForce row in sizing.md, an empty RTX4090_TESTS table, edge profiles that are unified-memory. Absence is ambiguous; it could have meant "next quarter." A PM owns roadmap, so it does not mean that.

It also retires the in-house risk. The governance record showed issue #1345 closed with a maintainer stating they intended to build Redis Stream support themselves — the "we'll do it ourselves" pattern. That pattern does not apply here.

What this does NOT settle, and must not be read as settled: redistribution rights (the microservices ship under an Evaluation license; README.md:99 requires AI Enterprise to self-host a NIM; SLA §8.9 bars publishing benchmark data — a legal question, not a roadmap one), whether NVFP4 computes on sm_120, or the biometrics exposure in 06.

Open, and worth asking next: which "not on the roadmap" this is — (a) we'd want it but have no headcount, (b) we evaluated the segment and judged against it, or (c) we have not considered it. (a) makes this a welcomed contribution; (b) means the demo must precede the argument; (c) means you are introducing the idea and carrying it alone. The three imply different first moves.

Timing note. Not-on-the-roadmap is a window, not a moat. The thing most likely to change a roadmap is a working demonstration — which argues for early visibility over a privately perfected fork, and is a second reason to lead with the conformance test rather than a finished derivative.

Why profiles, and why this is the right frame¶

Profiles are VSS's native idiom, not an imposition on it [V]:

• deploy/docker/developer-profiles/ — base, search, lvs, alerts
• deploy/docker/industry-profiles/ — warehouse-operations and others
• hw-DGX-SPARK.env, hw-AGX-THOR.env, hw-IGX-THOR.env, hw-OTHER.env per NIM, each with a -shared variant
• Foundations: alerts, search, warehouse, public safety, smart city

Adding hw-GEFORCE-* is speaking VSS's language. It asks the project to extend a pattern it already maintains rather than to change shape.

The consumer market is not one tier. The 32 GB RTX 5090 is the halo card. The volume segment is 12-16 GB — 5080, 5070 Ti, 4070 Ti Super, 3060 12 GB. A single "consumer profile" aimed at 32 GB addresses the smallest slice of the market it claims.

The tiers¶

The line where the VLM approach dies [C]¶

At 16 GB the budget is 13.6 GB. The 12B-VL NVFP4 checkpoint is 13.78 GB — it does not fit solo, before RT-CV, before RT-Embed's fixed 10 GB.

Below roughly 24 GB, a single large VL model is not viable. The only remaining shape is specialized small models producing text, with a small text-only LLM reasoning over it. That is precisely the architecture in ai/, designed for a 24 GB budget and compressible further.

The structural advantage¶

VSS's single-GPU blocker is architectural, not arithmetic [V] (deploy/docker/scripts/dev-profile.sh:1161-1165):

Two vLLM engines cannot share one GPU at all. Each sizes itself as (fraction × total) − (memory held by every other process) and then expands its KV cache to fill the remainder, so no pair of fractions satisfies both.

RT-VLM and the LLM NIM are both vLLM engines, so VSS's default shape cannot run on one card at any precision.

Our architecture does not have this problem. YOLO26, Florence-2, and CLIP are plain HTTP model servers (since the consolidation, one ai-gateway container on port 8090 with routers /yolo26 /florence /clip /enrichment /enrich-lt), not vLLM engines. One vLLM engine for the text LLM plus N small model servers is a supported configuration. The constraint that blocks VSS on a single GPU does not bind here.

This is the core of the contribution claim: the existing architecture is better suited to consumer hardware than VSS's is. Not a compromise — a genuine fit advantage, and the reason a reference implementation for the volume tier already exists.

Corroborating: the duty-cycle gap¶

VSS has no idle/evict/on-demand path; its alert route is literally /api/v1/realtime/always-on [A]. This repo has gpu_oom_handler, model_zoo LRU eviction, and 90-second event-triggered batching. On a consumer box the GPU is shared with whatever else the owner is doing, so always-on residency is a product defect, not just an efficiency one.

Caveat [?]: whether this layer is good or merely present is unverified. It was written by the same regime that produced a 54% mutation score, and its own tier's tests largely do not collect (see 06-repo-a-readiness.md). Making ai/ collectable answers this, and is the reason that task outranks further mutation census work.

The reasoning model for the volume tier¶

Requirement: text-only (vision handled by the CV zoo), small footprint, permissive license, no NGC entitlement.

Qwen is strategically safer than it looks¶

VSS already ships Qwen support [V] — Qwen3-VL-30B-A3B-Instruct and Qwen3-Omni-30B-A3B-Instruct in docs/real-time-vlm.mdx, Qwen3-VL and Qwen3.5-27B in sizing.md. NVIDIA's own blueprint blesses Qwen, so a Qwen-based tier is not off-message inside NVIDIA.

It also directly dissolves the credential blocker: Apache-2.0 weights on HuggingFace, no NGC gate, no AI Enterprise licence, nothing for a consumer user to hold an entitlement for. Compare the NVIDIA path, where microservices ship under an Evaluation license, README.md:99 requires AI Enterprise to self-host a NIM, and SLA §8.9 bars publishing benchmark data — which covers the sentence "this runs well on a 5090" [A].

Candidates to size¶

Keep both families in play; the tier should be model-agnostic by construction.

⚠️ Size from HuggingFace blob sizes, not the VSS formula. The formula understated the 12B-VL NVFP4 checkpoint by 43% because the vision tower stays BF16. Text-only models lack that particular trap, so the formula should behave better — but the lesson stands: verify against blobs before committing to a tier boundary. See 04-fp4-and-deployment.md.

What the volume tier has to prove¶

Not latency. Salience.

Home security is a false-positive suppression problem; VSS is a description problem. A model that returns schema-valid MEDIUM on an empty porch is product-worthless regardless of how fast it does it. The volume tier's acceptance test is the boring-frames test in 06-repo-a-readiness.md §4, not a tokens-per-second number.

Open questions for this document [?]¶

1. What are the real blob-level footprints of the candidate text LLMs at FP8 and 4-bit?
2. What does the CV zoo actually cost in VRAM when co-resident? ai/ has YOLO26, Florence-2, CLIP, enrichment, and enrichment-light. Which are simultaneously resident vs LRU-evicted?
3. Can RT-CV be swapped in for YOLO26 in the volume tier, or does DeepStream drag in too much? RT-CV is ~3.0 GB but hits 95-96% SM utilization at 2-3 streams [A] — on a shared consumer GPU that may be disqualifying regardless of VRAM.
4. Is there an 8 GB tier that still does something a user would pay for, or does the product floor sit at 12 GB?

Related¶

• 04-fp4-and-deployment.md — the halo-tier numbers and the FP4 trap
• 06-repo-a-readiness.md — what must be fixed here first
• 03-open-questions.md — the open register