12 — Postponed Roadmap¶

Items the VSS gaming-GPU effort deliberately postponed on 2026-09-23, so a future agent neither re-discovers them nor mistakes their absence for an oversight. The current effort is the design in 2026-09-23-vss-gaming-gpu-profile-design.md: detector plus one VLM, FTP stills, a single A5500, with milestones M0-M4.

Each item records why it waits, the trigger that reopens it, and where its evidence lives. When an item is picked up, move it into its own spec and mark it here with the date and a link.

Index¶

The test-platform items open in 06 (§1.5 deletion cashing, the §1.6 import-bound seam fixture, the §1.7 frontend classification) belong to that program and stay tracked there.

R1. Live-stream ingest¶

Goal (owner, 2026-09-23): the platform streams every camera in real time and runs its own motion and object detection instead of relying on camera FTP uploads.

Why it waits: the design fixes ingest to FTP stills to keep the first go-live bounded. The VLM path is ingest-agnostic (key_frame_selector takes image references from any source), so a stream front-end plugs in above the detector without changing it.

Candidate front-ends:

Constraint to carry forward: continuous decode and detection spend GPU all the time, which works against the duty cycling the design builds. Budget NVDEC and SM headroom alongside the VLM on the same card. RT-VLM's live-stream alert mode fires on the substring "yes" and records no rejections, so it is unusable as shipped for false-positive suppression (09 §5).

R2. The MQTT / Home Assistant / Frigate chain¶

The chain is built but never started. mqtt_client.py (828 lines), mqtt_publisher.py, ha_discovery.py and frigate_integration.py have no non-test caller, and compose runs no broker. Wiring it means adding a broker service (port variable in .env.example first, per the root AGENTS.md port rule), starting the chain in the application lifespan, and covering it with integration tests.

R3. NemoClaw on the VLM¶

NemoClaw's custom provider accepts our llama.cpp ai-vlm endpoint, and the candidate VLMs ship tool-calling templates. RT-VLM cannot serve this role because it rejects tools. Full findings are in the 10 addendum. To pick it up, solve four things:

1. Capability: score the chosen VLM with VSS's skill-eval nemoclaw runtime against the frontier default.
2. Contention: partition llama.cpp slots, or prioritize, so agent sessions leave the per-event 30 s p95 intact.
3. Reachability: design how an OpenShell sandbox reaches the endpoint while loopback binding stays the security boundary.
4. Parsing: confirm llama.cpp parses the chosen model's tool calls.

The design prepares for this by making "tool calling through llama.cpp works" a bake-off criterion.

R4. Agent features¶

These are natural-language Q&A and chat over events, an MCP tool server over the events API (with auth, which VSS's lacks), natural-language search with a VLM re-check of results, and narrative incident reports. 09 §4 shows the VSS agent (NAT 1.8) satisfies its LLM contract against llama.cpp through configuration: --alias, llm_reasoning: false, and an LLM_NAME matching the model family. Estimated effort is 300-600 LoC for the MCP server plus ~150 lines of YAML [A]. Two small enablers are deferred from the design: adding scene_description to the events full-text search trigger, and using VLM verdicts as feedback-learning signal.

R5. Per-camera plain-language rules¶

An example rule: "person at the side gate after dark". It runs through the same vlm_assess engine on event stills. It also gives the stored-but-never-evaluated alert rules (E26) a real evaluator.

R6. YOLO-World¶

YOLO-World is in the model set but unused. Wiring it gives open-vocabulary, text-prompted detection at the gate, without adopting VSS's detection service.

R7. Audio¶

Glass break, alarms and raised voices are valuable signals. The smallest VSS audio path (Omni 30B-A3B) needs ~29 GB and fits no consumer card. First confirm whether Foscam clips carry audio.

R8. Post-go-live deletion¶

Picked up — 2026-09-29 [O]. R8 left the deferred index by owner instruction and has its own spec: 2026-09-28-r8-legacy-retirement-design.md. Its trigger here ("Go-live holds 14 days") was reversed, not met — the owner opened it early, ruled that backwards compatibility is not required and that PIPELINE_MODE=legacy must raise, and ruled out the A5500 for this work. That spec, plus the measured scope at 2026-09-28-r8-legacy-retirement-scope.md, is now the authority on what goes; the text below stays as the 2026-09-23 record and is where §2a's parked question and the __pycache__ trap are still explained.

What goes: retire Florence-2, the heavy enrichment models, the in-process loaders the VLM path no longer calls, the pose/demographics/clothing child tables, and their frontend panels (the design's empty states become deletions).

Since spec rev 5 (2026-09-25) it also removes the unsupported legacy LLM path: ai-llm, the per-event route in nemotron_analyzer.py (including the 0.3 changes), and PIPELINE_MODE=legacy.

What stays: the face, re-ID and plate specialists.

What changes elsewhere:

• This resolves 06 §2a's parked in-process-tier question by retirement, not by swapping.
• Before deleting, clear stale __pycache__ directories. A stale backend/api/helpers/__pycache__ false-reddened a deletion guard on 2026-09-23.

R9. Upstream contributions to VSS¶

Ordered so that the earliest contributions ask the least of VSS reviewers:

1. Defect reports with patches:
2. The README NVFP4 recipe fails at boot because of the allowlist variable name (E13).
3. The RT-VLM live-alert substring trigger (09 §5).
4. dev-profile.sh requires NGC_CLI_API_KEY even for all-remote deployments (08 §0).
5. Expose RT-VLM's fixed-KV knobs in compose: RTVI_VLLM_KV_CACHE_MEMORY_BYTES, the KV dtype, and eager mode (08 §2.6).
6. Wire sleep and unload in RT-VLM. VSS model services have no unload path.

7. A consumer hardware tier on the base Foundation:

8. naming RTX4090 / RTX5090, or a VRAM-class alias;

9. roughly 39 files per the checklist in 08 §1.7;
10. helm parity enforced in CI;
11. the three dormant RTX 4090 runners as CI hardware (E17).

The design's Delta build (M4) is the downstream prototype.

Before item 4, ask the VSS team which kind of "not on the roadmap" applies (no headcount, judged against, or not considered); 05 explains why the answer changes the first move.

R10. Final model choices¶

Reasoning LLM (owner). The owner reports that Nemotron 3.5 Lightning replaced the Nano line. NVIDIA-Nemotron-3-Nano-4B was the interim placeholder; spec rev 5 (2026-09-25) removed it. Facts about Lightning, gathered 2026-09-23:

Under the design, the per-event path uses no separate text LLM. A text LLM returns only for the R4 features, and the chosen VLM may serve them too.

VLM. The candidates are Nemotron-Nano-12B-v2-VL, Qwen3-VL-4B/8B and Cosmos3-Edge-4B. The design's bake-off produces the evidence; the owner picks at M2.

R11. Tiers beyond 24 GB¶

Partly pulled in (2026-09-23). Brev VMs (A10G, L4, RTX PRO 4500, T4) now supply the measurements: design step 2.3 records S1 and S4 per tier and settles the sm120 NVFP4 question. What stays postponed is _product support for those tiers: profiles, defaults and documentation.

R12-R14. Distribution prerequisites¶

These must be settled before any consumer distribution. None of them blocks the design.

• R12, licensing: model licenses (the Nemotron Open Model License, the NVIDIA Open Model License, Apache-2.0 Qwen); VSS microservices ship under an Evaluation license; SLA §8.9 bars publishing benchmarks; image redistribution. These are legal questions.
• R13, biometrics: ArcFace templates and plate text have no retention clock, consent record or erasure path. This is a product-shape decision.
• R14, version pinning: VSS restructures often and removes old NGC images on a schedule. Mirror the images you depend on, and budget for re-integration at each release.