Design rationale

Nix and NixOS: everything is a declaration

Most toolchains are a pile of imperative steps: install a runtime, pip install some packages, docker build an image, tofu init some providers — each step fetching whatever the internet serves that day. This lab rejects that pattern: every artifact is a declared, hash-pinned derivation, and the tools that “build” (docker, tofu) are demoted to loaders and appliers. If you have never used Nix, four files carry the whole toolchain:

File Declares Updated by Verified by
flake.lock the tools — kubectl, helm, tofu, node, uv nix flake update validate CI
uv.lock (+ pyproject.toml) every Python package the lab uses Renovate gate-contract CI (verdict matrix + app import chain)
nix/images.nix the container images — layer graph, entrypoint, MiniLM weights pinned to a Hugging Face revision with per-file sha256 Renovate / review docker-build CI (both images build on x86_64)
terraform/.terraform.lock.hcl the tofu provider set and their hashes review tofu init -lockfile=readonly in CI

Nothing is “installed” in the classic sense. nix develop does not run pip — it materialises the venv uv.lock already pins, byte-identically, from a content-addressed store. The container images are not built by docker either: nix/images.nix constructs each image as a streamLayeredImage — layers [python venv][model cache][app config], docker-loadable, base-image-free — and .#load-* pours the finished stream into docker load or k3s ctr images import. Docker never assembles layers from a floating FROM line and unpinned pip layers; it only loads and runs what Nix already decided. The NixOS module (nixos/configuration.nix) declares the machine the same way: k3s (via services.k3s — the cluster’s existence is reviewable text in Git), the toolchain, KUBECONFIG, the hosts entries — and no docker daemon, because no docker building is left to do. One nixos-rebuild switch --flake .#aisec-lab converges a fresh OrbStack machine into a working lab.

The practical consequences, for someone who has never touched Nix:

  • No “works on my machine”. Every colleague, CI runner, and the VM resolve the exact same hashes for tools, packages, images, and providers.
  • Bumps are safe by construction. Renovate PRs a lock change; CI re-tests the actual contract (Laya verdicts, Nova rule scanning, the FastAPI chain) against the new set before anything merges.
  • Supply chain you can audit. The MiniLM weights the semantics tier loads are fixed to a specific HF repo revision — an upstream weight swap changes nothing here, because the bytes are hash-pinned in the flake.
  • One command per plane. nix develop (shell), nix run .#load-* (images), nixos-rebuild switch (machine), nix run .#tofu apply (cluster) — each realises declarative text, none runs a script.

NGINX Gateway Fabric at the edge

Prompt screening is a security control, and a security control must fail closed. Gateway-API inference extensions (EPP, InferencePool) are load-balancer machinery and default to failureMode: FailOpen — if the pool is unavailable, traffic flows past the check. Screening inside the pool also means the prompt has already entered the cluster: flow established, metrics exposed, session state spent. NGF at the edge blocks with a 403 before any of that exists, and keeps policy decoupled from the serving path’s version churn — the same reason a WAF sits in front of the app, not inside it.

Figure 04 — Why the gate sits at the edge, not inside the inference layer.

Figure 04 — The placement argument in one picture: fail-closed edge enforcement beats in-pool screening.

A vulnerable application, not a simulator

Gate tests against a simulated LLM establish only that strings are blocked, not that the attack succeeds. Atlas runs a real model with a real tool loop, so the ungated path first demonstrates the leak, and the gated path then demonstrates the block.

Model selection

Each model is the smallest tool that does its job:

Role Model Size Why this one
NOVA LLM-tier judge llama3.2:3b ~2 GB Reads intent in ~200 ms–2 s per judgement on the Mac’s GPU
Atlas engine gemma4:12b-mlx ~8 GB Behaves like a real assistant (multi-turn tool loop, instruction following); MLX build = native Metal speed
NOVA semantics tier all-MiniLM-L6-v2 ~90 MB Pinned to a HF revision, hash-verified, pre-baked into the gate image by the flake
Laya decision model (bundled with laya[serve]) ~808 MB Typed injection/jailbreak/benign verdict, one CPU forward pass; loads on gate-container boot

The entire defence stack is local, air-gap-capable, and costs $0/token.

Notebooks and nbdev

Code, tests, prompts, and prose in one place — nbdev_test is the lab’s acceptance suite; each markdown cell states the following cell’s function and expected output; nbdev_docs publishes the set as a static site. The notebooks are the source of truth; gate/, terraform/, and nova-rules/ are generated from them and committed, so artifacts and notebooks cannot diverge — and the artifacts themselves stay runtime-agnostic: moving hosts changes where they run, not what they are.

The hybrid constraint

An OrbStack Linux machine runs on Apple’s Virtualization.framework, which exposes no GPU/Metal to guests — containerized inference in the VM is CPU-only, 3–6× slower than native on Apple Silicon (measured publicly on 8B-class models). So the lab is hybrid, with exactly one seam:

  • macOS host (native, Metal): Ollama — the LLM-tier judge (llama3.2:3b) and the victim’s engine (gemma4:12b-mlx), at full GPU speed. Nothing else lives on the Mac.
  • NixOS VM (aisec-lab): k3s cluster, NGINX Gateway Fabric edge, the nova-gate pod (NOVA + Laya, CPU), the Atlas victim app, and the whole lab toolchain — all declared in flake.nix + nixos/configuration.nix.

The gate pod reaches native Ollama over host.orb.internal:11434 (OrbStack DNS, OpenAI-compatible endpoint). Everything stays on the MacBook; no cloud keys anywhere — no external API keys are involved at any point.

Figure 05 — The flake pins everything; the machine is the declaration.

Figure 05 — Toolchain: the flake pins every tool and the NixOS module declares the machine — one path, no drift.

Known limits

  • Laya zero-shot is weak — fine-tune for production (the project ships a Kaggle fine-tuning notebook). Its verdicts are probabilities, not verdicts; benchmark the judge on your own corpus before trusting thresholds.
  • Latency stacks by tier — keywords <1 ms, semantics ~15 ms, the LLM tier adds ~200 ms–2 s per judged prompt. That is why the condition logic short-circuits: cheap stages run first.
  • The victim’s canary key is fake by design, but it models the real failure: an assistant that carries credentials in its system prompt and can be steered into writing them to an integrated sink.
  • Model-level resistance is probabilistic — a 12B aligned model may sometimes refuse the poisoned note. The gate’s fail-closed verdict is not probabilistic; the gate’s verdict is deterministic while the model’s is not.
  • The gate scans the prompt channel only. Atlas’s poisoned RAG note is an indirect injection — attacker-controlled text arriving in a tool result, not a user prompt — so prompt screening cannot see it. That is why the direct-path proof exists, and why a production gate must also screen tool/RAG traffic: out of scope here, stated so the boundary is explicit.
  • Nova/Laya versions move fast — pin them in pyproject.toml/uv.lock and bump deliberately between lab runs, never mid-run; a bump that breaks the contract fails dep-contract CI before it can merge.
  • The VM’s k3s runs with default sandboxing; the gate pod’s 2–5Gi memory limits assume the VM gets ≥8 GB.