raggate quickstart — run the gate in a few minutes

Step 1

Run the whole story on the sample corpus

git clone https://github.com/ywc668/rag-rollout-policy-gate
cd rag-rollout-policy-gate

uv sync --extra dev                      # install
bash scripts/demo.sh                     # ingest, gate twice, diff, render report
open out/report.html                     # the report

The demo ingests the bundled two-tenant corpus, runs the gate against two permission snapshots, diffs the access surface across two embedding configurations, and renders the HTML report. It leaves everything in out/.

Expect a FAIL. The sample corpus has three planted leak classes, so a successful run exits 1 and names them. That is the tool demonstrating it works. A silent pass would be the thing to worry about — which is why 10 negative-control scenarios run alongside and must stay clean.

Or drive it by hand

uv run raggate ingest --corpus corpus --db out/gate.db

uv run raggate run   --db out/gate.db --snapshot corpus/acl_snapshots/v2.yaml \
                     --scenarios corpus/scenarios --evidence out/evidence.jsonl   # exit 1 = leaks found

uv run raggate diff  --corpus corpus --snapshot corpus/acl_snapshots/v2.yaml \
                     --scenarios corpus/scenarios --out-jsonl out/diff.jsonl

uv run raggate report out/evidence.jsonl -o out/report.html --diff out/diff.jsonl

There is also raggate quickstart, which runs the gate on a bundled sample and opens a report without any setup.

Step 2

Point it at your own retrieval

Two ways in, depending on whether you want to test your pipeline or your store.

Your pipeline, over HTTP

Wrap your retriever — filters, reranking, query rewriting, all of it — behind one endpoint that accepts a query, an identity and a top_k, and returns the chunks it would really serve. Then run the gate against that endpoint.

This is the honest option: what gets tested is the pipeline you ship, including any ACL filtering you already apply. The request and response shapes are documented in docs/spec/retrieval-endpoint.md.

Your vector store, directly

Ten adapters ship in the repository: sqlite-vec, Elasticsearch, OpenSearch, pgvector, Qdrant, Milvus, Weaviate, Redis, Azure AI Search, and one for a self-hosted RAG product. Ingest a corpus and gate it without writing any glue.

Relevance scores are recomputed locally as exact cosine, so a threshold means the same thing regardless of how the backend normalises its own scores.

What you have to supply

Input	What it is
Identities	The roster of principals that issue queries — an id and the tenant each belongs to.
A permission snapshot	Who may read which document, and group membership. Versioned, so you can gate a stale index against current policy.
Scenarios	An identity plus a query, and optionally the documents that would be a leak if returned. Ordinary questions work best — the useful ones are what a real user would actually ask.

The corpus and scenario formats are plain YAML and JSON, kept deliberately backend-neutral.

Step 3

Put it in CI

The gate is a command with meaningful exit codes, which is the whole integration story.

Exit code	Meaning
`0`	No leaks in the scenarios you ran.
`1`	At least one forbidden document reached the context window. Fail the build.
`2`	The run could not be trusted — bad arguments, an unreachable backend, a malformed response. Never confused with a clean result.

Same inputs, same verdicts. Runs are deterministic by design — the default embedder is seed-stable and offline — so a gate result that changes without an input changing is treated as a bug rather than noise. That is what makes it usable as a build gate.

See what it produces What costs money

See it catch three planted leaks before you trust it with anything real.