Skip to content

Commit 52a40be

Browse files
varun369claude
andcommitted
docs: update README with v0.6.2 benchmarks and corrections
- Add Performance section with measured operator latencies (4.4µs chain, +0.85ms proxy overhead) — reproducible via bench scripts - Fix coverage badge: 100% → 94% (accurate for packages/core) - Add Tests badge: 292 passed - Fix CLI command: agentassert-proxy proxy start → agentassert-proxy start - Fix docs table: "All 7 operators" → "All 10 operators" Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 5820ae9 commit 52a40be

1 file changed

Lines changed: 40 additions & 6 deletions

File tree

README.md

Lines changed: 40 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@
1515
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
1616
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
1717
[![arXiv](https://img.shields.io/badge/arXiv-2602.22302-b31b1b.svg)](https://arxiv.org/abs/2602.22302)
18-
[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen.svg)](packages/core)
18+
[![Coverage](https://img.shields.io/badge/coverage-94%25-brightgreen.svg)](packages/core)
19+
[![Tests](https://img.shields.io/badge/tests-292%20passed-brightgreen.svg)](packages/)
1920
[![Twitter](https://img.shields.io/badge/Twitter-@varunPbhardwaj-1DA1F2.svg)](https://twitter.com/varunPbhardwaj)
2021

2122
---
@@ -47,7 +48,7 @@ It is **not** a content filter (Guardrails AI does that). It is **not** an offli
4748
pip install agentassert-typec-proxy
4849

4950
# Start the proxy with a safety contract
50-
agentassert-proxy proxy start --contract safety-minimal.yaml
51+
agentassert-proxy start --contract safety-minimal.yaml
5152

5253
# Point your agent at it (works with any tool that reads env vars)
5354
export ANTHROPIC_BASE_URL=http://localhost:9000/anthropic
@@ -96,7 +97,7 @@ export GEMINI_BASE_URL=http://localhost:9000/gemini
9697

9798
Start the proxy using a contract containing the required limits (e.g., safety, cost ceiling):
9899
```bash
99-
agentassert-proxy proxy start --contract safety-minimal.yaml --port 9000
100+
agentassert-proxy start --contract safety-minimal.yaml --port 9000
100101
```
101102
Restart your Antigravity environment to apply.
102103

@@ -244,7 +245,7 @@ agentassert-typec proxy start --contract contract.yaml
244245

245246
```bash
246247
brew install agentassert-typec
247-
agentassert-proxy proxy start --contract safety-minimal.yaml
248+
agentassert-proxy start --contract safety-minimal.yaml
248249
```
249250

250251

@@ -386,6 +387,39 @@ satisfaction:
386387
387388
---
388389
390+
## Performance
391+
392+
Type-C is designed to be invisible at runtime. Benchmarks run on Apple M5 Pro, Python 3.14, 1,000 iterations each.
393+
394+
### Operator Evaluation Speed (all 10 operators, no API key needed)
395+
396+
| Operator | Mean | p99 |
397+
|---|---|---|
398+
| `tool_blocklist` (deny path) | 0.8µs | 1.0µs |
399+
| `tool_blocklist` (allow path) | 0.3µs | 0.4µs |
400+
| `tool_allowlist` | 0.3µs | 0.3µs |
401+
| `must_precede` (deny path) | 0.8µs | 0.9µs |
402+
| `cost_ceiling` | 0.9µs | 1.0µs |
403+
| `repetition_guard` | 0.3µs | 0.3µs |
404+
| `pii_filter` (6 pattern types) | 0.2µs | 0.3µs |
405+
| `drift_tracker` update | 0.2µs | 0.3µs |
406+
| **All 10 operators combined** | **4.4µs** | **< 6µs** |
407+
408+
**0.004ms total** for the full operator chain. Typical LLM response latency is 500–3,000ms — Type-C evaluation is < 0.1% of that.
409+
410+
### Proxy Round-Trip Overhead (200 requests, mock upstream, no LLM cost)
411+
412+
| Metric | Direct | Via Type-C | Added |
413+
|---|---|---|---|
414+
| Mean latency | 0.45ms | 1.30ms | **+0.85ms** |
415+
| p99 latency | 0.60ms | 1.90ms | **+1.30ms** |
416+
417+
**+0.85ms mean overhead per request.** Invisible in practice — network RTT to any LLM provider alone exceeds this by 50–100×.
418+
419+
Reproduce with: `uv run python scripts/bench_operators.py` and `uv run python scripts/bench_proxy_overhead.py`
420+
421+
---
422+
389423
## The Math Behind the Contracts
390424
391425
AgentAssert Type-C is backed by [arXiv:2602.22302](https://arxiv.org/abs/2602.22302) — *"Formal Behavioral Contracts for AI Agents"*.
@@ -409,7 +443,7 @@ Where C̄ = constraint satisfaction rate, D̄ = mean JSD drift, E = error rate,
409443
You run a coding agent in production. One bad session could `rm -rf` a directory. The proxy blocks it before the API call is ever made.
410444
411445
```bash
412-
agentassert-proxy proxy start --contract safety-minimal.yaml
446+
agentassert-proxy start --contract safety-minimal.yaml
413447
export ANTHROPIC_BASE_URL=http://localhost:9000/anthropic
414448
```
415449

@@ -474,7 +508,7 @@ You want to catch low-conviction responses ("it depends", "either works") withou
474508
| Doc | Contents |
475509
|---|---|
476510
| [Getting Started](docs/getting-started.md) | All 5 integration paths with full examples |
477-
| [Contract DSL Reference](docs/contracts.md) | All 7 operators, YAML schema, examples |
511+
| [Contract DSL Reference](docs/contracts.md) | All 10 operators, YAML schema, examples |
478512
| [Proxy Guide](docs/proxy.md) | Multi-provider setup, streaming, hot reload |
479513
| [SDK Guide](docs/sdk.md) | Python SDK wrap, async, OpenAI + Anthropic |
480514
| [Claude Code Guide](docs/claude-code.md) | Hook adapter, templates, installer |

0 commit comments

Comments
 (0)