Installation

Today — Install From Source

Registry Publish — Pending v1.0 GA Tag roadmap

5-Minute Quickstart

1Register Audit Codes — Once, at Startup

2Init — Reads Env Vars; FASTEN_AUDIT_DSN Required

3Emit Anywhere

Production: Env-Var Driven

Other Languages — Same Shape

Per-SDK READMEs in the repo carry the equivalent quickstart for each language; they all share spec/row-schema.json as the wire-format source of truth.

What You Actually See

Every emit() and log.* writes one NDJSON line to stdout. Docker's log driver captures and rotates it.

emit("USER_CREATED", …) → stdout

log.info("signup_complete", …) → stdout

Querying Back

Debugging a Real Incident

Scenario: A pipeline config change was applied at 14:32. By 14:33 the MQTT broker shows message loss.

Without fasten

1. docker logs gateway | grep 14:32 — hundreds of lines, no actor info
2. docker logs pipeline-engine | grep 14:32 — different format, different timestamps
3. Cross-reference manually — which request caused which effect? Unknown.
4. Ask on Slack "who changed the config?" — 20-minute delay

With fasten — One Query, 30 Seconds

The 7 Anchors

5 Ws + H + CORRELATION — enforced at the type level. Caller supplies only code, target, and optional detail. Everything else auto-fills.

WHY lives in detail.reason (free text). A policy plugin can enforce it on mutation codes.

Three Streams

All three carry request_id. A ?request_id=<id> query returns rows from all services that carried it.

Shims

Nothing default-on. Import only what your transport uses. Each shim does one thing: read an id from the wire → stash in context → propagate downstream.

Schema

fasten owns its audit table; your product owns its own DB. They share a request_id join key and never share a table. Two storage topologies are supported:

fasten_audit Columns

Catalog YAML — Single Source Across SDKs

Code catalogs can be declared in *.codes.yaml files instead of programmatically. Same file works across every SDK; reload is atomic + fault-tolerant (parse / validate fully into a fresh dict, then swap under a lock; on any failure the previous catalog stays active).

Audit-Store Failure Handling

Audit failures must never break the request being audited. emit() defaults to queue mode: rows are pushed onto a bounded in-memory queue, drained by a background thread with exponential backoff (100 ms → 60 s, ±20 % jitter). A locked / down store no longer cascades into 5xxs on the request path.

Sys-Stream Self-Report

The drainer never silently degrades. State transitions write {shape:"sys"} NDJSON lines so existing log aggregation (Loki, Splunk, journald, or whichever hosted log indexer you run) catches audit-pipeline issues without any new alerting plumbing. Adopter request_id propagates onto every line.

Programmatic Snapshot — queue_stats()

Equivalents: Go fasten.GetQueueStats() (returns *QueueStats); JS queueStats(); Rust fasten::queue_stats(); C++ fasten::queue_stats(). All return null / nil / None in raise mode.

Deterministic Shutdown — flush()

For k8s preStop hooks, CLI exit paths, or test teardown — block until pending rows drain. No-op + true in raise mode so adopter shutdown code is mode-agnostic across both strategies.

Security Model

fasten is responsible for the row's integrity from emit to insert, and for any surface fasten itself exposes — but never for the security of storage you brought.

Responsibility Split

Wire Your Existing Auth

The reader is a FastAPI router. Mount it like any other route, behind whatever auth your app already runs.

fasten's reader doesn't add its own check here. Two auth layers fight each other; one well-placed layer is the point.

Reserved Headers

What Stays Out of Scope

• TLS termination — your reverse proxy / load balancer.
• Network ACLs — your VPC / firewall / k8s NetworkPolicy.
• Key distribution — your secret manager (Vault, k8s Secret, AWS SSM).
• DB encryption at rest — your DB layer (e.g. Postgres TDE) or OS-level on the SQLite file.

API Mounting

fasten's reader is a mountable router — not a standalone service. Wire it in like any sub-router under whatever prefix you choose.

Python (FastAPI)

Go (chi)

Node.js (built-in http or Express)

JS doesn't ship a mountable router today; wire X-Request-ID via withRequestID() in your handler, and surface the audit / sys / api streams over your existing HTTP framework. See the js/examples service for the pattern.

Rust (tiny_http or axum)

The Rust SDK is sync-by-default. Wrap each request in with_request_id() and call EmitBuilder::submit() inside.

C++14 (single-header)

Use the bundled FastenReader + reader_simplehttp_main.cpp wiring; or call into your own HTTP server.

Swift (SPM)

SQLite-backed store out of the box; uses system sqlite3 and swift-crypto. Async request_id via @TaskLocal.

Mounted Reader Endpoints — at a Glance

Adopter Middleware Accessors

For adopters writing custom middleware or logging layers that need the same primitives emit() uses internally. Python and Go expose these as accessor functions; JS, Swift, and C++ expose equivalent standalone functions.

Env-Var Reference

No fasten.yaml. No config daemon. Env-vars only.

Retention + PII

PII — Three Mechanisms

1. Redaction at emit time — two passes, applied before writing anywhere:

Key-pattern pass — substring match, case-insensitive: api_key, password, passwd, token, secret, authorization, bearer, m2m_key, cert_private, private_key, access_key, session_id, cookie, credential. Any key containing one of these patterns (e.g. customer_token, user_password) has its value replaced with ***. Extend with FASTEN_REDACT_KEYS=ssn,dob or extra_redact_keys= in init().

Value-shape pass — string values that look like known secret formats are replaced with a type-hinting token regardless of key name: JWT → ***JWT***, PEM private key → ***PRIVATE_KEY***, AWS access key (AKIA/ASIA) → ***AWS_KEY***, GitHub token (ghp_/ghs_/…) → ***GH_TOKEN***, Stripe live key → ***STRIPE_KEY***, OpenAI key → ***OPENAI_KEY***, credit-card numbers passing Luhn check → ***CC***. Key-pattern fires first — if the key matches, the value is never inspected for shape.

2. PII class flag — pii_in_detail=True does two things: forces retention to short (30 days) regardless of declared class, and replaces the entire detail dict with {"_redacted":"***","_pii_in_detail":true} at emit time. Individual fields can be preserved via detail_passthrough_keys:

3. TTL sweep — rows beyond their class are deleted on schedule. For GDPR right-to-erasure: delete rows by actor or target, then let TTL sweep clean up the rest.

Operational FAQ

What Is the Latency Overhead?

Queue mode (default): emit() returns in <0.1 ms — the background drainer handles the store write asynchronously. Raise mode: ~0.1–0.5 ms per emit() with SQLite WAL, ~1–5 ms with Postgres. fasten.log.* is ring + stdout only — sub-millisecond in either mode. High-volume codes (>100/sec): set high_volume=True in Meta to skip the SQL insert.

What Is the Memory Footprint?

~8 MB at full occupancy: syslog ring (10k × ~500 B) + API log ring (10k × ~300 B). Tune with FASTEN_SYS_RING_SIZE and FASTEN_API_RING_SIZE.

Does SQLite Contend Under Concurrent Writes?

WAL mode allows concurrent readers + one writer. Typical audit volumes are well below contention. Mark hot codes high_volume=True or switch to Postgres if you see it.

Does fasten Add a Thread or Process?

One drainer thread per process when audit-store failure handling runs in queue mode (the default). Python, Go, and C++ bind to the shared fasten-core C ABI drainer (a single Rust std::thread inside the shared library); Rust runs it natively. JS uses a setImmediate chain (single-threaded event loop, no native deps). Swift uses a Thread-based pure Swift in-process drainer. Set audit_store_failure_strategy="raise" if you want strict synchronous semantics with no background drainer.

What Happens if the DB Is Unavailable?

• The row is written to stdout regardless — Docker / journald captures it.
• Queue mode (default): the row is buffered in-memory; the drainer retries with exponential backoff (100 ms → 60 s, ±20 % jitter). emit() never raises on store failure. The drainer self-reports state transitions to the sys stream (audit_drain_failed, audit_drain_degraded, audit_drain_recovered) so existing log aggregation catches issues.
• Capacity covers queued + in-flight retry combined (default queue_capacity=100). When saturated, emit() blocks rather than silently drops. JS deviates here — single-threaded event loop means emit() stays sync; capacity is the high-water-warn threshold instead.
• Raise mode: emit() calls insert synchronously and raises fasten.AuditStoreError on failure. Adopter chooses how to react.
• Use fasten.queue_stats() for a programmatic snapshot, or GET /logs/audit/doctor for the same data over HTTP.

Does fasten Work in Air-Gapped Environments?

Yes. Stdout is the primary transport — no network required. SQLite works fully offline. Rows drain upstream when connectivity resumes.

Deployment Recipes

fasten is in-process, no sidecar. The only orchestration concerns are: pass env vars, mount a writable volume for the SQLite file (or wire Postgres), and call fasten.flush() from your shutdown / preStop hook so the queue drains before the container dies.

Docker Compose

Kubernetes

systemd

Observability — Wiring Drainer Events to Your Stack

P1-15 ships five sys-stream events covering the audit pipeline. They land on your existing log channel (the same stdout NDJSON your aggregator already scrapes) — no separate metrics endpoint required. Wire them to alerts in whatever you already use.

Loki / Grafana

Alert when the audit drainer degrades:

Hosted log monitor (vendor-neutral)

Most hosted log indexers expose the same idea — search by the JSON fields, roll up over a window, alert above a threshold. The query syntax below uses one common dialect; translate to your provider's selector + roll-up grammar as needed.

Prometheus (via vector / fluent-bit log → metric)

Convert the sys events to counters in your log pipeline; fasten doesn't expose a Prometheus endpoint directly (stays in-process, zero deps).

k8s Liveness Probe

The reader's GET /audit/doctor endpoint is k8s-friendly out of the box — JSON response with store.reachable and queue.retry_count_active. Wire it as a liveness probe so the pod restarts if the audit pipeline stays degraded.

Compliance Auditor Cheatsheet

Logging — Structured Sys Stream

fasten's four log functions write {"shape":"sys"} NDJSON lines to stdout and push them into the in-memory syslog ring (queryable via GET /logs/sys). They are not audit rows — no code catalog entry needed, no durable storage, no 5 Ws enforcement. Use them for operational diagnostics alongside emit() for compliance events.

All four auto-stamp request_id from the ambient context, service_id from init(), and timestamp. Any extra keyword arguments (Python), key-value pairs (Go), field object (JS), or Fields map (C++) are merged into the output line.

Python

Go

Key-value pairs follow slog-style: alternating string key + any value. ctx carries the request_id via fasten.WithRequestID(ctx, rid).

JavaScript / TypeScript

fields is a plain object. request_id comes from the active AsyncLocalStorage store set by withRequestID().

C++14

fasten::Fields is std::vector<std::pair<std::string,std::string>>. Request id comes from the active RequestScope.

Logger Shims — Use Your Existing Logging Library

If you already have a logging framework in the codebase, opt-in shims mirror every log call into fasten's syslog ring without changing any existing call sites.

Public Accessors — flush & queue_stats

Two functions expose the internal queue state from outside the audit pipeline. Use them in shutdown hooks, health probes, and tests.

flush(timeout) — Deterministic Shutdown

Blocks until every queued audit row has been written to the store, or until the timeout expires. Returns True / true if fully drained; False / false if timed out with rows still pending. In raise mode (no drainer) it is a no-op that returns True immediately — shutdown code is therefore mode-agnostic.

queue_stats() — Runtime Health Snapshot

Returns a snapshot of the drainer's current state. Returns None / nil / null in raise mode (no drainer running). Use it in health probes, status pages, or to feed your own metrics pipeline without mounting the full reader.

Reader Endpoint Reference

All three endpoints are served by the mountable fasten.reader.router(). Mount it under a prefix (e.g. /api/v1/logs) — the paths below are relative to that prefix. No built-in auth; always pass dependencies=[Depends(...)] or mount behind a gateway.

GET /audit

Query durable audit rows from the fasten_audit SQL table. Supports pagination. All parameters are optional and combinable.

Response shape:

GET /sys

Query the in-memory syslog ring (default 10 000 lines, configurable via FASTEN_SYS_RING_SIZE). Ring-only — no offset, no total. Rows are gone when the ring wraps.

Response shape:

GET /audit/doctor

Single-call audit-pipeline health snapshot. Same auth as /audit (applied at router level). Use as a k8s liveness probe, compliance auditor curl, or status-page data source.

Glossary

Code Evolution + Compatibility

Renaming a Code

Schema Contract

The 7 anchor columns are stable — never change type. detail is yours to evolve.

• Identity: id, origin_id, monotonic_seq, timestamp
• WHAT: code, action, severity
• WHERE: service_id, source_node_id, tenant_id
• WHO: actor, actor_kind
• WHOM: target, category, domain
• HOW: method
• CORRELATION: request_id
• Free JSON: detail — you own its schema

OTel / trace_id Bridge

Pass your OpenTelemetry trace_id as fasten's request_id — they become the same id. A future transport/otlp plugin will export audit rows as OpenTelemetry LogRecord entries, letting existing Grafana/Jaeger stacks query fasten rows.