Skip to content

Add activity.jsonl event log at festival and campaign levels #152

@obey-agent

Description

@obey-agent

Summary

Add a comprehensive activity.jsonl event log at two levels:

  1. Per-festival at <festival>/.fest/activity.jsonl — every fest CLI action that mutates that festival's state.
  2. Per-campaign at .campaign/fest/activity.jsonl — every fest CLI action that creates, promotes, or archives festivals at the campaign level.

Today we have <festival>/.fest/progress_events.jsonl, which only records task status transitions (completed, blocked, etc.). That is far narrower than the full set of state-mutating operations the CLI performs, and there is no campaign-level equivalent at all.

Motivation

  • Auditability. We cannot currently answer "when did this phase get created?", "when did fest validate last run?", "who ran fest workflow skip?" from inside the festival. Those answers exist only in shell history (if anywhere).
  • Metrics & retros. The festival methodology already emphasizes measured work (e.g., 327x leverage measurements). Without a canonical activity record we cannot compute time-to-promote, validation cadence, or phase scaffolding velocity.
  • Campaign-level orientation. .campaign/fest/ has navigation.yaml but no chronological record of festival creation, promotion, or archival. When a user returns to a campaign after a break, there is no way to see "what happened in the festival layer since I was last here."
  • Debuggability. When a festival ends up in a weird state, we need to replay "what commands ran against it" without grepping shell history.
  • Ecosystem integration. Other tools (camp, obey-daemon, reporting dashboards) should be able to tail these files to surface activity without each tool reimplementing fest-cli introspection.

Scope

In Scope (v1)

  • New activity.jsonl files at both the festival and campaign levels.
  • A shared event-emission API inside fest that every state-mutating command calls.
  • Documentation in docs/ describing the schema and the full list of events.
  • Log-rotation policy (see "Open Questions").

Out of Scope

  • Shipping a query/replay CLI (fest activity log, fest activity replay). That can be a follow-up once the files exist. This issue is about emission.
  • Backfilling history from progress_events.jsonl or status_history.json.
  • Real-time event streaming (webhooks, watchers). Other tools can tail the JSONL.

Files

Festival-level — <festival>/.fest/activity.jsonl

Records every fest CLI action that mutates this specific festival's state. Co-exists with the current progress_events.jsonl; task status events MAY be mirrored into activity.jsonl (see Open Questions).

Campaign-level — .campaign/fest/activity.jsonl

Records every fest CLI action that affects festival existence or lifecycle at the campaign level. Example: a fest create festival event appears in the campaign log when the festival is born; a fest promote event appears when the festival moves from planning/ to ready/ or ready/ to active/.

Events logged at the campaign level are generally also logged at the festival level so each festival's history is self-contained, but the campaign log is the single scrollable timeline of "what happened in this campaign's festival layer over time."

Event Schema

JSONL: one JSON object per line, newline-terminated, append-only.

{
  "v": 1,
  "ts": "2026-04-12T19:42:11.123456789Z",
  "event": "phase.created",
  "actor": {
    "user": "lancekrogers",
    "host": "workstation.local",
    "fest_version": "0.14.2"
  },
  "scope": {
    "campaign_root": "/Users/lance/Dev/AI/obey-campaign",
    "festival_id": "CS0003",
    "festival_name": "corp-site-build",
    "festival_path_relative": "festivals/active/corp-site-build-CS0003",
    "phase": "002_PLAN_SITE_CONTENT",
    "sequence": null,
    "task": null
  },
  "source_cmd": "fest create phase --name PLAN_SITE_CONTENT --type planning",
  "data": {
    "phase_type": "planning",
    "phase_order": 2,
    "created_path": "festivals/active/corp-site-build-CS0003/002_PLAN_SITE_CONTENT"
  },
  "result": {
    "ok": true,
    "error": null
  }
}

Field conventions

Field Required Notes
v yes Schema version. Integer. Starts at 1. Bump on breaking changes.
ts yes RFC3339Nano UTC.
event yes Dotted namespace: <noun>.<verb>. See Event Catalog.
actor.user yes $USER at emission time.
actor.host yes Hostname.
actor.fest_version yes fest --version.
scope.campaign_root yes Absolute path.
scope.festival_id where applicable e.g. CS0003. Null for pure-campaign events that do not reference a festival.
scope.festival_name where applicable Human-readable festival name.
scope.festival_path_relative where applicable Path relative to campaign root, stable across status promotions via the path: field in fest.yaml.status_history.
scope.phase / sequence / task where applicable Null if not scoped to that level.
source_cmd yes Verbatim invoked command with arguments (redact secrets per the rules below).
data yes Event-specific payload. Schema per event in the catalog.
result.ok yes Boolean. Failed invocations SHOULD still be logged so the activity log includes errors.
result.error no Error message and category when ok: false.

Redaction rules

  • Never log full tokens, signing keys, or file contents.
  • source_cmd MUST have recognized sensitive flags replaced with <REDACTED> (e.g. --token, --password, --secret, --signing-key).
  • Arbitrary user messages (e.g. fest workflow skip --reason "...") are logged verbatim. This is expected.

Event Catalog (minimum required)

Every command below, where it currently exists in the CLI, MUST emit an event.

Festival lifecycle (emit at both campaign and festival level unless noted)

Event Triggered by
festival.created fest create festival
festival.deleted Any destructive removal path
festival.promoted Any status transition: planningreadyactivecompleted/archived/someday. data.from and data.to mandatory.
festival.linked / festival.unlinked fest link / fest unlink
festival.renamed Any rename path

Phase / sequence / task scaffolding (festival-level)

Event Triggered by
phase.created / phase.deleted fest create phase, destructive ops
phase.started / phase.completed Status transitions
sequence.created / sequence.deleted fest create sequence
sequence.started / sequence.completed Status transitions
task.created / task.deleted / task.renamed fest create task, destructive/rename ops
task.started / task.completed / task.blocked / task.reset Existing progress-tracking events (may be mirrored from progress_events.jsonl)
gate.applied fest gates apply
gate.skipped Any skip path with data.reason

Workflow / operations (festival-level)

Event Triggered by
validate.ran fest validatedata.ok, data.errors[], data.warnings[]
init.ran fest init or project-setup commands — data.steps[]
next.resolved fest nextdata.resolved_to (task path)
go.navigated fest go <target> — include resolved path. Navigation mutates no files but IS useful campaign telemetry. Festival-level only, not campaign-level, to keep campaign log signal-dense.
workflow.skipped fest workflow skip --reason "..."data.reason verbatim
commit.made fest commitdata.git_sha, data.message, data.scope
tui.action Any mutating TUI action. Mirror the underlying event namespace; do NOT emit a generic "tui interacted" event.

Read-only events

Pure read-only commands (fest status, fest show plan, fest list, fest understand) MUST NOT emit events. The log is an activity record, not a telemetry stream.

Implementation Notes

Single emission API

Add a package, e.g. internal/activity/, with:

type Emitter interface {
    Emit(ctx context.Context, ev Event) error
}

type Event struct {
    V          int
    Ts         time.Time
    Event      string
    Actor      Actor
    Scope      Scope
    SourceCmd  string
    Data       any
    Result     Result
}

Every state-mutating command must call the emitter on both its success and error paths. Treat this as a code-review gate: a PR that adds a new mutating command must also register an event.

Write semantics

  • Append-only, O_APPEND | O_CREATE.
  • Use fsync after every write; activity history matters more than throughput.
  • Writes MUST be durable before the command returns to the user, so a killed process cannot drop the event for work that already happened on disk.
  • Use an advisory file lock (flock / Go syscall.Flock) to prevent interleaved writes from concurrent fest processes.

Path resolution

  • Emit to the festival-level file based on the currently-scoped festival (see existing fest.yaml detection).
  • Emit to the campaign-level file based on the campaign root (walk up until .campaign/ is found, consistent with existing detection).
  • If the campaign root cannot be detected (edge case: fest used outside any campaign), skip the campaign-level emission with a WARN log and still emit the festival-level event.

Dual emission

The campaign log is a superset-by-lifecycle of the festival log for lifecycle events. Every festival.* event and every phase.started/completed / sequence.started/completed event emitted at the festival level is ALSO emitted to the campaign level. Granular scaffolding (task.created, validate.ran, next.resolved, etc.) is festival-only.

The exact split lives in a registry table inside internal/activity/catalog.go so new events declare their destination when they are added.

Log rotation

JSONL files can grow unbounded. Implementation:

  • Default: no rotation. JSONL is cheap and disk cost is negligible for almost every real campaign.
  • Safety hatch: if the file exceeds 50 MiB, rotate to activity.<N>.jsonl.gz and start fresh. Size threshold configurable via fest.yaml.

Per-file size is tiny compared to the rest of a campaign — rotation is a safety net, not a steady-state concern.

Interaction with existing progress_events.jsonl

  • Keep progress_events.jsonl as-is for backward compatibility.
  • The task-status events (task.completed, task.blocked, etc.) SHOULD be written to activity.jsonl in addition to progress_events.jsonl.
  • A follow-up issue can consolidate them once downstream consumers have migrated.

Documentation

Update docs/:

  • New doc: docs/activity_log.md — schema, event catalog, consumer examples (jq, tail, awk).
  • Update docs/architecture.md to describe the emitter package and the two-file layout.
  • Cross-reference from docs/lifecycle.md where status promotions are described.

Acceptance Criteria

  • New package internal/activity/ with an emitter and a tested event catalog.
  • Every existing state-mutating fest subcommand emits the appropriate event(s) on both success and error paths.
  • <festival>/.fest/activity.jsonl is created on the first mutating event for that festival and appended to thereafter.
  • .campaign/fest/activity.jsonl is created on the first mutating event in that campaign and appended to thereafter.
  • Schema version v: 1 frozen and documented.
  • Redaction rules enforced for known sensitive flags.
  • Integration tests cover:
    • festival creation emits on both files
    • phase / sequence / task creation emits on festival file only (not campaign)
    • promotion emits on both files with data.from/data.to
    • validate emits on festival file only
    • workflow skip emits with verbatim reason
    • read-only commands emit nothing
    • failing commands still emit with result.ok = false
  • File lock prevents interleaved writes (test via concurrent go test invocations).
  • docs/activity_log.md is the canonical reference.
  • Existing progress_events.jsonl continues to work for backward compatibility.

Open Questions

  1. Should progress_events.jsonl be deprecated? Current recommendation: keep for now, revisit in a follow-up once activity.jsonl has shipped and downstream consumers (camp, reporting) migrate.
  2. Should fest go be logged at all? Recommendation in this issue: yes, festival-level only, because it is useful for "where did I leave off" reconstructions. If noise is a problem we can drop it.
  3. Schema version bump policy. Proposed: additive fields do not bump; removed/renamed fields or changed semantics bump from v: 1 to v: 2. Document in docs/activity_log.md.
  4. git integration. Should commit.made include the git SHA even when committing across submodules? Proposal: yes, record all touched SHAs in data.git_shas[].

Related

  • Existing .fest/progress_events.jsonl — narrow task-status log this issue broadens.
  • Existing .fest/status_history.json — festival status snapshot; this issue adds chronological activity on top.
  • Existing .campaign/fest/navigation.yaml — unchanged by this issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions