---
title: "05a Agents / Transcripts / Sessionstream (Partition A)"
description: "05a Agents / Transcripts / Sessionstream (Partition A) Executive summary - Partition A covers two intertwined but separable architectures: (1) retrospective transcript analysis via go-minitrace, and (…"
doc_version: 1
last_updated: 2026-10-15
canonical_url: "http://parc.yolo.scapegoat.dev/note/ttmp/2026/06/22/project-maps-001-concept-maps-for-recent-project-topics/sources/05a-agents-transcripts-sessionstream"
---

# 05a Agents / Transcripts / Sessionstream (Partition A)

05a Agents / Transcripts / Sessionstream (Partition A) Executive summary - Partition A covers two intertwined but separable architectures: (1) retrospective transcript analysis via go-minitrace, and (…

## Content

# 05a Agents / Transcripts / Sessionstream (Partition A)

## Executive summary

- Partition A covers two intertwined but separable architectures: (1) **retrospective transcript analysis** via go-minitrace, and (2) **live streaming-agent observability** via Geppetto/Pinocchio/Sessionstream. The first-pass map already names this split; this condensed slice sharpens the nodes and edges.
- The transcript arc converged on a four-stage pipeline: native JSONL → canonical minitrace archive → normalized SQLite (`mt.db()`) → JS query repository → self-contained HTML report. DuckDB `UNNEST` over JSON arrays is a documented dead end; normalized SQLite is the canonical substrate.
- The streaming arc converged on a layered evidence chain: provider stream → Geppetto provider engine → chatapp runtime sink → sessionstream Hub projections → WebSocket transport → browser reducer → durable timeline. The strongest invariant is **provider-to-browser traceability** joined by typed `events.Correlation`.
- Both arcs share a design rule: **reusable packages emit neutral records; the application owns storage, debug APIs, and exports.** This is the same rule repeated at the minitrace query-safety authorizer and at the Pinocchio recorder/SQLite reconcile export.
- Canonical starting files: `Projects/2026/05/07/ARTICLE - Observer Instrumentation - Geppetto Pinocchio Sessionstream Deep Dive.md` (live observability) and `Projects/2026/06/22/ARTICLE - Analyzing Agent Tool-Calling Behavior with go-minitrace.md` (retrospective analysis).

## Scope and search method

- Corpus: Markdown project/article reports under `Projects/2026/{03,04,05,06}/` filtered to partition A of `sources/05` — namely "Transcript analysis and go-minitrace" and "Sessionstream, Pinocchio, Geppetto agent flows."
- Excluded (partition B): "Pi core and extensions", "LLM proxy/provider work and tool-calling behavior", "Dashboards, observability, readability/a14y."
- Selection rule: deeply read canonical architecture files (the two "start here" reports plus files defining invariants, schema, or hard cutovers); heading-scan or first-100-line reads for origin/cleanup/adjacent reports to confirm cross-links without duplicating detail.

## Evidence ledger

| Path | Evidence level | Lines / basis | Cluster | Why it matters |
|---|---|---|---|---|
| `Projects/2026/06/22/ARTICLE - Analyzing Agent Tool-Calling Behavior with go-minitrace.md` | read | lines 1-end | Transcript analysis | Canonical normalized-SQL pipeline; documents UNNEST dead end and `mt.db()` as the right substrate |
| `Projects/2026/05/07/ARTICLE - Observer Instrumentation - Geppetto Pinocchio Sessionstream Deep Dive.md` | read | lines 1-end | Live observability | Defines the neutral-record ownership model and provider-to-browser evidence chain |
| `Projects/2026/04/01/PROJ - go-minitrace - Web UI and Transcript Explorer.md` | read | lines 1-end | Transcript analysis | Web UI identity (session browser / transcript reader / SQL workbench); block decomposition |
| `Projects/2026/04/21/PROJ - go-minitrace - JS Commands and Structured Query Catalog PR #6.md` | read | lines 1-end | Transcript analysis | Scanner-first JS verb system; structured query catalog; nightly review workflow |
| `Projects/2026/06/07/ARTICLE - Minitrace API Redesign - From Prototype Complexity to Normalized SQL and Fluent Builders.md` | read | lines 1-end | Transcript analysis | Hard removal of legacy surfaces; nine-table normalized schema; `mt.db()` fluent builder |
| `Projects/2026/06/08/ARTICLE - SQLite Authorizer and Query Safety - Deep Dive Technical Analysis.md` | read | lines 1-end | Transcript analysis | Defense-in-depth query safety; regex table extraction failure mode; authorizer fix |
| `Projects/2026/05/09/ARTICLE - Canonical Chat Event Protocol - Provider Streams to Browser State.md` | read | lines 1-end | Live observability | Canonical event lifecycles; reducer-shaped provider adapters; sparse-patch contract |
| `Projects/2026/06/02/ARTICLE - Geppetto JS Overhaul - Wrapper First Agents Events and Storage Boundaries.md` | read | lines 1-end | Live observability | Wrapper-first API; hidden-ref enforcement; runAsync EventEmitter; storage boundary |
| `Projects/2026/05/20/ARTICLE - Pinocchio Structured Streams - Protobuf JSONL RPC and Chatapp TUI Migration.md` | read | lines 1-~1176 (50KB truncation) | Live observability | Protobuf JSONL RPC; TUI migration onto chatapp/sessionstream; debug-events-jsonl |
| `Projects/2026/06/14/ARTICLE - goja-sessionstream - Deep Dive into xgoja Sessionstream Integration.md` | read | lines 1-end | Live observability | xgoja binding for sessionstream; Promise-native APIs; Redis/Watermill host-owned distribution |
| `Projects/2026/03/17/PROJ - CoinVault - RAG Web Chat for Gold Coin Inventory.md` | read | lines 1-120 | Live observability | Concrete Pinocchio/Geppetto app; safe SQL tooling; projection widgets |
| `Projects/2026/03/17/PROJ - Claude Code Hook Events Logger - SQLite Analytics for Claude Sessions.md` | read | lines 1-100 | Transcript analysis | Capture-first telemetry origin; SQLite + views over hook events |
| `Projects/2026/04/29/PROJ - Sessionstream - Replay Store Remediation and Systemlab UI Refinement.md` | read | lines 1-120 | Live observability | Event cursor vs projection cursor split; SQLite hydration store |
| `Projects/2026/05/06/ARTICLE - Protobuf Payload Contracts and Sessionstream Schema Vet.md` | read | lines 1-120 | Live observability | Top-level `Struct` rejection; `sessionstream-lint` vettool; typed protobuf contracts |
| `Projects/2026/06/02/ARTICLE - Geppetto JS Session API - From Turns to Sessions.md` | read | lines 1-100 | Live observability | Turn→session API migration; Go-backed `AgentSession` invariants |
| `Projects/2026/05/22/ARTICLE - Sessionstream Runtime Events in Scraper.md` | read | lines 1-100 | Live observability | Sessionstream as generic substrate outside chat; domain-owned schema |
| `Projects/2026/06/22/ARTICLE - CozoDB Editor Modernization - Sessionstream Hard Cutover.md` | read | lines 1-100 | Live observability | Hard cutover pattern; notebook owns stream domain; sessionstream as substrate |
| `Projects/2026/05/04/ARTICLE - Sessionstream Chatapp CoinVault Cleanup - Protobuf Ordinals and Transcript Segments.md` | read | lines 1-100 | Live observability | Ordinal vocabulary; reusable ReasoningPlugin/ToolCallPlugin; widget schema separation |

## Condensed per-arc summaries

### Arc 1 — Transcript analysis and go-minitrace

- **Four-stage pipeline is now canonical**: native agent JSONL → `go-minitrace convert` → `.minitrace.json` archive (schema `minitrace-v0.2.0`) → `mt.db()` normalized SQLite (9–10 tables) → JS query repository → self-contained HTML report. DuckDB remains available but `UNNEST` over JSON arrays is a documented dead end inside the Goja path; normalized SQLite is the robust+ergonomic substrate (`Projects/2026/06/22` lines 27-34, 54-64; `Projects/2026/06/07` redesign).
- **Analysis is a repository of named verbs, not ad-hoc SQL.** Each question (tool frequency, transition matrix, retry loops, timing, token distribution) is a `__verb__`-declared JS command discovered by the scanner and exposed via `go-minitrace query commands`. SQL and JS sources coexist in one catalog with alias support; repository discovery precedence is `--query-repository` > env > config > embedded (`Projects/2026/04/21` PR #6).
- **Query safety is defense-in-depth.** Regex table extraction fails on quoted identifiers and CTE aliases; the SQLite authorizer (firing during `prepare` on parser-resolved names) is the correct primary check, layered with read-only prepared-statement check and text-based prefix validation (`Projects/2026/06/08`).
- **Self-contained HTML report is a deliberate artifact.** Python assembles the file (substitution + I/O); JavaScript renders client-side from inlined `DATA`/`STATS` JSON; `marked`+`DOMPurify` sanitize model markdown. Swiss minimalist typography (single red-orange accent) is a constraint, not decoration (`Projects/2026/06/22` Stage 4).
- **Timing analysis must derive from timestamps and state caveats.** The Pi adapter does not populate `output.duration_ms` or `exit_code`; inter-call gaps are upper bounds including model reasoning and user think-time. This data-quality rule shapes every timing query (`Projects/2026/06/22` warnings).

### Arc 2 — Sessionstream, Pinocchio, Geppetto agent flows

- **Neutral-record ownership model is the spine.** Sessionstream emits Hub pipeline + WebSocket transport observations; Geppetto emits provider + publish-boundary observations; Pinocchio owns the recorder, debug API, browser stream recorder, and SQLite reconcile export. Reusable packages must not learn about app storage (`Projects/2026/05/07` lines 25-34, 57-64).
- **Provider-to-browser traceability is the strongest invariant.** A provider reasoning delta can be related to a Geppetto record, a Sessionstream UI event, a frontend parsed frame, and a durable timeline entity. Provider IDs (`response_id`, `item_id`, `output_index`, `summary_index`) are carried into `ReasoningUpdate` protobuf and segment keys — sequence order is a fallback, not a contract (`Projects/2026/05/07` reasoning identity section).
- **Streaming chat is protocol design, not callback plumbing.** Canonical event lifecycles separate provider-call, text segment, reasoning segment, and tool call. Provider adapters are reducer-shaped (setup → consume → complete → persist). Sparse updates are patches, not replacements; `missing field ≠ clear value`. `events.Correlation` is first-class identity, not debug decoration (`Projects/2026/05/09`).
- **Protobuf contracts are enforced, not conventional.** Top-level `google.protobuf.Struct` is rejected by `sessionstream-lint` vettool; every command/event/UI-event/timeline-entity payload must be a concrete, feature-owned protobuf message. Pinocchio chatapp has reusable `ReasoningPlugin` and `ToolCallPlugin`; CoinVault owns only widget-specific schemas (`Projects/2026/05/06`; `Projects/2026/05/04`).
- **The substrate generalizes beyond chat.** Scraper runtime events and CozoDB notebook hints both use sessionstream as a generic typed-command/event/projection/hydration/websocket substrate, keeping domain schemas local. The hard-cutover pattern (delete legacy SEM/raw-forwarder paths, do not wrap) is the recurring migration discipline (`Projects/2026/05/22`; `Projects/2026/06/22` CozoDB).

## Candidate concept-map material

### Nodes

| Node | Type | Confidence | Notes |
|---|---|---|---|
| go-minitrace | project | high | Central transcript-analysis tool; converts native transcripts to canonical archives |
| Canonical minitrace archive | artifact | high | `.minitrace.json` schema `minitrace-v0.2.0`; stable top-level fields + `tool_calls` analytical center |
| Normalized transcript SQLite | concept | high | 9-10 tables via `mt.db()`; replaces DuckDB UNNEST and legacy `mt.legacy.*` |
| JS query repository | workflow | high | Scanner-first `__verb__`/`__section__` commands; coexists with sqleton SQL in one catalog |
| Self-contained HTML report | artifact | high | Inlined DATA/STATS JSON + CDN markdown libs; Python assembles, JS renders |
| go-minitrace web UI | platform | high | `go-minitrace serve`; session browser / transcript reader / SQL workbench; block decomposition |
| SQLite authorizer | concept | high | Compile-time table allowlist; fixes regex extraction failure mode |
| Tool-call churn analysis | workflow | high | Frequency/transition/retry-loop queries over normalized `tool_calls` table |
| Pi adapter duration_ms gap | failure-mode | high | Null `duration_ms`/`exit_code`; timing must derive from timestamps with caveats |
| UNNEST DuckDB fragility | failure-mode | high | Cast-inference errors in Goja path; documented dead end |
| Manifest advisory drift | failure-mode | medium | `manifest.json` can reflect only last `--source-session`; query file globs directly |
| Claude Code hook events logger | project | high | Capture-first telemetry origin; Python hook → SQLite + 6 views |
| Sessionstream Hub | concept | high | Generic event runtime; commands → backend events → UI/timeline projections → fanout |
| Sessionstream WebSocket transport | technology | high | Protobuf-defined frames; snapshot-before-live subscribe hydration |
| Geppetto provider engine | concept | high | Normalizes provider stream events; emits neutral observability records |
| Pinocchio chatapp | project | high | Web-chat runtime owning recorder, debug API, SQLite reconcile export |
| Pinocchio StreamDebugRecorder | artifact | high | Implements all three observer interfaces; bounded in-memory ring (default 10k) |
| Reconcile SQLite export | artifact | high | Joins backend pipeline + transport + Geppetto + frontend + timeline state |
| Canonical chat event protocol | concept | high | Provider-call/text/reasoning/tool lifecycles; `events.Correlation` identity |
| Reducer-shaped provider adapter | concept | high | setup → consume → complete → persist; per-provider native fixtures, shared lifecycle scenarios |
| Sparse-patch contract | concept | high | Missing field = absent update; empty meaningful field = provider sent empty |
| Protobuf payload contract | concept | high | Concrete feature-owned messages; `Struct` rejected at top level |
| sessionstream-lint vettool | technology | high | `go vet` analyzer enforcing typed payload registration |
| ReasoningPlugin / ToolCallPlugin | concept | high | Reusable chatapp plugins; projection of reasoning/tool lifecycle |
| Provider-to-browser traceability | concept | high | Invariant: provider delta → Geppetto record → UI event → frontend frame → timeline entity |
| Reasoning identity / provider IDs | concept | high | `response_id`/`item_id`/`output_index`/`summary_index` carried into `ReasoningUpdate` |
| Geppetto wrapper-first JS API | concept | high | Hidden-ref `__geppetto_ref`; wrappers not plain maps; explicit turns/sessions |
| runAsync EventEmitter | technology | high | Builder-level `.events(emitter)`; run-scoped refs; owner-thread settlement |
| Pinocchio protobuf JSONL RPC | technology | high | `RpcLine` protojson frames; `--rpc`/`--output jsonl`; `--debug-events-jsonl` |
| chatapp TUI migration | workflow | high | Bubble Tea over chatapp/sessionstream; removed raw Geppetto forwarders |
| goja-sessionstream xgoja binding | technology | high | `require("sessionstream")`; Promise-native; host-owned Redis/Watermill |
| Redis/Watermill event distribution | technology | medium | Host-owned; distributes events not commands; empty consumer group for fanout |
| Event cursor vs projection cursor | concept | high | Separate cursors diagnose "event stored but projection failed" |
| Sessionstream as generic substrate | concept | high | Scraper runtime events + CozoDB notebook hints reuse the substrate |
| Hard-cutover migration discipline | workflow | high | Delete legacy paths once typed substrate exists; do not wrap |
| Profile-to-browser reasoning duplication bug | failure-mode | medium | `reasoning-summary` created second thinking entity; fixed by segment-key identity |
| Subscribe hydration race | failure-mode | high | Old flow missed live events during snapshot load; fixed by hydrating-then-live ordering |
| CoinVault safe SQL tooling | project | high | PingCAP/pg_query_go parser validation; SELECT-only; LIMIT clamping |
| CoinVault projection widgets | artifact | high | `<gec:TYPE:v1>` markup → typed protobuf widget events |
| Should minitrace and sessionstream converge on one event schema? | open-question | medium | Live-vs-retrospective systems currently separate |
| Is `mt.db()` normalized SQLite the sole canonical analysis substrate? | open-question | medium | DuckDB still present but demoted for single-session analysis |

### Edges

```text
Native agent JSONL --converted by--> go-minitrace convert [high] (Projects/2026/06/22 Stage 1)
go-minitrace convert --emits--> Canonical minitrace archive [high] (Projects/2026/06/22 Stage 1)
Canonical minitrace archive --materialized by--> mt.db() normalized SQLite [high] (Projects/2026/06/22 Stage 2; Projects/2026/06/07)
mt.db() normalized SQLite --queried by--> JS query repository [high] (Projects/2026/04/21 PR #6; Projects/2026/06/22 Stage 3)
JS query repository --renders--> Self-contained HTML report [high] (Projects/2026/06/22 Stage 4)
UNNEST DuckDB fragility --superseded by--> mt.db() normalized SQLite [high] (Projects/2026/06/22 Approach 1 vs 3)
mt.db() normalized SQLite --protected by--> SQLite authorizer [high] (Projects/2026/06/08)
Pi adapter duration_ms gap --forces--> timestamp-derived timing with caveats [high] (Projects/2026/06/22 warnings)
Tool-call churn analysis --runs over--> mt.db() normalized SQLite [high] (Projects/2026/06/22 Stage 3 queries)
Claude Code hook events logger --precedes--> go-minitrace convert [medium] (Projects/2026/03/17 origin; Projects/2026/04/01 UI)
go-minitrace web UI --wraps--> JS query repository [high] (Projects/2026/04/01; Projects/2026/04/21)

LLM provider stream --decoded by--> Geppetto provider engine [high] (Projects/2026/05/07)
Geppetto provider engine --emits--> Geppetto observability Record [high] (Projects/2026/05/07)
Pinocchio chatapp --publishes--> Canonical chat event protocol events [high] (Projects/2026/05/09)
Canonical chat event protocol events --projected by--> Sessionstream Hub [high] (Projects/2026/05/07; 2026/05/04)
Sessionstream Hub --emits--> PipelineRecord [high] (Projects/2026/05/07)
Sessionstream WebSocket transport --emits--> TransportRecord [high] (Projects/2026/05/07)
Pinocchio StreamDebugRecorder --implements--> Geppetto + Sessionstream observer interfaces [high] (Projects/2026/05/07)
Pinocchio StreamDebugRecorder --exports--> Reconcile SQLite export [high] (Projects/2026/05/07)
Provider-to-browser traceability --joins--> Geppetto record + UI event + frontend frame + timeline entity [high] (Projects/2026/05/07)
Reasoning identity / provider IDs --carried into--> Pinocchio ReasoningUpdate protobuf [high] (Projects/2026/05/07)
Reducer-shaped provider adapter --normalizes--> Canonical chat event protocol events [high] (Projects/2026/05/09)
Sparse-patch contract --preserves--> previous accumulated state [high] (Projects/2026/05/09)
Protobuf payload contract --enforced by--> sessionstream-lint vettool [high] (Projects/2026/05/06)
Pinocchio chatapp --registers--> ReasoningPlugin / ToolCallPlugin [high] (Projects/2026/05/04)
Geppetto wrapper-first JS API --enforces--> hidden-ref wrappers not plain maps [high] (Projects/2026/06/02)
runAsync EventEmitter --settles on--> runtime owner thread [high] (Projects/2026/06/02)
Pinocchio protobuf JSONL RPC --routes through--> chatapp Runner + Sessionstream Hub [high] (Projects/2026/05/20)
chatapp TUI migration --removed--> raw Geppetto forwarder path [high] (Projects/2026/05/20)
goja-sessionstream xgoja binding --exposes--> Sessionstream Hub to JavaScript [high] (Projects/2026/06/14)
Redis/Watermill event distribution --injected by--> custom Go host via host services [high] (Projects/2026/06/14)
Event cursor vs projection cursor --diagnoses--> projection failure without losing events [high] (Projects/2026/04/29)
Sessionstream as generic substrate --reused by--> Scraper runtime events [high] (Projects/2026/05/22)
Sessionstream as generic substrate --reused by--> CozoDB notebook hints [high] (Projects/2026/06/22 CozoDB)
Hard-cutover migration discipline --deletes--> legacy SEM / raw-forwarder paths [high] (Projects/2026/06/22 CozoDB; 2026/05/20)
CoinVault safe SQL tooling --validates--> SELECT-only via PingCAP parser [high] (Projects/2026/03/17 CoinVault)
CoinVault projection widgets --parsed from--> <gec:TYPE:v1> markup in assistant text [high] (Projects/2026/03/17 CoinVault)
Subscribe hydration race --fixed by--> hydrating-then-live ordinal ordering [high] (Projects/2026/05/07)
Profile-to-browser reasoning duplication bug --fixed by--> segment-key identity [medium] (Projects/2026/05/07)
```

## Cross-links to other topic slices

- **Topic 02 (JavaScript/Goja/xgoja/DSLs):** go-minitrace JS query commands and the `mt.db()` fluent builder follow the `goja-text` wrapper-first builder pattern. Geppetto's `require("geppetto")` and `require("sessionstream")` are xgoja provider-registered modules; `goja-sessionstream` is the canonical xgoja integration example. The hidden-ref `__geppetto_ref` enforcement and `Validate()`/`Build()` lifecycle are shared DSL rules.
- **Topic 06 (Data/RAG/OCR/search):** CoinVault is a RAG web chat over a MySQL inventory database with safe SQL tooling — a data system with a chat surface. Transcript-driven design-system recovery (`Projects/2026/06/07`) uses DuckDB/SQLite/git over converted Pi sessions. SQLite canonical store recurs across minitrace, reconcile export, and CoinVault turn/timeline persistence.
- **Topic 07 (Web UI/apps/media/productivity):** React/Redux timeline reducers, WebSocket chat overlays, and the single-binary Go+SPA pattern (`go-minitrace serve`, Pinocchio web-chat, CozoDB editor) are shared. Self-contained HTML reports and `--debug-events-jsonl` traces are agent-readable/portable artifacts. Bubble Tea TUI migration is a terminal-app-shell concern.
- **Topic 04 (Infra/auth/deployment/gitops):** SQLite canonical store and query safety (`go-go-host/internal/sitejs/dbguard` first-token classification is referenced as a sibling but not importable by go-minitrace). The authorizer analysis explicitly notes `dbguard` lives under a different module's `internal/`.
- **Topic 01 (Hardware/embedded/ESP32):** Applied transcript analysis was used to trace the Loupedeck serial bug (`Projects/2026/04/22` — title-only in this slice, referenced by parent report).
- **Topic 03 (Typography/design-systems):** Transcript-driven design-system recovery (`Projects/2026/06/07`) bridges transcript analysis and design-system knowledge extraction.

## Start here

1. `Projects/2026/05/07/ARTICLE - Observer Instrumentation - Geppetto Pinocchio Sessionstream Deep Dive.md` — defines the live streaming-agent observability architecture, the neutral-record ownership model, and the provider-to-browser evidence chain. Read the summary, the end-to-end path diagram, and the "Reading the artifact" section.
2. `Projects/2026/06/22/ARTICLE - Analyzing Agent Tool-Calling Behavior with go-minitrace.md` — defines the retrospective transcript-analysis pipeline, the `mt.db()` normalized SQLite substrate, and the JS query repository pattern. Read the pipeline-at-a-glance diagram and Stage 2 (three ways to query).

## Report-format notes

- The two arcs are best kept as separate map regions with a bridge edge `Provider-to-browser traceability --diagnosed retrospectively by--> go-minitrace convert` because provider replay bugs and tool-call churn feed back into transcript analysis.
- The `events.Correlation` identity model and the SQLite canonical store are the strongest bridge concepts to other topic slices.
- Failure modes (UNNEST fragility, duration_ms gap, subscribe hydration race, reasoning duplication, `Struct` payload drift) should be first-class nodes, not footnotes — they explain why later projects changed direction.
- Several Sessionstream files are very long (the Pinocchio Structured Streams report exceeded the 50KB read limit); a second pass on the TUI turns-persistence design section would strengthen the persistence boundary node.

## Sitemap

- [Home](http://parc.yolo.scapegoat.dev/)
- [Home markdown mirror](http://parc.yolo.scapegoat.dev/index.md)
- [Markdown sitemap](http://parc.yolo.scapegoat.dev/sitemap.md)