explore-developer-ideas-from-tracked-researchers: monitor researcher feeds and triage into roadmap

[rysweet]

Goal

Advance the explore-developer-ideas-from-tracked-researchers goal by establishing a tracked, auditable workflow for monitoring a curated set of researchers/developers and triaging promising ideas into the Simard roadmap.

This issue exists so subsequent cycles have a concrete artifact to spawn engineers against (currently the goal has no backing issue, which is why it sits at 0%).

Scope

1. Define the tracked-researcher list. Create docs/research/tracked-researchers.md containing:

   • Name, primary affiliation, and 1–2 sentence rationale for tracking.
   • Canonical feed URL(s): GitHub profile, blog/RSS, arXiv author page, X/Mastodon handle, Substack, etc.
   • Topic tags relevant to Simard (e.g. agentic-loops, self-improvement, evals, memory, multi-agent, tool-use).
   • Seed the list with at minimum: Andrej Karpathy, Simon Willison, Nathan Lambert (Interconnects), Lilian Weng, Jason Wei, Riley Goodside, Shunyu Yao (ReAct/SWE-agent), the METR team, the Princeton NLP group, and the Anthropic + DeepMind public research blogs. Maintainers may add/remove.

2. Ingestion mechanism. Add a small Rust binary or simard subcommand (e.g. simard research scan) that:

   • Reads docs/research/tracked-researchers.md (or a sibling tracked-researchers.toml if structured data is preferred).
   • Pulls recent items (last 14 days) from each feed via existing HTTP client utilities — no new heavy deps.
   • Emits a JSONL digest to ~/.simard/research/digest-<date>.jsonl with {source, author, title, url, published_at, tags}.
   • Fails loud on feed errors (no silent skip — per the repo's no-fallback rule).

3. Triage loop. Document in docs/research/triage.md how Simard cycles will:

   • Read the latest digest.
   • For each item judged promising, open a roadmap: issue with label enhancement summarizing the idea, the source link, and a proposed Simard experiment.
   • Close/ignore irrelevant items with a one-line rationale logged to ~/.simard/research/triage-<date>.log.

4. Roadmap integration. Cross-link accepted items into docs/roadmap.md (create if missing) under a Research-sourced ideas section so they are visible alongside engineering work.

Acceptance criteria

• docs/research/tracked-researchers.md exists with the seed list and schema documented.
• simard research scan (or equivalent) runs end-to-end against the seed list and produces a digest file; failures surface as non-zero exit + stderr.
• docs/research/triage.md describes the cycle-by-cycle triage protocol.
• docs/roadmap.md exists with a Research-sourced ideas section (may start empty).
• At least one digest run is committed as a sample under docs/research/samples/ so future cycles can validate the format.

Out of scope

• Any LLM-based summarization of feed items (can be a follow-up issue).
• Slack/Discord delivery of digests.
• Automated PR generation from research items — triage stays human/Simard-cycle driven.

Notes for the next cycle

Once this issue is filed, the next Simard cycle should spawn_engineer against it with a bounded first step: scaffold docs/research/tracked-researchers.md with the seed list and the documented schema, then cargo check --lib to confirm no accidental code changes broke the tree.

[rysweet]

Tracked-researcher digest (manual scan, last ~7 days: 2026-04-17 → 2026-04-24)

Context: Issue #1195 defines the seed researcher list (Karpathy, Willison, Lambert, Weng, Wei, Goodside, Yao, METR, Princeton NLP, Anthropic, DeepMind) but the docs/research/tracked-researchers.md file and simard research scan tooling have not yet been implemented. This comment is a one-off manual triage covering canonical feeds for the seed authors, scoped to the last week, to give the next cycle concrete candidates.

Sources scanned this pass: Simon Willison Atom feed, Interconnects RSS (Nathan Lambert), Lil'Log RSS (Lilian Weng — no new items in window), Anthropic news index. Karpathy / METR / DeepMind / Princeton NLP / Wei / Goodside / Yao had no new long-form items in the 7-day window from their canonical channels (or only social posts, which the issue explicitly excludes from the schema until tooling lands).

Candidate ideas

1. Pub/sub primitives inside SQLite — Simon Willison, 2026-04-24
   Source: https://simonwillison.net/2026/Apr/24/honker/
   Relevance: russellromney/honker is a Rust SQLite extension implementing Postgres-style NOTIFY/LISTEN, durable queues, and Kafka-style streams. Simard already persists agent state in SQLite-shaped stores; an in-process notify/stream layer could replace ad-hoc polling in the OODA loop and cycle-coordination paths without adding a broker dependency.

2. Open-vs-closed benchmark dynamics & the limits of single-number evals — Nathan Lambert (Interconnects), 2026-04-20
   Source: https://www.interconnects.ai/p/reading-todays-open-closed-performance
   Relevance: Directly applicable to Simard's self-improvement scoring. Argues that composite indices (e.g., Artificial Analysis Intelligence Index) increasingly diverge from real agentic performance and that RLVR-style environment evaluations are the right correlate. Suggests we should keep prioritizing environment-grounded improvement signals over scalar deficit scores in src/self_improve/prioritization.rs.

3. Claude Design (collaborative artifact generation) — Anthropic Labs, 2026-04-17
   Source: https://www.anthropic.com/news/claude-design
   Relevance: Demonstrates an agent-collaboration UX for producing structured artifacts (slides, one-pagers, prototypes). A useful reference pattern for Simard's meeting/facilitator output surfaces — particularly for how to expose intermediate, editable artifacts mid-loop rather than only final reports.

4. Agentic Engineering Patterns (chapter drop in weekly) — Simon Willison, 2026-04-24
   Source: https://simonw.substack.com/p/gpt-55-chatgpt-images-20-qwen36-27b
   Relevance: New chapter in Willison's running catalog of working agentic patterns. Worth mining for concrete loop/tool-use idioms that the engineer-loop and orchestrator recipes could absorb (especially anything on tool gating, retry shape, or prompt scoping).

5. Project Glasswing — multi-vendor critical-software security coalition — Anthropic, 2026-04-07 (just outside the 7-day window, but the most recent supply-chain item)
   Source: https://www.anthropic.com/news/project-glasswing
   Relevance: Background signal for Simard's tool-use sandbox and shell-allowlist decisions. As major vendors converge on shared hardening primitives, our shell allowlist (cargo/git/gh/rustfmt/clippy + read-only inspection tools) and engineer-loop sandbox should track that direction rather than diverge.

Recommended next steps (for the next Simard cycle)

• Land the scaffolding from explore-developer-ideas-from-tracked-researchers: monitor researcher feeds and triage into roadmap #1195 (the docs/research/tracked-researchers.md schema and simard research scan binary) so future digests are reproducible rather than hand-curated.
• File roadmap: issues only for items 1, 2, and 4 — they have direct, near-term implications for already-shipping subsystems (persistence, prioritization, engineer-loop). Items 3 and 5 are watch-list signals; log to triage and re-evaluate next cycle.

Generated manually from canonical feeds; no code or repo files were modified by this scan.

[rysweet]

Tracked researcher activity — past 14 days (2026-04-10 → 2026-04-24)

Source: gh api /users/karpathy/events/public and gh api /users/simonw/events/public, cross-checked against per-repo commit APIs.

@karpathy

Quiet fortnight. Two repos touched, no new repos, no releases.

• karpathy/nanochat — merged two community PRs:
  • #706 fix/cpu — adds setuptools so the repo runs on CPU-only boxes; uv.lock refresh.
  • #686 Initialize smear and backout lambdas in init_weights() — switches to nn.init and explicitly initializes the smear gate weight. Karpathy's own follow-up commit just adds an explanatory comment.
• karpathy/karpathy.github.io — two cosmetic edits adding/tweaking the "microgpt triptych" link.

Roadmap signal for Simard: low. Both changes are ergonomics/correctness fixes. The smear/backout gate-init pattern is a useful reminder that any architecture that adds learnable scalar gates needs an explicit init story — worth checking our own self-improvement scoring layers if we ever introduce gating.

@simonw

Very active. Two brand-new repos, one alpha release, ongoing work on liteparse.

New repos

• simonw/llm-openai-via-codex (created 2026-04-23, Python, 23★ in <24h). Plugin that lets the llm CLI hit OpenAI models by piggy-backing on an existing Codex subscription instead of an API key. Released as 0.1a0. Commits: Initial working plugin → pip install . --group dev → Link to blog.
• simonw/monthly-newsletter-archive (created 2026-04-24, Shell). Sponsors-only newsletter mirror, one-month-delayed. Tooling/process repo, no engineering content yet.

Notable ongoing work

• simonw/liteparse (web branch) — adds a browser-side parsing harness with a Playwright test suite: tests 7-12 cover Tesseract OCR path on scanned PDFs, OCR toggle regression guard, parser-error guards, reparsing-clears-output, Safari ReadableStream polyfill + screenshots, mobile viewport regression. Also CI caching for Playwright browsers and Vite deps. Companion PR run-llama/liteparse#148 opened upstream.
• simonw/tools — new milliseconds.html utility; iterations on blog-to-newsletter (drag-drop improved by "GPT-5.5", TIL dedup, sponsorship promo at end of newsletter); Remove --size validation in openai_image.py (feat: Meeting session persistence — periodic auto-save and crash resume #273).
• simonw/llm-prices — routine pricing data refreshes.
• simonw/claude-code-transcripts — issue activity (no commits in window).

Roadmap signals for Simard

1. llm-openai-via-codex pattern is directly relevant. It demonstrates a working approach to "use the user's existing chat-product subscription as a model gateway" — exactly the kind of provider-adapter shape Simard's RuntimeConfig / provider abstraction (src/runtime_config.rs, src/session_builder.rs) already accommodates. Worth a 30-min read to see how Simon handles auth-token capture, request/response translation, and rate-limit failure modes; we may want a similar BaseTypeAdapter for Codex-routed traffic.
2. liteparse browser-side parsing + Playwright regression matrix. If Simard ever surfaces session transcripts or meeting artifacts in a web UI, Simon's test-numbered approach (each Playwright test guards a specific historical regression) is a good template for our own regression discipline — it's the same philosophy as our tests_*.rs files keeping a paper trail per fix.
3. blog-to-newsletter "had GPT-5.5 improve the drag-drop" is a nice example of an iterative human-in-the-loop refactor where the commit message names the model used. Cheap to adopt: when we run self-improvement OODA cycles that touch UX surfaces, recording the model in the commit trailer would help future analysis.
4. No karpathy moonshot to chase this cycle. Reassess in two weeks.

Methodology

• Window: events with created_at >= 2026-04-10T00:00:00Z from each user's public events feed (capped at 30 by the API).
• Per-repo commit detail pulled via /repos/{owner}/{repo}/commits?since=... to recover messages that aren't included in PushEvent payloads.
• "New repo" defined as a CreateEvent with ref_type=branch and ref=main|master on a freshly-created repo (verified via /repos/{owner}/{repo} created_at).

[rysweet]

Tracked researcher activity — past 30 days (2026-03-25 → 2026-04-24)

Sources: gh api users/<handle>/events/public --paginate for karpathy, simonw, natolambert, jasonwei20, lilianweng, goodside, ysymyth (the last three had no public GitHub events in the window). Window starts 2026-03-25T00:00Z; CreateEvent/ReleaseEvent cross-checked against repo metadata.

Top 5 ideas to triage

1. Nathan Lambert — RLHF book Code v0.2: Rejection Sampling + SAPO + RM improvements (natolambert/rlhf-book release 2026-04-12, plus ~80 follow-up PRs through 2026-04-21).
   Simard relevance: SAPO and the reward-model upgrades are a ready reference implementation for the scoring/reward shape we need in the OODA self-improvement loop (src/self_improve/prioritization.rs). Worth comparing his rejection-sampling protocol to our PrioritizedDimension trend amplifier.

2. Simon Willison — llm-openai-via-codex 0.1a0 (new repo 2026-04-23, released same day). Plugin that routes the llm CLI to OpenAI models by reusing an active Codex subscription instead of an API key.
   Simard relevance: Direct template for a BaseTypeAdapter that piggy-backs on an existing chat-product session — same provider-abstraction shape Simard already exposes via RuntimeConfig / SessionBuilder. Read the auth-token capture + request translation; potential follow-up issue to add a Codex-routed provider.

3. Simon Willison — Datasette 1.0a27 / 1.0a28 with Sec-Fetch-Site CSRF replacement (simonw/datasette releases 2026-04-15 and 2026-04-17; tracking issue "Replace token-based CSRF with Sec-Fetch-Site header protection").
   Simard relevance: If/when Simard exposes any HTTP surface (research-digest viewer, meeting UI), the Sec-Fetch-Site pattern is a much cheaper CSRF defence than per-request tokens and aligns with our "no fallback, fail loud" rule (browsers that don't send the header are rejected outright).

4. Simon Willison — llm-anthropic 0.25 (released 2026-04-16). Refreshed Anthropic provider plugin for the llm CLI ecosystem.
   Simard relevance: Worth diffing against src/base_type_copilot.rs for any new tool-use / interleaved-thinking surface area that the upstream Anthropic SDK exposes — historically Simon's plugin tracks new Anthropic features within days.

5. Karpathy — nanochat smear/backout gate init fix (karpathy/nanochat PR test-only #686 merged 2026-04-13).
   Simard relevance: Reminder that any architecture introducing learnable scalar gates needs an explicit nn.init story. Low immediate priority — flag for the day Simard's self-improvement layers add gating coefficients.

Lower-signal items (logged, not promoted)

• Karpathy: otherwise quiet; only nanochat ergonomics PRs and two karpathy.github.io cosmetic edits. No new repos, no autoresearch activity beyond closing one notebook bug (2026-03-26).
• Simon Willison — simonw/research (new repo 2026-04-13). Personal scratch-pad style; nothing actionable yet.
• Simon Willison — simonw/tools flurry (8 new sub-tools 2026-04-13 → 2026-04-21, e.g. milliseconds.html, blog-to-newsletter iterations). Useful as a portfolio of "GPT-assisted small utilities" but no Simard hook.
• Simon Willison — simonw/liteparse (web branch) + Initial prototype of browser-based LiteParse run-llama/liteparse#148. Playwright regression-per-test pattern is a nice template if we ever need browser-side parsing tests.
• Jason Wei (jasonwei20.github.io) — 10 site pushes 2026-04-11 → 2026-04-15. Posts not introspectable via the events API; recommend a one-shot fetch of https://jasonwei20.github.io/ next cycle to capture any new essays.
• Lilian Weng / Riley Goodside / Shunyu Yao — zero public GitHub events in the window. Their primary feeds (lilianweng.github.io, X/Twitter, arXiv) need a non-GitHub fetcher; tracked for the simard research scan ingestion design (issue body §2).

Methodology notes & follow-ups

• The seed list in this issue includes blogs (Anthropic, DeepMind, Interconnects), arXiv author pages, and X/Mastodon handles that the GitHub events API cannot reach. The next iteration of the digest tool needs a generic feed fetcher (RSS/Atom + arXiv API + optional Mastodon JSON) before the triage criteria can be applied uniformly across all tracked researchers.
• Recommend opening follow-up roadmap: issues for items Implement the first high-quality version of Simard in /home/azureuser/src/Simard based on the completed spec reviews. This is a development task and must use default-workflow. Required outcomes: (1) u #1 (SAPO comparison), Fix the confirmed Simard audit findings in /home/azureuser/src/Simard, using this clarified policy: explicit defaults are allowed and may be desirable for bootstrap/tests; what is forbidden is silent  #2 (Codex-routed provider adapter), and Fix the confirmed Simard audit findings in /home/azureuser/src/Simard using the clarified policy: explicit defaults are allowed, but silent fallback after failure and hidden degradation are not. Confi #3 (Sec-Fetch-Site CSRF) per the triage protocol described in the issue body.

[rysweet]

Tracked-researcher triage — past 30 days (2026-03-25 → 2026-04-24)

Source of truth scanned: src/research_tracker/watches.rs:14-39 (DEFAULT_DEVELOPER_WATCHES = ramparte, simonw, steveyegge, bkrabach, robotdad). Activity pulled via gh api users/<handle>/events/public, then cross-referenced against the originating repos / PRs. This complements the prior triage comment, which used the issue-body seed list (Karpathy, Lambert, …) rather than the in-code watch list.

Top 5 ideas to triage

1. bkrabach — SessionConfigurator: live session manipulation library (microsoft/amplifier-foundation #160, merged 2026-04-16; cleanup PR #174).
   Simard relevance: directly parallels Simard's SessionBuilder / RuntimeConfig (src/runtime_config.rs, src/session_builder.rs, PR fix: kill Tier-1 fallbacks — end the fake-success OODA loop (#1062) #1141). Worth diffing against our builder to see whether mid-session reconfiguration (provider swap, tool-set toggle, memory-policy change) is something we should expose without tearing down the session — currently Simard requires a rebuild. Roadmap mapping: candidate roadmap: issue under Runtime / session ergonomics.

2. bkrabach — feat/work-estimation-protocol branch (microsoft/amplifier-foundation, branch created 2026-04-24 — https://github.com/microsoft/amplifier-foundation/tree/feat/work-estimation-protocol).
   Simard relevance: Simard's OODA loop already prioritises dimensions via PrioritizedDimension (src/self_improve/prioritization.rs:198, trend amplifier 5×; plateau threshold 0.08), but it has no explicit work-cost estimation feeding the selector — engineer plans are picked on deficit, not on expected effort. An upstream protocol for estimating step cost is exactly the missing input for suggest_next_target_excluding. Roadmap mapping: OODA self-improvement → cost-aware target selection.

3. robotdad — amplifier-module-provider-openai-chatgpt (new public repo 2026-04-22, https://github.com/robotdad/amplifier-module-provider-openai-chatgpt — "raw httpx + manual SSE against chatgpt.com/backend-api"); paired with amplifier-module-provider-openai feat/oauth-subscription-auth branch (12 pushes 2026-04-22).
   Simard relevance: a working reference for routing through a ChatGPT subscription (not API key) via captured OAuth tokens + manual SSE. Direct template for a third BaseTypeAdapter alongside base_type_copilot.rs, and a near-twin of Simon Willison's llm-openai-via-codex flagged in the prior triage — two independent signals pointing at the same provider-abstraction gap. Roadmap mapping: Provider adapters → ChatGPT-subscription provider (file as roadmap: issue, link both refs).

4. simonw — monthly-newsletter-archive (new repo created 2026-04-24, https://github.com/simonw/monthly-newsletter-archive: "sponsors-only monthly newsletter on a one-month delay").
   Simard relevance: operational pattern, not a code drop. The "publish on a delay, in a separate repo, automated from a private source" workflow is a clean template for how Simard could publish research digests (issue-body §2 calls for ~/.simard/research/digest-<date>.jsonl) without leaking pre-triage notes. Roadmap mapping: Research digest publication pipeline.

5. ramparte — neolink battery-aware-backoff branch (created 2026-04-17, https://github.com/ramparte/neolink/tree/battery-aware-backoff; CI hardening commits up through 2026-04-17).
   Simard relevance: lower priority but instructive — the branch implements adaptive polling backoff under resource pressure. Simard's research_tracker::activity_polling currently has no backoff story; if/when we turn simard research scan (issue body §2) into a recurring job, an analogous "fail loud, then back off" loop avoids hammering GitHub's events API. Roadmap mapping: Research-tracker ingestion → adaptive polling.

Lower-signal items (logged, not promoted)

• simonw — heavy simonw/tools and simonw/llm-prices churn 2026-04-23 → 2026-04-24 (small utility additions, price-table updates). No Simard hook.
• simonw — run-llama/liteparse issues + PR activity 2026-04-23/24. Watch as a pattern source for browser-side parser regressions, no immediate action.
• steveyegge — exclusively gastownhall/gastown (private-org-style) churn via the polecat/* PR-bot branch convention. No public-research signal in the window.
• bkrabach — large DOT-as-Docs PR sequence (Fix the meeting repl UX and add meeting memory persistence. TWO issues: (1) The meeting repl treats all input as commands and shows Unknown command for natural language. Fix: any input NOT starting wi #145–Create src/session_builder.rs that unifies session creation across all modes. Extract the duplicated BaseTypeSessionRequest + RustyClawdAdapter factory pattern from operator_commands_meeting.rs open_m #149) early April; relevant to docs tooling but already overlaps Simard's existing mkdocs setup, not a new direction.
• robotdad — spark-tools push 2026-04-23, microsoft/amplifier issues #?? (operational, not idea-shaped).

Methodology notes & follow-ups

• The in-code seed (DEFAULT_DEVELOPER_WATCHES) is disjoint from the seed list documented in this issue's body (Karpathy, Lambert, Weng, …). That divergence is itself a triage item: either reconcile watches.rs with docs/research/tracked-researchers.md (still TODO per the acceptance criteria) or document why the in-code list is intentionally narrower (Amplifier-ecosystem focus vs. general LLM-research focus).
• All five tracked handles produce GitHub-visible activity, so the existing activity_polling path is sufficient for this watch list — the broader RSS/arXiv/Mastodon fetcher noted in the prior triage is only required if the issue-body seed list is adopted.
• Recommend opening follow-up roadmap: / enhancement issues for items Implement the first high-quality version of Simard in /home/azureuser/src/Simard based on the completed spec reviews. This is a development task and must use default-workflow. Required outcomes: (1) u #1 (SessionConfigurator parity), Fix the confirmed Simard audit findings in /home/azureuser/src/Simard, using this clarified policy: explicit defaults are allowed and may be desirable for bootstrap/tests; what is forbidden is silent  #2 (work-estimation input to OODA), and Fix the confirmed Simard audit findings in /home/azureuser/src/Simard using the clarified policy: explicit defaults are allowed, but silent fallback after failure and hidden degradation are not. Confi #3 (ChatGPT-subscription provider adapter) — these are the three with concrete Simard touchpoints already in-tree.

[rysweet]

Cycle update (2026-04-24): Kicking off researcher-feed triage. Next concrete step is to scaffold a tracked-researchers source list and a lightweight ingestion script under scripts/researcher_feed/ so subsequent cycles can run it and triage outputs into roadmap issues.

Proposed acceptance criteria for the first slice:

• scripts/researcher_feed/sources.toml lists tracked researchers (name, handle, feed URL, focus area).
• scripts/researcher_feed/fetch.py (or shell equivalent) pulls recent posts/papers per source and writes JSONL to scripts/researcher_feed/out/.
• README section in scripts/researcher_feed/README.md documents how to run it and how triaged ideas become GitHub issues labeled enhancement.

Next cycle will spawn an engineer against this issue to implement the scaffold.

[rysweet]

Researcher-feed triage — 2026-04-24

Surveyed recent public activity from tracked researchers (simonw, karpathy, jxnl) via GitHub events API. Three promising ideas surfaced for the Simard roadmap:

1. karpathy/autoresearch — agent-driven training experiments

• Source: https://github.com/karpathy/autoresearch ("AI agents running research on single-GPU nanochat training automatically", updated 2026-04-24)
• Why it matters for Simard: Direct conceptual sibling to our self_improve OODA loop — agents autonomously proposing, running, and evaluating experiments against a fixed substrate (nanochat training). Worth studying their experiment-proposal schema, scoring rubric, and how they avoid mode-collapse / repeated experiments. Could inform prioritization.rs rotation logic and engineer_loop plan generation.
• Proposed Simard experiment: Read autoresearch's loop structure and write a short docs/research/notes/autoresearch-comparison.md mapping their primitives onto our OODA stages; identify any missing rotation or novelty-bonus mechanism we lack.

2. simonw/tools — "vibe-built" single-purpose disposable utilities

• Source: https://github.com/simonw/tools (active April 2026, e.g. blog-to-newsletter, milliseconds.html)
• Why it matters for Simard: Simon's pattern is to LLM-generate one-file HTML/JS utilities and auto-publish docs alongside. Mirrors a possible simard tool scaffold <prompt> subcommand for cycle-generated throwaway helpers (e.g. quick log-grep dashboards, telemetry visualizers). Also note PR Refactor size parameter handling in OpenAI image CLI simonw/tools#273 — "treat --size like -m/--model: pass user value through, list known values in help but don't enforce" — aligns with our no-fallback / fail-loud philosophy: validate where wrong values cause silent corruption, hint elsewhere.
• Proposed Simard experiment: File a follow-up issue exploring a minimal simard scaffold recipe that emits self-contained Rust binaries from a prompt + smoke test.

3. jxnl/instructor-go PR #61 — empty-JSON detection with actionable error

• Source: jxnl/instructor-go@f75374e
• Why it matters for Simard: When an LLM returns empty content under ModeJSON, instructor now raises a specific error suggesting ModeJSONSchema or ModeToolCall. Directly applicable to base_type_copilot.rs::BaseTypeOutcome parsing — today an empty execution_summary propagates as a vacuous success. Per repo memory ("BaseTypeOutcome.execution_summary is the actual LLM response"), we should surface "LLM returned empty structured output — retry with strict schema or tool-call mode" rather than continuing with empty plan text.
• Proposed Simard experiment: Add a guard in the adapter that errors on empty execution_summary, with a hint pointing at the structured-output mode toggle. Cite no-fallback rule.

Out of scope this pass

• Karpathy/nanochat itself (training-substrate work, not agent-loop tooling).
• jxnl personal repos (dots, tao-te-ching, book-of-disquiet) — not dev-tooling signal.
• simonw/llm-prices and monthly-newsletter-archive (data refreshes, not ideas).

Next action

Once docs/research/tracked-researchers.md lands per this issue's scope, fold these three sources in with tags agentic-loops, self-improvement, structured-output, and dev-tooling. The autoresearch comparison note is the highest-leverage immediate follow-up.

— Posted by engineer cycle explore-developer-ideas-from-tracked-researchers-1777012701-d423ab.

[rysweet]

Researcher triage — last 30 days (2026-03-25 → 2026-04-24)

Automated scan of public output from a curated list of researchers/practitioners whose work overlaps Simard's roadmap (agentic coding loops, LLM orchestration, self‑improving systems). GitHub activity sourced via gh api users/<u>/events/public; blog/feed via curl; arXiv via the export API.

Tracked: Andrej Karpathy (@karpathy), Georgi Gerganov (@ggerganov), Jason Liu (@jxnl), Simon Willison (@simonw), Harrison Chase (@hwchase17).

Note: jxnl and hwchase17 had no public GitHub events in the 30‑day window — they appear to be working in private repos or on platforms outside GitHub. Recommend supplementing with their respective Substacks / LangChain blog before the next triage.

---

1. Andrej Karpathy — karpathy

Activity centred on karpathy/nanochat and a new repo karpathy/autoresearch.

• Idea: "autoresearch" loop — Karpathy's new public repo experiments with LLM‑driven research notebook execution. Relevant to Simard's OODA / engineer loop because it explores how to keep an LLM grounded inside an executable analysis workflow with explicit best/checkpoint state. Worth lifting the design pattern of named scoreboard variables for our prioritization.rs rotation logic. → Bug: 'best' variable used before definition in analysis.ipynb karpathy/autoresearch#262
• Idea: nanochat MMLU split correctness — A flurry of small fixes around dataset split naming (auxiliary_train vs train) is a reminder that eval‑harness fragility silently corrupts self‑improvement signals. We should add a sanity check in self_improve/observation that asserts dataset split identity before recording velocity. → MMLU main split is named auxiliary_train, not train karpathy/nanochat#639

2. Georgi Gerganov — ggerganov / ggml-org/llama.cpp

Heavy week of bugfixes around structured output parsing and server context limits.

• Idea: structured‑output parser hardening — PR #22302 "parser: fix structured output bug" + #21328 "json‑schema‑file loaded but is ignored". llama.cpp is converging on a robust grammar/JSON‑schema execution path. Our base_type_copilot.rs adapter telemetry path could borrow the test fixtures here to harden how we deserialise .execution_summary payloads. → parser: fix structured output bug ggml-org/llama.cpp#22302 , Eval bug: json-schema-file loaded but is ignore ggml-org/llama.cpp#21328
• Idea: RoPE / context‑length escape hatch — Discussion in #17459 about server‑imposed context restrictions vs model‑native length is directly relevant to long‑horizon engineer loops where we run into context‑budget cliffs. Suggests adding a --allow-rope-extrapolation knob to our local‑model harness. → Misc. bug: Can't use longer context than model via RoPE due to server-imposed restrictions ggml-org/llama.cpp#17459
• Idea: leak‑logger‑on‑shutdown trick — PR #22273 intentionally leaks a logger singleton to avoid Windows shutdown hangs. Same class of bug we hit in persistence_memory_operations ordering — confirms the "drain‑before‑clear" fix is the right shape. → common: Changed to leak logger singleton to prevent hanging on Windows ggml-org/llama.cpp#22273

3. Simon Willison — simonw (blog + GitHub)

Most prolific public output of the cohort; several pieces touch agentic coding directly.

• Idea: "Agentic engineering patterns" guide — Willison's new ongoing series documents concrete prompt + workflow recipes for adding features via coding agents. Maps almost 1:1 to Simard's engineer_loop planner; we should mine it for additional shell allow‑list candidates and PlanStep templates. → https://simonwillison.net/guides/agentic-engineering-patterns/adding-a-new-content-type/
• Idea: Claude Code quality regression watch — His 2026‑04‑24 post aggregates user reports of recent Claude Code degradation and Anthropic's response. Argues for our own adapter‑level regression telemetry so we detect provider drift before it shows up in review_improvements. → https://simonwillison.net/2026/Apr/24/recent-claude-code-quality-reports/
• Idea: Claude system‑prompt git timeline — Treating provider system prompts as a versioned artifact and diffing them. Strong fit for the meeting / facilitator agents — we should snapshot upstream system prompts on every run so we can attribute behavioural changes. → https://simonwillison.net/2026/Apr/18/extract-system-prompts/

4. arXiv signal (cs.AI, last week, agentic / self‑improving)

Sourced from export.arxiv.org/api/query since neither jxnl nor hwchase17 produced public GitHub activity.

• Tool Attention Is All You Need: Dynamic Tool Gating and Lazy Schema Loading for Eliminating the MCP/Tools Tax in Scalable Agentic Workflows (2026‑04‑23) — directly addresses the tool‑schema token tax we pay on every engineer turn. Worth a spike to compare against our current static allow‑list. → https://arxiv.org/abs/2604.21816
• AEL: Agent Evolving Learning for Open‑Ended Environments (2026‑04‑23) — proposes a continual self‑improvement loop with environment‑adaptive curricula. Compare with our prioritization.rs rotation/exclusion design. → https://arxiv.org/abs/2604.21725
• StructMem: Structured Memory for Long‑Horizon Behavior in LLMs (2026‑04‑23) — relevant to the memory‑consolidation drain‑order bug we tracked in src/memory_consolidation.rs; the paper's eviction policy may inform a cleaner consolidate_episodes contract. → https://arxiv.org/abs/2604.21748
• Agentic AI‑assisted coding offers a unique opportunity to instill epistemic grounding during software development (2026‑04‑23) — argues coding agents should surface uncertainty/grounding markers; ties into our anti‑sycophancy + anti‑fallback policy. → https://arxiv.org/abs/2604.21744

---

Recommended next actions

1. File a spike issue to evaluate dynamic tool gating (arXiv 2604.21816) against our allow‑list.
2. Add a dataset/split assertion in self_improve/observation (Karpathy nanochat lesson).
3. Snapshot upstream provider system prompts as a versioned artefact per Simard run (Willison).
4. Re‑run this triage in 2 weeks; manually pull jxnl (Substack) and hwchase17 (LangChain blog) since GitHub events were empty.

Generated automatically by the engineer worktree on branch engineer/explore-developer-ideas-from-tracked-researchers-1777013796-62959a. Read‑only scan; no source code modified.

[rysweet]

Follow-up scan — 2026-04-24 07:14 UTC

Re-ran the tracked-feed scan ~hour after the prior triage to surface items not yet captured. Fetched users/karpathy/events/public (gh api), simonwillison.net/atom/everything/ (curl), and export.arxiv.org/api/query (cs.AI, agentic/multi-agent/self-improving, last 24h). jxnl and hwchase17 still show no new public GitHub events; recommend supplementing via Substack/LangChain blog as previously noted.

3 most promising new ideas for amplihack/Simard

1. Learning to Communicate: Toward End-to-End Optimization of Multi-Agent Language Systems (arXiv 2604.21794, 2026-04-23) — Treats inter-agent messages as differentiable signals and jointly optimises the protocol alongside per-agent prompts. Highly relevant to our meeting-* facilitator stack and the OODA review_improvements channel: today inter-agent messages are free-form prose, which makes regression attribution noisy. A spike to encode meeting/observe→orient→decide payloads as a versioned schema (and measure end-to-end success delta when we vary it) would line up directly with this paper. → https://arxiv.org/abs/2604.21794

2. From Research Question to Scientific Workflow: Leveraging Agentic AI for Science Automation (arXiv 2604.21910, 2026-04-23) — Generalises Karpathy's autoresearch pattern: a planner that decomposes an open question into a DAG of executable workflow steps with explicit checkpoint state. Maps onto our engineer_plan.rs PlanStep + prioritization.rs rotation: their checkpoint-graph idea is a cleaner abstraction than our current flat plan list and could let us resume mid-plan after a failed shell step instead of re-planning from scratch. → https://arxiv.org/abs/2604.21910

3. Simon Willison — "An update on recent Claude Code quality reports" + "Serving the For You feed" (2026-04-24) — Two posts that bracket the same problem from different angles: (a) Anthropic's own postmortem on silent provider regressions, and (b) a deep dive on how a recommender keeps feature parity while swapping serving infra. Combined takeaway for Simard: we need a provider-fingerprint snapshot per cycle (model id, system-prompt hash, sampler params) persisted alongside ImprovementCycle so a future regression in review_improvements velocity can be attributed to upstream drift vs. our own changes. This is a concrete extension of Willison's earlier "system-prompt git timeline" idea already in the prior triage. → https://simonwillison.net/2026/Apr/24/recent-claude-code-quality-reports/ , https://simonwillison.net/2026/Apr/24/serving-the-for-you-feed/

Suggested next actions

• Open a spike issue: "Encode meeting-agent messages as a versioned schema; measure cycle-success delta" (refs arXiv 2604.21794).
• Open a spike issue: "Persist provider fingerprint per ImprovementCycle (model id + system-prompt hash + sampler params)" (refs Willison 2026-04-24 posts).
• Defer the checkpoint-graph PlanStep refactor (arXiv 2604.21910) to a design doc first — it touches engineer_plan.rs, engineer_loop/selection.rs, and prioritization.rs together.

Read-only scan; no source files modified. Sources: gh api users/karpathy/events/public, curl https://simonwillison.net/atom/everything/, curl https://export.arxiv.org/api/query?....