Asa Shepard

Researching and engineering how language models learn and remember

New blog Silent speech demo

What It Costs to Compose, Rebuild, and Correct Precomputed Memory Asa Shepard Saved KV caches and trained compressions degrade when assembled from separately prepared parts, stay current only through costly rebuilds, and can ignore corrections served beside them (Aug 2026)

Learned, Then Lost: A Measured Single-Example Counterfactual in Pre-training Zachary Speck, Asa Shepard Measuring, rather than estimating, one training example’s effect across paired GPT-2 pre-training runs: a passage is learned 50 steps after injection and undetectable by the final step; accepted to NeurIPS ATTRIB 2026 (Aug 2026)

Probe-and-Refine Tuning of Repository Guidance for Coding Agents Asa Shepard, Jeannie Albrecht Refining repo-level guidance for AI coding agents via synthetic probes; guidance helps on SWE-bench Verified; accepted to COLM WAB 2026 (Jun 2026)

A Model’s A Priori Is Frozen A Posteriori The philosophy of knowledge discovery, and what six weeks on a project called Aedos taught me about the cost of being sure.

Cross-Repository Guidance for Coding Agents Guiding a coding agent on a codebase it was never tuned on, using what probe-and-refine learned on 50 other repositories.

Emergent Misalignment is Co-occurrence Density Made Visible A response to a recent paper on narrow finetuning and broad generalization, reframed in terms of the pretraining clusters being amplified.

Structural Grammar of the Voynich Manuscript A 579-experiment autonomous investigation of the manuscript, and what the scaffolding around the agent caught.

Sediment Researcher and Co-Founder (Oct 2025–present)

sedimentlabs.ai

Visa Software Engineer Intern; bay area, creating agents, Cybersource team (Summer 2026)

cybersource.com

World Wide Technology Technology Consulting Intern; built an evaluation of recursive retrieval vs. baseline RAG for compliance extraction (Jan 2026)

Yosa Co-founder; HIPAA-aligned mental wellness platform with 30 active providers (2025–2026)

OnCabaret Contract software engineer; social app for a NYC startup with 2,000 users (2025)

Web Surfing Studios Software engineer intern; built full-stack web applications (2024)

websurfingstudios.com

WSO (Williams Students Online) Front-end developer and tester (2024)

wso.williams.edu

Cycorp Software engineering and testing intern; wrote 2,000+ higher-order logic statements for the Cyc knowledge base (2022)

WeaveHacks 4: HeartTwin Agentic cardiac digital-twin simulator; finalist — top 10 of ~70 teams (Jun 2026)

JacHacks SF: Jaul Agent that negotiates medical bills for uninsured and underinsured patients, grounded in hospital financial-assistance policy (Jul 2026)

TreeHacks 2026: Santé Multi-agent speech analysis for biomarkers (Feb 2026)

Campus Web App Williams College Marriage Pact analysis; 600 users (2025–Jan 2026)

williamstf.com (williams.edu only)

FocalRouter Intelligent request routing for LLM APIs (2025)

HeatSheet F# DSL for optimizing track meet lineups (2025)

Minecraft Plugins Suite of ~10 Java plugins (~2021–2022); first shipped code — top plugin at 7,000 downloads

What 40 Hours of Autonomous Research Taught Me About Long-Running Agents

Oracle Tuning Results and the Path to Adaptive Context Selection

Benchmarking AI Codebase Context — From −24 to +25 Net Tests

Aspect Code: Giving AI Assistants the Context They’re Missing