A developer's quiet notebook — practical notes from experiment to production
Holding NIA's 'AI infrastructure hegemony' prediction against six months of reality. Compute alliances and bloc formation landed as forecast (🔵), semiconductor diversification was only directionally right (🟢), and the real deciders - HBM/power/serving-software bottlenecks and the capex bubble - were outside the forecast.
AI products are fundamentally different from traditional software. This post distills 12 counterintuitive lessons from 20 builders, the CC/CD framework for AI development lifecycles, and a 3-phase eval system - all sourced from Lenny Rachitsky's newsletter.
Self-Tuning Loop 4 steps (Generate → Capture → Analyze → Evolve) extracted as a universal module. Supabase DDL, diff capture utilities, analysis/evolution prompts, email/blog examples, GitHub reference implementation.
Evans 2026 deck notes - three chapters held together by one question: infrastructure boom rarely turns into value capture
I ran a seventeen-persona roundtable twice. I discarded almost the entire business track and kept the safety-design track word for word in the spec. I only later realized the line wasn't advice quality, it was identity. Chapter 6.
Reads NIA's 2026 Top 12 trends report on its own terms - identity, structure, derivation method - and flags the backward-looking nature of text-network analysis. Series entry post.
At first I just thought of it as an interactive edu tool. Then, passing through a domain with no right answer and the LLM's tendency to agree, the name became not a label but a safeguard. Chapter 5 of a serialized notebook.
A read of DRIFT and TELBench, which drop to the claim level to flag the error spans that affect the answer. Finding harmful spans climbs past halfway, but the first error stalls around 20%. Series Part 4, the finale.
The Fable 5 hidden reroute and the government shutdown, and where two assumptions we make when paying for a chatbot break: is it the model I picked, and is access my right?
A method-first read of the three papers for evaluating custom agents (MAST, AgentRx, DRBench): a vocabulary for failure, a way to pinpoint where it broke, a testbed shaped like your data. How a builder borrows research-scale work. Series Part 3.
A method-first read of the three papers that evaluate well-packaged agents (TRACE, DeepHalluBench, TRAIL): the altitude that rewrites the score, the one that verifies claims, the one that changes the substrate. Series Part 2.
Starting from the high-score illusion where the top-accuracy model ranks last on utility, this piece lays out why evaluation is moving to process and maps the methods of seven papers. Series Part 1.
Evans and Jensen both treat the model as a commodity, then split on whether value flows up to apps or down to infrastructure
I set the closest things beside mine to check if the spot was really empty. Where Khanmigo, Replika, and Brilliant part ways, and why the philosophy-chatbot line stays blurry. Chapter 4 of a serialized notebook.