Portfolio

Field notes

Field notes from building production AI systems.

A hardcoded websocket runtime replaced by a manifest registry and a compiler with fail-closed gates
01Architecture

The Second Agent Forced a Platform

When we added a second production agent, the websocket handler already branched on agent type in four places, and a location lookup that existed on only one path failed without an exception. Two agents were enough to justify a manifest, a compiler, and a fail-closed runtime.

agent platformsorchestrationarchitecturegovernance
July 2, 20267 min
Four context pillars supporting a ladder from assisted work to accountable autonomy
02AI literacy

Prompt Training Does Not Produce Operational Fluency

Prompt workshops explain less and less of the gap between a fluent demo and a system a business can run. The literacy that transfers is four kinds of context—data, tools, governance, and process—and how each one changes what a model is allowed to do.

context engineeringai literacyagentic systemsgovernanceenterprise adoption
July 20266 min
A gated delivery loop moving through planning, implementation, verification, triage, and audit
03Operating model

A Forty-Nine-Hour Run That Did Not Ship

After 49 hours, 124 iterations, and 40 fixes, the Dark Factory loop scored 78.85 against a release threshold of 80 and did not release. Most of the 313 findings were not product defects. The run is evidence that acceptance and triage have to sit outside the worker that wants to be done.

dark factoryautonomous deliveryverificationagent harnessgovernance
June 29, 20268 min
An agent fleet bill decomposed into context-reading and output-generation costs
04Unit economics

Most of the Agent Bill Was Spent Reading Context

In one development workload, a one-day fleet snapshot came in near $1,450 at list price. About 92 percent of that cost was input and cache-read activity rather than output tokens. Shortening the answers would have barely moved the bill. Context policy, routing, and hard dispatch budgets would have.

unit economicscost controlagentsbudgets
June 26, 20267 min
An insight card expanded to reveal its SQL, parameters, row count, and result hash
05Trust

A Guest Count That Was Wrong by 288 Percent

An analytics agent reported 5,121 guests where the database held 1,320. The model had called the expected tool and summarized the returned value correctly. The tool counted joined rows instead of distinct guests. Treating provenance as part of the answer made the defect diagnosable and the correction auditable.

provenancetrustevalssemantic layer
June 24, 20267 min
A layered memory system with scoped facts, share grants, an access log, and a revoke path
06Production notes

Memory Has to Survive a Revoke Test

Retrieval is the easy part of agent memory. The harder requirements are scope, sharing, access history, expiration, and revocation that takes effect on the next turn. Adding those properties did not produce a defensible lift in answer quality. It produced a system whose access could be explained, withdrawn, and audited.

agent memoryarchitecturegovernance
June 22, 20267 min
A chat stream producing a structured artifact that moves onto a persistent canvas
07Product architecture

The Work Product Has to Outlive the Conversation

A chat stream is a useful place to gather intent and a weak unit of work. One protocol consolidation reduced more than fifty event variants to six and made a persistent artifact the handoff between conversation, review, and reuse.

artifactsstreamingagentsproduct design
June 20, 20266 min
One compiled agent pipeline serving chat, voice, CLI, scheduling, and delegation clients
08Architecture

An Agent That Only Runs in Chat Is Not a Platform

An agent that only works behind a live chat socket contains hidden coupling. Running the same compiled path with no socket and no assumed human identity is a direct test of whether voice, schedules, CLI tools, and agent-to-agent calls are real clients or a second product that has not been admitted yet.

agent platformsarchitectureautomationgovernance
June 18, 20266 min
Acceptance authority moving from a worker claim to an independently executed grader
09Research note

A Completion Claim Should Not Authorize a Merge

Coding agents often end with a fluent claim that the task is complete. If the orchestrator treats that sentence as acceptance, an unsupported claim can terminate repair and authorize a merge. A simulation of claim-trusting and independently verified orchestration isolates the safety benefit, the repair tradeoff, and the assumptions that limit the result.

verificationorchestrationcoding agentsevalstrustworthiness
June 15, 202610 min
A mid-session model crossover comparing orchestration behavior and fleet cost
010Field method

Swapping the Orchestrator Made the Fleet More Expensive

One live session changed frontier orchestrator models without resetting the harness or the task. The trial cannot rank the models, but it revealed a cost failure that appears only at fleet scale: delegated workers inherited the expensive model, and token burn reached about 2.7 times the prior peak hour.

orchestrationevalsautonomous developmentmodel selection
June 10, 20268 min
A funnel reducing raw training rows to distinct behavior clusters and an unresolved duplication outlier
011Model training

A Corpus of 104,272 Rows Covered 868 Behaviors

A distillation corpus can look large and still cover a small set of behaviors. After normalization, deduplication, clustering, and labeling, 104,272 captured records collapsed to 868 distinct target behaviors. The original row count had been measuring the collection pipeline, not coverage.

model distillationdata qualityfine-tuning
June 8, 20267 min
A latency ladder separating a warm model benchmark from production time to first audio
012Local inference

Why a 0.19-Second Model Produced Nine-Second Audio

A warm text-to-speech microbenchmark measured 0.19 seconds. The live voice pipeline took about nine seconds to produce first audio. Both numbers were true. They answered different questions, and only the second one described the product.

local inferenceApple SiliconMLXbenchmarking
June 6, 20267 min
A green status dashboard contrasted with a broken evidence chain beneath it
013Trust and verification

How Autonomous Systems Produce False Green Metrics

A green dashboard can come from a stale test, a self-reported completion, or a benchmark that never touched the live product. None of those failures requires a lying model. They require a reporting path that can detach from the artifact it describes.

autonomous agentsverificationgovernance
June 4, 20266 min
A task-specific eval comparing nine models on accuracy, cost, latency, and severe-error rate
014Evaluation

Build an Eval Around the Decision You Have to Make

A public leaderboard cannot tell you whether a model is reliable on your work. An 84-prompt planner-intent eval could: a local model reached 96.7 percent strict accuracy, a more expensive model reached 94.6 percent, and price stopped being a useful proxy for fitness.

evalsmodel selectioncost control
June 2, 20266 min
A historical transition arc with AI adding a jagged verification constraint
015Strategy

The Value Appears After the Work Is Redesigned

Electricity, spreadsheets, and the internet created value after organizations redesigned work around them. AI follows that pattern, with one additional complication: the system can generate plausible actions faster than an organization can verify them.

technology transitionsai strategyverificationoperating model
May 26, 20266 min
A routing board comparing frontier, small proprietary, and open-weight models by task quality and cost
016Cost per task

Route Models by Task, Not by Reputation

Model rankings collapse distinct jobs into one score. A routing policy that actually saves money measures the work that matters, sets a quality floor, and assigns the cheapest eligible model to each task class, with a way to abstain when the cheap route should not guess.

cost per taskmodel routingopen-weight AI
May 20265 min
A hybrid inference architecture with private endpoints, hosted models, local hardware, observability, and policy controls
017Inference

Inference Strategy Is a Portfolio, Not a Slogan

Inference strategy is no longer a choice between one hosted API and one local model. Privacy, latency, continuity, quality, and operations differ by workload, which makes a governed portfolio more useful than a blanket claim that the company should own inference.

inferenceAI infrastructuresecurity
May 20265 min
Multiple agent harnesses feeding a shared substrate of traces, tests, budgets, and tool policy
018Operating model

Harness Choice Can Be Plural if Evidence Authority Is Singular

Teams can use different coding-agent harnesses without abandoning governance, provided that traces, acceptance tests, budgets, and artifact identity live outside any one harness's control. Otherwise every client becomes its own governance regime.

AI harnessgovernanceobservabilityevals
May 20266 min
A planning map with contracts, dependencies, verification paths, and approval checkpoints
019Planning

Use a Planning Model to Expose Disagreement

Planning models are most useful when they expose missing decisions before execution. A strong plan names interfaces, evidence, ownership, and failure boundaries. It does not quietly make the product decision.

planningagentic workexecution
April 20265 min
Repeated social-post patterns grouped into archetypes with a signal audit overlay
020Signal

A Short Test for Writing That Adds No Evidence

AI writing often converges on the same hooks, contrasts, and conclusions. A small audit can separate a new coat of rhetoric from evidence that actually changes what a reader should do.

writingAI literacysignal
April 20265 min
Three AI adoption levels with distinct blockers, guardrails, and evidence requirements
021Enterprise adoption

Answering, Collaborating, and Acting Fail for Different Reasons

AI adoption is often treated as one maturity curve. In practice, answer-seeking, workflow collaboration, and bounded orchestration fail for different reasons and require different controls. Forcing every job up the ladder is how an organization buys governance cost it did not need.

enterprise AIliteracyoperating model
April 20266 min
An operator coordinating tools, agents, data, and enterprise systems through typed contracts
022Organization design

When Generation Is Cheap, Contract Design Becomes the Scarce Skill

When more people can generate interfaces, workflows, and automations, the scarce skill moves toward defining the contracts that determine what may render, execute, persist, or fail. Without those contracts, participation becomes accidental authority.

design engineeringAI operating systemleadership
April 20265 min
A layered enterprise AI stack annotated with the cost of replacing each layer
023Architecture

Eighty-Seven Hardcoded Model Names Is Self-Inflicted Lock-In

Architecture diagrams often describe replaceable components that remain entangled in code. A more demanding audit measures how many contracts, callers, and tests must change when a model, data source, or business domain is removed. In one platform, callers had learned 87 direct model references. No vendor required that coupling.

architectureAI gatewaygovernancemodel routing
April 20266 min