Page 3
12 articlesThe Fine-Tune Artifact Your Departing Engineer Took With Them
A fine-tuned model is not a file in a registry; it is the closure of a pipeline over a training set. The teams that ship only the weights discover their bus factor the day a base-model migration arrives and the original engineer is gone.
The JSON Schema Your Output Passed and Your Downstream Consumer Rejected for Semantic Drift
A JSON schema validates shape, not meaning. When an LLM upgrade shifts the distribution of values inside a still-valid schema, downstream consumers detonate while the producer's dashboard stays green.
The KV Cache Eviction Your Provider Called Cache Pressure and Your Bill Called a Doubled Prefix Charge
Prompt caching looks like a configured discount, but KV cache eviction on shared LLM infrastructure turns it into a probabilistic one — the same conversation can cost several times more on a busy day with no code change.
The KV Cache Warm-Up Cron That Ran in Blue and Never in Green Because the Host Pinning Never Moved
A blue/green deployment orphaned a cron pinned to the old color, the prompt cache went cold, and the bill quietly tripled — anatomy of a silent regression and the four practices that close the seam.
The Legal Disclaimer That Leaked From The Answer Into The Tool Call Arguments
A safety disclaimer added to your system prompt does not stop at the user-facing reply. It rides along into every tool call argument the model produces — and into the downstream systems those calls fire against.
The LLM-as-Judge Ensemble That Agreed Because All Judges Were the Same Family
An LLM-as-judge ensemble drawn from one provider family measures family-internal consistency, not judgment quality, and the high agreement number is an artifact of provider selection nobody named.
The Logprobs Field Your Provider Removed That Broke Your Confidence Router Silently
A confidence router that stopped escalating low-confidence answers, the silent provider tier change that caused it, and how response-shape contracts, population-level alerts, and a fallback written for the wrong failure mode hide together.
The max_tokens Default Your Provider Raised That Doubled Your Tail Response Length
An LLM provider quietly raised the default max_tokens value and your p99 output length doubled overnight. The parameter you did not send is the configuration that changed under you — here is how to stop inheriting defaults you do not control.
The MCP Server Your Team Forgot Was Running with Prod Credentials
An MCP server running on a developer laptop with a CI-grade OAuth token is a production attack surface. Here is how DNS rebinding, bad bindings, and shared tokens turn one compromised tab into a deploy-key leak.
The Model Card Benchmark Whose Methodology Shifted While Your Contract Cited the Number
A benchmark number is a measurement under a protocol, and the protocol is what your vendor controls. Pin the methodology, or contract on your own eval suite.
The Model Identifier Your Provider Re-Pointed to a Finetune for One Tenant and to Base for Everyone Else
Treating an LLM model identifier as a name for the weights instead of a label on a routing decision lets the provider silently swap your tenant from a fine-tune to base while the eval suite stays green and customers notice first.
The Model Registry Your Platform Team Built That Nobody Updated
A model registry's promotion gates only work when reviewers have time, independent evidence, and aligned incentives. Most teams build the first half of that contract and skip the rest, and the registry decays into a paperwork pipeline that approves whatever the producer ships.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
