2 posts tagged with "production"

Calling an LLM provider's SDK directly is a good way to build a prototype. Letting every product feature call that SDK directly is how a provider detail becomes an application architecture.

The distinction matters. “Stop calling LLM APIs directly” does not mean every team needs to install a commercial gateway. It means production code should call a boundary your application owns. Behind that boundary, you can make explicit decisions about providers, deadlines, errors, data policy, and degraded behavior.

Without it, adding a second API key gives you another provider. It does not give you a fallback.

MCP's HTTP transport has worked like a phone call since Streamable HTTP shipped in March 2025: a client dials in with initialize, the server answers and hands back an Mcp-Session-Id, and every request after that stays pinned to whoever picked up. Hang up on the wrong server instance and the whole conversation is gone with it. Sixteen months later, the 2026-07-28 spec revision rips the phone call out entirely. MCP is now stateless. Every request carries everything it needs, and any healthy server behind a load balancer can answer it cold.