Chapter 7: Shipping It

Time: 25 minutes. Cost: $0 with Ollama, a few cents with OpenAI or Anthropic.

A recipe that only works in your kitchen, with your specific pots, your specific oven, isn't really a recipe anyone else can follow. Every lab so far has been a script you run yourself, on your machine, in your terminal. That's fine for learning, it's useless for shipping. A real product needs to run on a server nobody's sitting in front of, get called by other programs (a website, a mobile app, another service), and behave identically whether it's running on your laptop or someone else's cloud account. This chapter takes the support bot from earlier chapters and does the two things that turn a script into a service: wraps it in an HTTP API, and packages that API so it runs the same way anywhere.

An API, then a container

Turning a script into an API means putting a small web server in front of your function. Instead of running python script.py and reading the terminal, another program sends an HTTP request (POST /ask with a question) and gets an HTTP response (JSON with an answer) back. FastAPI is a lightweight Python framework built for exactly this, a few lines of code turn any function into an endpoint.

Packaging that API in Docker solves a different problem: "works on my machine" is not the same as "works." Docker packages your code together with everything it needs to run, the Python version, every dependency, down to the operating system libraries, into one portable image. Build it once, and it runs identically on your laptop, a teammate's laptop, or a cloud server, because it's not relying on whatever happens to already be installed there.

Hands-on lab: FastAPI, then a Dockerfile

The lab's app.py doesn't introduce new agent logic, it takes the exact same ask() function from every earlier chapter and puts an HTTP layer in front of it.

Full instructions: labs/advanced/07-shipping-it

Run locally first, with Ollama:

$ curl http://localhost:8000/health
{"status":"ok","provider":"ollama"}

$ curl -X POST http://localhost:8000/ask -H "Content-Type: application/json" -d '{"question": "What is your best-selling drink?"}'
{"answer":"Our best-selling drink is the Depot Latte - a rich and creamy blend of espresso, steamed milk, and a hint of vanilla flavoring."}

💡 On Windows PowerShell, run curl.exe instead of plain curl -- PowerShell aliases curl to Invoke-WebRequest, which doesn't accept -d the same way. curl.exe runs the real curl binary that ships with Windows 10 and later, and the command above works as written.

Then build and run the exact same app inside Docker:

docker build -t fernwood-api .
docker run --rm -d --name fernwood-api --env-file .env -e OLLAMA_URL=http://host.docker.internal:11434 -p 8000:8000 fernwood-api

$ curl http://localhost:8000/health
{"status":"ok","provider":"ollama"}

$ curl -X POST http://localhost:8000/ask -H "Content-Type: application/json" -d '{"question": "How many locations do you have?"}'
{"answer":"We currently have three locations in our home state."}

Same code, same questions, same shape of answers, one running directly on the machine, the other running inside an isolated container. That -e OLLAMA_URL=http://host.docker.internal:11434 is the one line that makes the second one work: inside a container, localhost means the container itself, not your host machine. The app's normal http://localhost:11434 would try to find Ollama running inside the container, where it doesn't exist. host.docker.internal is Docker's special hostname for "actually, the machine running this container", overriding the URL through an environment variable is how the same code adapts to that without an if running_in_docker: check anywhere.

Checkpoint

The exact same ask() function from Chapters 4-6 is reused unchanged in this chapter's app.py. What did actually change to turn it into a web service?

Nothing about the model-calling logic changed at all. What's new is the layer around it: FastAPI's @app.post("/ask") decorator turns the function into something reachable over HTTP, and Question/Answer Pydantic models define what a valid request and response look like. The service boundary moved from "you, typing into a terminal" to "any HTTP client," the actual work being done is identical.

Running the app with uv run uvicorn app:app works fine and reaches Ollama with no special configuration. Running the same app in Docker needs OLLAMA_URL=http://host.docker.internal:11434. Why the difference?

A container has its own isolated network namespace. Inside it, "localhost" resolves to the container itself, not the machine hosting it, so http://localhost:11434 looks for an Ollama server running inside the container, which doesn't exist. host.docker.internal is a hostname Docker provides specifically to reach back out to the host machine from inside a container, which is where Ollama is actually running.

The Dockerfile copies pyproject.toml and uv.lock and runs uv sync before copying app.py. Why that order, rather than copying everything at once?

Docker builds images in layers, and reuses a layer from a previous build if nothing that layer depends on has changed. Dependencies (pyproject.toml/uv.lock) change far less often than application code. Installing them in an earlier layer means editing app.py and rebuilding doesn't reinstall every dependency from scratch, only the last layer (copying the code) actually redoes work. On a project with real dependencies, that's the difference between a multi-minute rebuild and a near-instant one.

Check Your Knowledge

Click to start quiz

1. The chapter says Docker "packages your code together with everything it needs to run... into one portable image." What problem is that solving that running the script locally doesn't already solve?

It makes the code itself run computations fasterIt bundles the code with its exact Python version, dependencies, and OS libraries, so "works on my machine" becomes "works," regardless of what's already installed on whatever machine runs it nextIt replaces the need for a .env file entirelyIt automatically converts the app to a different programming language for portability

2. The chapter describes "turning a script into an API" as putting a small web server in front of a function. What actually changes for something calling that function once FastAPI is in front of it?

The function's internal logic has to be rewritten to be web-awareAnother program can send an HTTP request (like POST /ask) and get an HTTP response back, instead of running the script directly and reading its terminal outputThe function can only be called by other Python programs, never by curl or a browserThe function starts running continuously in the background instead of only on demand

3. In the chapter's architecture diagram, the FastAPI app and the ask() function both sit inside the "Docker container" subgraph. What sits outside that boundary?

Nothing -- the whole diagram is inside the Docker containerThe curl client making the request, and the model provider (Ollama/OpenAI/Anthropic) that ask() ultimately calls out toOnly the model provider is outside; the curl client is also inside the containerOnly the curl client is outside; the model provider runs inside the container too

4. The lab's Question and Answer classes are Pydantic BaseModels, even though this curriculum's lab code otherwise avoids classes. Why is that not a contradiction of the "no unnecessary abstraction" rule?

Pydantic models aren't actually classes under the hoodFastAPI's normal, expected way to define request/response schemas is a Pydantic model -- using the framework's own idiom isn't adding an abstraction on top of the problem, it's how the problem is meant to be solvedThe rule against classes only applies to labs that don't use any external frameworkQuestion and Answer are actually optional and could be removed with no effect

What's next

You now have every individual piece: retrieval, agents, guardrails, tracing, caching, and a shippable container. The capstone puts them all in one system, an agent with real tools, evaluated, guarded, and traced, wrapped in the same API-and-container shape as this chapter, so it's not just a working system, it's one you could actually deploy.