Running a fleet of small services without a fleet of headaches

The interesting thing about ending up with a dozen small services is that no one decides to. You build one, it works, someone wants another, and by the time you look up you’re running a dozen-odd of them. Each was reasonable on its own. The problem is that the last one costs you far more to operate than the first, because every service you added its own slightly-different way of doing things, and now “deploy a service” means remembering which of a dozen dialects this particular one speaks.

The setups I’ve seen stay sane are fleets of small API services, each doing one narrow job, where what keeps things manageable is not clever infrastructure. It’s ruthless sameness. Every service is boring in exactly the same way, and that sameness is the actual product.

The rule: one shape, no exceptions

Every service in the fleet has the identical file layout and the identical lifecycle. If you’ve seen one, you can operate all of them:

service-name/
  server.py            # the actual app, the only interesting file
  Dockerfile           # identical base, identical structure
  requirements.txt
  buildspec.yml        # identical build steps
  task-definition.json # same shape, different name/image/env
  docker-compose.yml   # local dev, identical pattern
  README.md

The temptation, every single time, is to do this one “a little differently because it’s special.” It is never special enough to justify the cost. A service that follows the shape can be built, deployed, debugged, and handed to someone else with zero new knowledge. A service that deviates is a small tax you pay forever. The sameness is worth defending like it’s load-bearing, because operationally it is.

Fargate so there are no servers to run the servers

The fleet runs on ECS with Fargate. This is the boring choice and that’s the point: there is no cluster of EC2 hosts to patch, right-size, or reason about. Each service is a task definition with a modest CPU and memory allocation, and the “infrastructure” is a JSON file in the repo. When a service needs more, you change two numbers and redeploy. When it needs less, same. Nobody logs into a box, because there is no box.

The cost story matters here. A dozen tiny always-on services can quietly add up, so the discipline is to size each one to what it actually uses, not to a round number that felt safe at 2am. Most of these services idle most of the time; they get the smallest viable task size and scale up only if the metrics say so.

The template is the platform

The thing that makes a new service cheap is a template directory. Not a framework, not a shared library that couples everything together. Just a _template/ folder with all the boring files already filled in, and a script that stamps out a new one:

./new-service.sh my-thing
# copies _template/, renames, wires in the service name,
# leaves you with only server.py to write

New service, from nothing to deployable, in the time it takes to write the actual logic. Every file that isn’t the logic is already correct because it came from the template that every other service also came from. When I fix a build-step wart, I fix it in the template and it propagates to the next service, and the existing ones get it the next time they’re touched.

A short checklist rides alongside the template: register the service, create its secret, add it to the build pipeline, confirm the health check. Boring, in writing, followed every time. That checklist has saved me from the classic “deployed but forgot the secret exists” afternoon more than once.

Shared where it helps, isolated where it counts

The fleet shares plumbing: the same VPC, the same cluster, the same build account. It does not share state or identity. Each service has its own task role scoped to only what it needs, its own secrets namespace, and its own storage if it needs any. Sharing the cheap plumbing keeps the bill sane; isolating identity and state keeps one service’s bad day from becoming the whole fleet’s bad day.

The boring conclusion

A fleet of small services is not hard because the services are hard. It’s hard when each one is a special snowflake with its own deploy ritual. Pick one shape, put it in a template, generate new services from that template, and refuse to deviate without a genuinely good reason. Run it on Fargate so there are no hosts to babysit, size each service honestly, and share only the plumbing. The result is unexciting on purpose: a fleet of services that all behave like one, and a thirteenth that costs almost nothing to add.