Docker compose inside docker, autoscaled to zero on Fly.io

"Is it terrible or is it wonderful?"

Check out the repo here: (Github) Docker Compose On Fly.io

I want to go to prod with docker compose and I want the multi-container stack to autoscale to zero.

I want to do this as ephemeral preview deployment per branch of a multi-container application.

Spoiler alert: it wasn’t as straightforward as I’d hoped. But the journey led me to some interesting places, and I think it’s worth sharing for anyone facing similar hurdles.

Initial Plan: EC2

My first thought was to deploy the application using an EC2 instance. This approach would allow me to run Docker Compose just as I did locally. However, I wanted to get features like horizontal autoscaling and scaling to zero when the application wasn’t in use and managing ec2 like this via api or manually is a lot of undifferentiated work.

Managed Platforms like ECS Fargate, Cloud Run, or Railway?

I then turned to managed platforms like AWS ECS Fargate, Google Cloud Run, and Railway. These services offer automatic scaling. However, they really only accept only individual Docker containers, not Docker Compose configurations. This meant I couldn’t directly deploy my docker-compose.yml file to these platforms.

This is leading towards some very complex kubernetes with stacked control plane land, not exactly my level of expertise, right?

Attempting Docker-in-Docker

An idea that came to mind was using Docker-in-Docker (DinD) to run Docker Compose inside a Docker container. In theory, this could allow me to execute docker-compose up within a container on these platforms, and allow the outer docker itself to be managed to zero itself by the platform.

So I learned that Docker-in-Docker requires sharing the host’s Docker socket with the container, which poses straightforward security risks about file access. You get that by running containers in privileged mode—a practice that managed platforms generally prohibit to maintain a secure environment. This approach wasn’t viable given those security implications and platform restrictions.

Fly.io

While searching for alternatives, I came across Fly.io (link). Fly.io runs applications using Open Container Initiative (OCI) images, similar to Docker images, but instead of using Docker, it employs Firecracker microVMs to run containers as lightweight virtual machines. This setup intrigued me because it seemed to offer the scalability and efficiency I was seeking without the need for Docker-in-Docker, since you'd be runner docker-on-vm/firecracker.

Implementing Docker Compose on Fly.io

Let's do it.

Here’s the approach I took:

1. Creating a Custom Image: I built a Docker image that included both Docker and Docker Compose installed. This image would serve as the environment to run my docker-compose.yml file.
2. Deploying to Fly.io: Using Fly.io’s command-line tool, flyctl, I initialized a new application with flyctl launch and deployed it using flyctl deploy.
3. Running Docker Compose: Once deployed, the Fly.io machine started, and I was able to run docker-compose up inside it, orchestrating the services as defined in my Compose file.

Steps to Run

flyctl launch

Boom, it works.

Drawbacks

Is this the textbook way to deploy applications to production? Nope. There are trade-offs to consider—like added complexity and potential performance overhead. That's why people doing this for real maybe end up with kubernetes and bespoke control planes.

But for my needs, it worked immedaitely. It's a practical solution that delivered on autoscaling and efficient resource use without requiring a major overhaul of my existing setup.

Thoughts

This solution is not really production-ready for real by itself and isn't really suitable yet for many scenarios, and there might be more efficient ways to achieve similar goals. It's left me thinking more about best practices for deploying multi-container applications to arbitrary N (customer) targets and how what the production scale of this would look like.