Table of Contents
This post ams to explain what docker is, the basics of how it works, and how to reason about it from a self-hosting perspective. This may seem a little bit strange, but the content of this article may contradict minor points of what you already knowabout docker, this is on purpose because we want to gain an intuitive, rather than purely technical, understanding. The goal is not so much to teach precisely accurate information about docker, but rather to provide a way to reason about it effectively in a self-hosting context. Feel free to skip this article if you believe you have a strong understanding of docker.
This is a very long article, running close to 3,000 words, so its probably best consumed in pieces. I did my best to break it down. Here is what you can expect to learn:
- Firstly, we're going to look at the world before Docker, and compare and contrast Docker to Virtual Machines, which are an intuitively similar technology despite being radically different.
- Once we understand the problems that Docker can solve for us, we're going to explore what underlying concepts in Linux make Docker possible (and, coincidentally, why it only runs on Linux).
- After that, we're going to explore how Docker Images are defined by Dockerfiles and their structure, including elements like the ability to build Docker Images on top of one another.
- We're going to learn about the Docker file system, and why its a huge advantage that its read-only, as well as understanding what a Docker Image is, and what volume mappings are.
- We're going to learn what a Docker Container is, building on top of all the concepts above.
- Finally, we're going to turn everything we understand in theory, into an actual configuration file, called a
docker compose
or otherwise known as astack
which demonstrates all the key concepts we've learned in a real example.
The World Before Docker: What Problem Does Docker Solve? #
Using VMs to Submit School Assignments #
Imagine for a moment, that you're a university computer science student, and you're writing a complex project. In my case, it was a custom cryptographic algorithm written in C for my Cryptography course. You've written the assignment, and it works, but the course does not have strict requirements about the runtime of the assignment and your code requires specific libraries to compile properly. You wrote the code for the assignment on a specific version of Ubuntu which has those libraries installed.
What do you do when it comes time to submit the assignment? You have two basic approaches:
- You can write up complex, detailed instructions for the instructor to follow to make sure their Ubuntu machine is set up precisely the same way, compilation instructions to make sure they compile it precisely the right way, and finally runtime instructions appropriate to the assignment.
- You can just package up a VM with the code, libraries, and all the dependencies, and submit the VM (with instructions on how to run the code that is already present, compiled, and installed). You can probably guess that I went with option 2.
This is a phenomenal use case for Virtual Machines. A virtual machine is essentially guaranteed to have all the right compile-time and runtime dependencies for a given bit of code. You can totally deploy software by simply configuring a VM with the software and uploading that.
This eliminates a LOT of problems in enterprise and "cloud" software. For one thing, as a cloud services provider, you no longer have to worry about running god-only-knows-what on your precious hardware, you can provide a basic VM interface and let them run everything themselves (this is what a virtual private server is BTW). However, despite their advantages, VMs have a lot of issues. Critically, VMs are expensive to run relative to "bare metal" software. Because VM instructions are run on a host operating system and then translated to local hardware instructions using a Hypervisor, a VM is computationally at least twice as expensive as running the same software on the machine.
In some applications, particularly where security is paramount, this is an acceptable cost, but in many cases it is not. This is the problem that docker solves. One way to reason about docker is that a docker container is kind of like a VM, but without a hypervisor, meaning that code is running on the host operating system, but docker manages all the other runtime dependencies for us. Its kind of like a half-way measure to a VM, which has most (but not all) of the isolation advantages, without the extreme computational cost.
How does Docker Work (Basics) #
What Makes Docker Possible? #
Docker only runs on Linux. Even when its running on MacOS or Windows, its actually running in a Linux VM. Why?
Well, Linux has one really important design property that makes Docker work - everything is a file. You probably experienced this already if you've ever had to format a disk in Linux, you might've worked with file paths like /dev/sda
. /dev
is a virtual file system where the kernel provides an interface for you which looks like a file system to make it easier to use. Another example of this can be seen in /proc
where data on currently running processes can be viewed by exploring the files and directories (this is how programs like htop
work BTW, they're just scanning /proc
and making it look pretty).
There's no registry, no crazy database structures. Everything is a file. In theory, if you copied all the files necessary for a database from one server onto another with the same OS, it will just work.
OK, but how does this help with Docker? Enter chroot
- a program that launches another program within a directory which is presented as the entire file system for that program. So for example, if we launch the program ls
and tell it to list all files in /
within a chroot
with the directory /var/run
, the ls
program thinks its listing the contents of /
but its actually listing the contents of /var/run
. Its a virtual file system, much like a virtual machine's instructions are interpreted to local hardware, a chroot
file system is mapping a virtual file system to a real one. The difference is that this form of mapping is very cheap computationally, it only makes a difference when a file is initially accessed.
Docker as a File System #
The above is really profound. It may not look like much, but imagine that I told you that you need to run two slightly different versions of PostgreSQL on the same machine. If you tried to install 13.2 and 13.3 on the same machine using apt
they would conflict with each other during package installation, and even if you managed to overcome that, they'd end up conflicting with each other's configuration files. However, by setting up a chroot for each one, and including all the necessary dependencies in each chroot, you could totally run two different versions of PostgreSQL in a way that would not conflict with each other.
OK, chroot
seems great, but what is Docker? Well, its basically a chroot
. Unfortunately, not everything is a file (I know, I'm sorry, I lied). There are other things other than files necessary to make complex software work. For example, there are environment variables, user associations, and network elements like ports. Well, that's what docker is - its chroot
with all the other stuff, like networking, ports, and user mapping as well. There are, however, a few other important concepts that make docker work well.
Docker Images & Dockerfile #
One of the issues that you might have thought of when presented with the VM example above, is that its actually really hard to send an entire VM to a professor to grade - its quite large. I think the smallest I could get the VM to be was like 4GiB. What if we could write a script instead, that would tell the computer:
get the Ubuntu 24.04 LTS version ISO image
install all the dependencies with apt
download the source code from github
compile the source code
when someone wants to run this, run the code compiled above
This would still be consistent, and it would take up WAY less than 4GiB in transit, right?
Good news!
What I described above is a Dockerfile. In many different github repos, you might find a file called Dockerfile
which does precisely this. Let's take a look at an example:
1#FROM - Image to start building on.
2FROM ubuntu:14.04
3
4#MAINTAINER - Identifies the maintainer of the dockerfile.
5MAINTAINER ian.miell@gmail.com
6
7#RUN - Runs a command in the container
8RUN echo "Hello world" > /tmp/hello_world.txt
9
10#CMD - Identifies the command that should be used by default when running the image as a container.
11CMD ["cat", "/tmp/hello_world.txt"]
What's happening here? Well, we tell it to download Ubuntu 14.04, we run a command to print the words "Hello world" to a file called /tmp/hello_world.txt
, and finally, we tell the computer that if someone wants to run this, we print out the contents of the aforementioned file.
When you "compile" a Dockerfile, what you get is a docker image. The docker image is essentially the aforementioned chroot
file system, with some other metadata, packaged into a single file. You can send this docker image around and running this docker image will be consistent on any system with the same kernel version since the kernel is the only thing not included in the docker image.
Inheritance #
OK, the above is pretty cool, right? But how can we make it even easier for us? One cool thing that docker has, is the ability to take one docker image, and essentially, build on top of it!.
Let's look at an example: This is the Dockerfile for microbin, one of the applications that we will be setting up in the blog series: https://github.com/szabodanika/microbin/blob/master/Dockerfile
Note that it starts with the following:
1FROM rust:latest as build
2
3WORKDIR /app
4
5RUN \
6 DEBIAN_FRONTEND=noninteractive \
7 apt-get update &&\
8 apt-get -y install ca-certificates tzdata
There's not an ubuntu:14.04
or anything similar there, what the hell is rust:latest
??
rust
is a Dockerfile/Docker Image of its own. The glory of open-source software is, of course, that we can see everything. In this case, the rust
docker image is available over at DockerHub: https://hub.docker.com/_/rust and we can follow the links there to see the rust Dockerfile:
1FROM buildpack-deps:bookworm
2
3LABEL org.opencontainers.image.source=https://github.com/rust-lang/docker-rust
4
5ENV RUSTUP_HOME=/usr/local/rustup \
6 CARGO_HOME=/usr/local/cargo \
7 PATH=/usr/local/cargo/bin:$PATH \
8 RUST_VERSION=1.87.0
Here we see that the rust
image is based on buildpack-deps:bookworm
docker image, which is of course available on DockerHub as well. The dependency chain ultimately looks something like this:
rust
depends onbuildpack-deps:bookworm
buildpack-deps:bookworm
depends onbuildpack-deps:bookworm-scm
buildpack-deps:bookworm-scm
depends onbuildpack-deps:bookworm-curl
buildpack-deps:bookworm-curl
depends ondebian:bookworm
The last of those,debian:bookworm
is a base image which we ought to recognize as the current stable version of Debian. This is, once again, profound. We don't have to build anything from scratch, we can just grab the base image that works best for us and build on top of that.
Docker is a Contract #
I have a secret to tell: I did not like Docker when it first came out. I thought it was a stupid idea.
What made me change my mind? Years after Docker came out, my friend pointed out something really important:
The key advantage of docker is that it isolates you from the idiocy of whoever wrote the code. Its an enforceable contract that the software author has to abide by for their software to work. The contract promises that if they are provided a specific volume mount, a specific set of ports, and a specific set of environment variables, then their software will function as expected.
This is really really cool, because once we have a consistent interface to something that everyone is forced to respect, we can build other systems on top. This means that we can build awesome things like Kubernetes on top of docker by treating docker images and containers as "objects" in memory and on the file system. We can move them around, start them up, shut them down, and they continue to run consistently.
But how do we actually guarantee this? After all, one of the issues you may have considered above when I told you to just run PostgreSQL in two different chroot
s is that you have no idea where all those PostgreSQL files are. After all, the configuration files are in one place, the logs are in another (if they're not in systemd
), and the actual data being stored by the application is in a third. Add to this a bunch of user configuration nonsense and we get a really confusing list of files which might even change from one version of the software to another.
The core problem here is that programmers, generally, expect to have access to the whole file system (modulo some permissions stuff) and can store stateful information anywhere they want. The next profound element of Docker that makes it work so well is that the docker container's file system (except for mounted volumes) read-only.
Initially that seems pretty weird. If the file-system is read only, what's the point? Well, it forces programmers to only write information to the specific mapped volumes. If they write information anywhere else, it will not be available after the container is restarted. This forces a certain amount of consistency in these containers, it makes images behave consistently, and it enables all the key innovations that have been built on top of docker.
Volumes #
The last little bit left to understand in this section is volume mapping. Specifically, this is how we actually retain state for a docker container. We map a specific part of the file system on the host machine, to a different part of the file system of the running docker container. That's really it.
There is, just like with everything else, something profound about this though. Because, by explicitly listing all the volume mappings, we can know where all the state of a given docker container is. Why is this so cool? Because it allows really effective backups. By backing up the specific Docker Image, the configuration file for docker itself (see the Docker Compose Stacks section below), we know for a fact that we have the entire state of the running docker container saved and backed up. There is no need to worry that we missed something, because everything is listed explicitly.
Docker Containers #
To re-iterate: A Dockerfile is a text-file written in a special language that defines the key properties required to build a Docker Image. The Docker Image, in turn, is the "frozen", read-only file system compressed into a single file, that is used to provide all the runtime dependencies that can be expressed as files for a given piece of running software. Docker Images can be built on top of each other through relationships defined in the Dockerfile.
Now that we understand the relationship between Dockerfiles and Docker Images, we need to understand precisely what a Docker Container is (again, from our limited and potentially inaccurate self-hosting perspective).
The Docker Container is the actual thing that runs the software. Its the process that is launched inside the aforementioned chroot
of the docker image with all the other key stuff like networking configuration, port mapping, volume mappings, and environment variables.
Docker Compose Stacks #
So how do we combine all the information above, into something concrete? Cause the above is probably a LOT of description and talk, not a lot of actual configuration that does the thing.
Some of you, may have recognized an inherent problem in the above examples: Sure, its great that you can spin up containers for things like PostgreSQL or MicroBin, but what if I have a "real" service, something like Paperless or something that needs PostgreSQL, Redis, and RabbitMQ to function properly? These are not unreasonable requirements, the aforementioned services are extremely efficient database, caching, and queueing systems that you don't want to replicate in your software, its WAY better to use those as pre-packages services. If I have to set up my own PostgreSQL docker container, and have all my web apps connect to it, that's actually pretty damn complicated.
This is where docker compose
comes into play. The idea is that most applications of any appreciable complexity are going to need closely associated services. For this, we have the docker compose specification. The idea is that we can combine the configuration for several docker containers into a single file, and run them more-or-less as one unit. The PostgreSQL container, for example, would exist purely to support our paperless instance, and if we want to spin up another service that depends on PostgreSQL, well they can have their own, completely separate database too in a completely separate network and everything.
Let's look at an example to understand how it works!
1services:
2 db:
3 image: postgres:alpine
4 restart: always
5 volumes:
6 - /var/lib/docker-apps/nextcloud/db:/var/lib/postgresql/data:Z
7 environment:
8 - POSTGRES_PASSWORD={{ nextcloud.db_passwd }}
9 - POSTGRES_DB=nextcloud
10 - POSTGRES_USER=nextcloud
11
12
13 redis:
14 image: redis:alpine
15 restart: always
16
17 app:
18 image: nextcloud:apache
19 restart: always
20 ports:
21 - 127.0.0.1:{{ nextcloud.port }}:80
22 volumes:
23 - /var/lib/docker-apps/nextcloud/data:/var/www/html:z
24 environment:
25 - POSTGRES_HOST=db
26 - REDIS_HOST=redis
27 - POSTGRES_PASSWORD={{ nextcloud.db_passwd }}
28 - POSTGRES_DB=nextcloud
29 - POSTGRES_USER=nextcloud
30 - NEXTCLOUD_TRUSTED_DOMAINS={{ nextcloud.hostname }}
31 - OVERWRITECLIURL=https://{{ nextcloud.hostname }}
32 - OVERWRITEPROTOCOL=https
33 - OVERWRITEHOST={{ nextcloud.hostname }}
34 depends_on:
35 - db
36 - redis
37
38 cron:
39 image: nextcloud:apache
40 restart: always
41 volumes:
42 - /var/lib/docker-apps/nextcloud/data:/var/www/html:z
43 entrypoint: /cron.sh
44 depends_on:
45 - db
46 - redis
This happens to be a template I used to use for NextCloud. I would configure these with Ansible, which is why you see all these Jinja2-style variables like {{ nextcloud.port }}
but you can mostly ignore those, they're stand-ins for real values that would be there.
Let's go through this piece by piece and understand what's happening:
1services:
2 db:
3 image: postgres:alpine
4 restart: always
5 volumes:
6 - /var/lib/docker-apps/nextcloud/db:/var/lib/postgresql/data:Z
7 environment:
8 - POSTGRES_PASSWORD={{ nextcloud.db_passwd }}
9 - POSTGRES_DB=nextcloud
10 - POSTGRES_USER=nextcloud
A docker compose file typically begins with a services
section, which defines all the docker containers being run together as one unit. The first, in this case, is the service named db
, which is not very descriptive until we see the line image: postgres:alpine
. This tells us that we're using the postgres:alpine
image, so this is a PostgreSQL database. From there, everything basically should make sense with respect to everything we discussed before:
- We have a
restart
policy that basically says "always restart this container if its shut off". This is actually something I should correct to use theunless-stopped
policy which is much more conducive to what I want. - We then have all the volume mappings, which there's really only one, it maps
/var/lib/docker-apps/nextcloud/db
on the host machine to/var/lib/postgresql/data
inside the container (see above how critical volume mappings are the whole concept of containers). This way, whenever PostgreSQL is machine changes to the database, everything is stored in a consistent location on the host machine. I happen to keep all this stuff in/var/lib/docker-apps/
but you are, of course, free to do it however you like. - And finally we have environment variables, which are things that PostgreSQL expects to have, like the default database, username, and password. There are more that PostgreSQL will accept, and they're documented accordingly.
1 redis:
2 image: redis:alpine
3 restart: always
Here we have our redis container, by this point what's happening here should be obvious, we're creating the container based on the redis:alpine image, and we say it should always be restarted. Don't even need much else.
1 app:
2 image: nextcloud:apache
3 restart: always
4 ports:
5 - 127.0.0.1:{{ nextcloud.port }}:80
6 volumes:
7 - /var/lib/docker-apps/nextcloud/data:/var/www/html:z
8 environment:
9 - POSTGRES_HOST=db
10 - REDIS_HOST=redis
11 - POSTGRES_PASSWORD={{ nextcloud.db_passwd }}
12 - POSTGRES_DB=nextcloud
13 - POSTGRES_USER=nextcloud
14 - NEXTCLOUD_TRUSTED_DOMAINS={{ nextcloud.hostname }}
15 - OVERWRITECLIURL=https://{{ nextcloud.hostname }}
16 - OVERWRITEPROTOCOL=https
17 - OVERWRITEHOST={{ nextcloud.hostname }}
18 depends_on:
19 - db
20 - redis
Here is where we get into something interesting. This is the "main" NextCloud container definition. We see some things, like the image:
, volumes:
, and environment:
which should already be familiar, however, there are a few new things too:
- For the first time, we see a
ports:
definition which maps127.0.0.1:<PORT>
on the host to port 80 on the container itself. There are two interesting things going on here:- Note that by using a binding with
127.0.0.1
as the address, this means that the application can only be accessed from the local machine. Why would I do such a thing, after all, this is meant to run on a headless (CLI-only) server, it doesn't even have a browser. The reason for this is that I do not expose the application to the internet directly, it is instead fronted by a proxy server. We're going to discuss this later. If you wanted to make this accessible from anywhere, you would put in either0.0.0.0:<PORT>
or just the port number. - How do we know that NextCloud is running on port 80 inside the container? It has to be documented somewhere. The port is part of the "contract" that you have with the programmer. Some applications will have this be configurable, while others expect a dedicated default port to be available for them inside their container.
- Note that by using a binding with
- Then, there's something else that you might've missed:
POSTGRES_HOST=db
,REDIS_HOST=redis
. What in the flying spaghetti monster's holy meatballs is that? A host is typically a hostname, how is it possible that the hostname is justdb
andredis
, I don't own those hostnames, nevermind wanna spend the time configuring them to point to my container globally. What is going on here? Well,docker
has an interesting internal networking feature: Whatever you name a container, anything else in that stack (stack = the set of containers managed by a singledocker compose
) can reach that container over the local docker network at a hostname that is identical to the name of the service. So yes, inside the container there really is a "domain" name calleddb
and it points straight to our PostgreSQL container. - Finally, there's this section:
depends_on:
and it listsdb
andredis
. What this means is that theapp
container will wait for thedb
andredis
containers to be up. This is good so that your application doesn't suffer an error on startup because one of the containers hasn't booted up yet.
1 cron:
2 image: nextcloud:apache
3 restart: always
4 volumes:
5 - /var/lib/docker-apps/nextcloud/data:/var/www/html:z
6 entrypoint: /cron.sh
7 depends_on:
8 - db
9 - redis
And finally we have this cron thing, which is really just a nextcloud container repurposed for some specific tasks (like cron jobs). Not much new here other than the fact that we specify an entrypoint
which is a script that gets run inside the container on startup instead of what would normally be defined in the Dockerfile.
Conclusion #
By the end of this article, you should have a solid foundation of how to reason about docker
as a self-hoster. Note that some elements presented here may have inaccuracies, however, if you read the above thoroughly what you will find is that it is relatively easy to search for what you want to understand and dig into it for a more formal overview. Best of luck!
Next, we are going to dive into managing docker containers with dockge
.