                         Software Engineering II: (Docker/Podman) Compose

                                                                   Philipp Fruck

                                                     Podman Network Setup

            ██ How does rootless networking work?

               •  Podman 5.X: pasta as default network driver

                  ◦  Behaves like application is running on the host

                  ◦  Host IP, correct source IP, but still isolated

                     ▪  Port forwarding is required compared to host networking

               •  Previous default: bridge driver

                  ◦  Still default when connecting containers via shared network

                  ◦  NAT is used -> Source IP is always the NAT IP

                  ◦  IPv6: Interesting... Sometimes broken

            Q: How can two pods access each other over the network?

               •  Pod: Share network namespace, reach each other via localhost

               •  Bridge: Connect containers to bridged network, both gain a NATed IP

                                                 Software Engineering II: (Docker/Podman) Compose                                           2 / 23

                                              Podman: Connecting Services

            The following script spawn a Postgres client and server and connects them through a custom network

                           1 #!/bin/sh

                           2 set -x

                           3 net=test; client=pgclient; server=pgserver

                           4 pg_password='p4$$w0rd'; pg_version=18-alpine

                           5 # Create network and launch detached server

                           6 podman network create ${net}

                           7 podman run --rm --name ${server} --network ${net} -e POSTGRES_PASSWORD=${pg_password} \

                           8   -d postgres:${pg_version}

                           9 # Wait some time to avoid race condition

                          10 sleep 3

                          11 # Run the client (note different password variable)

                          12 podman run --rm --name ${client} --network ${net} -e PGPASSWORD=${pg_password} \

                          13   --entrypoint psql postgres:${pg_version} -h ${server} -U postgres -c "SELECT version();"

                          14 # clean up

                          15 podman rm -f ${client} ${server} && podman network rm -f ${net}

                        ——————————————————————————————————————————— [finished] ———————————————————————————————————————————

                                                 Software Engineering II: (Docker/Podman) Compose                                           3 / 23

                                              Podman: Connecting Services

                        + net=test

                        + client=pgclient

                        + server=pgserver

                        + pg_password='p4$$w0rd'

                        + pg_version=18-alpine

                        + podman network create test

                        test

                        + podman run --rm --name pgserver --network test -e 'POSTGRES_PASSWORD=p4$$w0rd' -d

                        postgres:18-alpine

                        51c2140c5614ecd907e79c0af3c6adab935ff24c4c8736721dcdd21de203d754

                        + sleep 3

                        + podman run --rm --name pgclient --network test -e 'PGPASSWORD=p4$$w0rd' --entrypoint psql

                        postgres:18-alpine -h pgserver -U postgres -c 'SELECT version();'

                                                                 version

                        -----------------------------------------------------------------------------------------

                         PostgreSQL 18.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit

                        (1 row)

                        + podman rm -f pgclient pgserver

                        + podman network rm -f test

                        test

                                                 Software Engineering II: (Docker/Podman) Compose                                           4 / 23

                                              Podman: Connecting Services

            ██ Shell Script Approach                                         ▓▓▓ Goal

            Do you like the manual setup?                                       •  Meta file containing setup instructions

                                                                                   ◦  More declarative than shell script

               •  Easy setup of two Postgres instances                             ◦  Better error handling

                  ◦  Can communicate over network                                  ◦  Cross platform

                  ◦  Includes DNS resolution -> Nice!                           •  Lifecycle handling?

                                                                                   ◦  Autostart?

            But:                                                                   ◦  Restart after crash?

                                                                                   ◦  Automatic updates??

               •  Verbose syntax

               •  Setup is error-prone

                  ◦  Lots of repeated custom logic

               •  What if one of the commands fails?

                                                 Software Engineering II: (Docker/Podman) Compose                                           5 / 23

                                                   Declarative Definition

            We can use docker-compose and Quadlets for declarative definition

            This can be seen as a subset of Infrastructure as Code

                                              │ docker-compose                 │ Quadlet

                            ──────────────────┼────────────────────────────────┼─────────────────────

                            Requirements      │ Docker/Podman + Compose Script │ Podman + systemd

                            Lifecycle Manager │ Docker Daemon                  │ systemd

                            Recommended for   │ Development (project-level)    │ Servers (host-level)

                            File format       │ Custom (compose)               │ systemd / Kubernetes

                                                 Software Engineering II: (Docker/Podman) Compose                                           6 / 23

                                                 Docker vs Podman Compose

            There are two major implementation of the compose spec (https://compose-spec.io)

            ██ Docker Compose                                                ██ Podman Compose

               •  Reference implementation (Docker 1st party)                   •  Community implementation for Podman

                  ◦  Always supports full feature set                           •  Translates compose file into podman CLI commands

               •  Utilizes Docker socket ("REST" API)                           •  Sequential operation (last time I checked)

               •  Allows parallel operation (image pulls etc.)

               •  v1: Standalone Script docker-compose                          •  Standalone Script podman-compose

               •  v2: Plugin for Docker docker compose

                                                 Software Engineering II: (Docker/Podman) Compose                                           7 / 23

                                                  Docker Compose + Podman

            We want to use Docker Compose with Podman

               •  v2 can still be used as standalone script

                  ◦  Many distros only ship v1.X though...

                  ◦  Ensure your distro ships docker-compose v2.X or install directly from GitHub Releases

                     ▪  docker-compose --version

               •  We need to enable the Podman Socket for Docker API support

                  ◦  systemctl --user enable --now podman.socket

               •  We need to tell docker-compose the path of our API socket!

                  ◦  export DOCKER_HOST=unix://${XDG_RUNTIME_DIR}/podman/podman.sock

                  ◦  Add this to your ~/.bashrc or similar

            All slides in this lecture assume this docker-compose + podman setup. If you use another setup, you need to replace the

            docker-compose commands (officially legacy) with docker compose or podman-compose.

                                                 Software Engineering II: (Docker/Podman) Compose                                           8 / 23

                                                  Compose: Simple Example

            ██ Compose Spec

                                                                        1 services:

               •  Simple YAML file (Minimal Syntax)                     2   web:

               •  All container under services                          3     image: docker.io/nginxinc/nginx-unprivileged

                  ◦  web: Implicit container name                       4     ports:

               •  Declarative definition of CLI                         5       - 8080:8080/tcp

                  parameters                                            6     volumes: # serve generated lecture slides

                                                                        7       - ../../_site:/usr/share/nginx/html:ro,Z

               •  Launching: docker-compose -f examples/compose/compose.simple.yml up

                  ◦  Relative paths are read from compose file

                  ◦  compose.yml and docker-compose.yml (and .yaml) are automatically detected --> no -f

                                                 Software Engineering II: (Docker/Podman) Compose                                           9 / 23

                                                     Compose: Subcommands

            Command │ Action                                                     Subcommands can be executed for all containers

            ────────┼─────────────────────────────────────────                   (default) or for a specific container

            config  │ Parse and show the final config

            pull    │ Pull service images                                           •  docker-compose [up/down/logs] myservice

            build   │ Build or rebuild services if required                            ◦  Use up -d to detach the logs

            create  │ Creates the containers & networks                             •  Use logs -f to stream updated logs

            start   │ Start services (must be created first)                        •  Use up --build for a build shortcut

            restart │ Restart services                                              •  Use run --rm to prevent zombies

            up      │ Shortcut for create and start

            stop    │ Shutdown all services (keep containers)

            down    │ Stop and remove containers & networks

            logs    │ View output from containers

            exec    │ Execute command in running container

            run     │ Execute command in a new container

            watch   │ Watch service, refresh when files change

                                                 Software Engineering II: (Docker/Podman) Compose                                          10 / 23

                                                     Compose: More Syntax

                                                                 We can specify any container option in the compose file

               1 services:

               2   app:                                             •  Build the service app from local Containerfile in directory .

               3     build:                                         •  When using Dockerfile in same folder, build . is sufficient

               4       context: .                                      ◦  Remember: No automatic rebuilds!

               5       dockerfile: Containerfile

               6     container_name: demo-app                    ▓▓▓ Naming

               7     command: ["--http", ":8080"]

               8     entrypoint: ["/bin/myapp"]                     •  COMPOSE_PROJECT_NAME: Parent folder of the compose file, e.g.

               9     user: "1000:1000"                                 compose

              10     working_dir: /app                                 ◦  Can be overridden (environment or top-level name:)

              11     env_file: .env                                 •  container_name sets explicit name demo-app instead of

              12     environment:                                      compose-app-1

              13       LECTURE: "SE2"

              14     ports:

              15       - 127.0.0.1:8080:8080/tcp

                                                 Software Engineering II: (Docker/Podman) Compose                                          11 / 23

                                                       Compose: Variables

                                                                    ██ Variables

               1 services:

               2   app:                                                •  Compose reads current environment and parses .env file

               3     image: alpine:${TAG:-latest}                         relative to compose file

               4     env_file: .env                                    •  Variable must be passed to containers explicitly

               5     environment:

               6       OTHERPASSWORD: ${MYPASSWORD}                 ██ Substitution

               7     command:

               8       - sh                                         Syntax          │ Evaluates To

               9       - -c                                         ────────────────┼────────────────────────────────

              10       - env | grep PASSWORD                        ${VAR}          │ Value of VAR

                                                                    ${VAR:-default} │ Value of VAR, otherwise default

                                                                    ${VAR:?error}   │ Require VAR or throw error

                                                 Software Engineering II: (Docker/Podman) Compose                                          12 / 23

                                                       Compose: Variables

               1 services:                                                  file=../examples/compose/compose.env.yml

               2   app:                                                     docker-compose -f $file config | grep -A2

               3     image: alpine:${TAG:-latest}                           environment

               4     env_file: .env

               5     environment:

               6       OTHERPASSWORD: ${MYPASSWORD}                      ——————————————————————— [finished] ———————————————————————

               7     command:

               8       - sh

               9       - -c                                                     environment:

              10       - env | grep PASSWORD                                      MYPASSWORD: s3cr3t

                                                                                  OTHERPASSWORD: s3cr3t

              # ../examples/compose/.env

              MYPASSWORD=s3cr3t

                                                 Software Engineering II: (Docker/Podman) Compose                                          13 / 23

                                                         Compose: Network

                                                                    ██ Config

               1 services:

               2   proxy:                                              •  Compose creates a default project network

               3     image:                                               ◦  Each container has implicit default network

                 nginxinc/nginx-unprivileged:alpine                    •  Custom networks can be created under networks:

               4     networks: ["web"]                                    ◦  Simple name is sufficient

               5   backend:                                               ◦  Advanced options like subnet and network driver can be

               6     image: python:alpine                                    specified

               7     networks: ["web", "db"]                           •  Bridged networks support DNS for container names

               8   db:                                                    ◦  E.g. backend can reach db via db hostname

               9     image: postgres:alpine                               ◦  Ensure Aardvark DNS plugin is installed!

              10     networks: ["db"]

              11 networks:

              12   db:

              13   web:

              14     enable_ipv6: true

              15     ipam:

              16       config:

              17         - subnet: 2001:db8::/64

                                                 Software Engineering II: (Docker/Podman) Compose                                          14 / 23

                                                                 Exercise

            Remember the script from the beginning? Your task is to reproduce this using a compose file!

            The script is located in the git repo under ../examples/compose/manual.sh

               •  Spawn two Postgres containers                              Use a .env file like this to share variables between the

                  ◦  client and server                                       two containers:

                  ◦  server should keep running (daemon)

                  ◦  client should only execute the query and exit

                                                                                   POSTGRES_VERSION=18-alpine

                                                                                   POSTGRES_PASSWORD=s3cr3t

                                                 Software Engineering II: (Docker/Podman) Compose                                          15 / 23

                                                Compose: Advanced Network

                                                                             As mentioned, source IP handling can be difficult with

               1 services:                                                   rootless networking

               2   proxy_host:

               3     image: nginxinc/nginx-unprivileged:alpine                  •  Other network modes than bridge can be used

               4     network_mode: host                                            ◦  No support for DNS resolution!

               5   proxy_old: # Workaround for Podman 4.X                       •  To get source IPs with bridge network:

               6     image: nginxinc/nginx-unprivileged:alpine                     ◦  Look into socket activation (

               7     network_mode: "                                                  https://systemd.io/DAEMON_SOCKET_ACTIVATION)

                 slirp4netns:port_handler=slirp4netns"

               8   proxy_new: # Workaround for Podman 5.X

               9     image: nginxinc/nginx-unprivileged:alpine

              10     network_mode: "pasta"

                                                 Software Engineering II: (Docker/Podman) Compose                                          16 / 23

                                                        Compose: Sidecars

            In K8s (or Pods in general) there is the "Sidecar"

            pattern:                                                                1 services:

                                                                                    2   app:

               •  Shared network namespace between containers                       3     image: nginx:alpine

               •  Can reach each other via localhost                                4     ports:

               •  Problem: No native Docker support for Pods                        5       - 8001:80   # app port

                  ◦  We can still share the network                                 6       - 8080:8080 # sidecar port

                  ◦  Ports must be exposed from main service                        7   sidecar:

                  ◦  Funny issues when starting/deleting containers                 8     image: nginxinc/nginx-unprivileged

                                                                                    9     entrypoint:

                                                                                   10       - sh

                                                                                   11       - -c

                                                                                   12       - sleep 1 && nginx -g "daemon off;"

                                                                                   13     network_mode: "service:app"

                                                 Software Engineering II: (Docker/Podman) Compose                                          17 / 23

                                      Compose: Healthcheck & Dependencies

                                                                       •  We can define a startup order between services

               1 services:                                                ◦  depends_on with multiple conditions

               2   app:                                                   ◦  service_running: Wait for service to start

               3     image: nginx:alpine                                  ◦  service_healthy: Wait for health check

               4     healthcheck:                                      •  Healthchecks...

               5       test: ["CMD", "curl","[::1]"]                      ◦  can be specified on image level

               6       interval: 3s                                       ◦  can be overridden by compose

               7       retries: 5                                         ◦  run inside the container

               8       start_period: 1s                                   ◦  use CMD or CMD-SHELL

               9       timeout: 10s                                          ▪  array or string notation

              10   dependend:

              11     image: nginx:alpine

              12     depends_on:

              13       app:

              14         condition: service_healthy

                                                 Software Engineering II: (Docker/Podman) Compose                                          18 / 23

                                                           Compose: Watch

                                                                    ██ Auto update builds

              1 services:

              2   app:                                                 •  For development, we can watch and reload project

              3     build: ./watch                                     •  Can be done for volume mounts (hot-reload)

              4     entrypoint: ["sleep", "9000"]                         ◦  action: sync

              5     develop:                                           •  Also for full container builds

              6       watch:                                              ◦  action: build

              7         - action: rebuild                              •  Use docker-compose up -w to watch

              8           path: ./watch/requirements.txt

                                                 Software Engineering II: (Docker/Podman) Compose                                          19 / 23

                                                         Compose: Anchors

                                                                       •  YAML anchors can be used to reuse config

               1 x-common-env: &common-env                             •  Can be placed anywhere in the file

               2   TZ: UTC                                                ◦  Use top-level x- attributes for config snippets (no invalid

               3   LOG_LEVEL: info                                           syntax errors)

               4                                                       •  Advantage: Ensure consistent config

               5 services:                                             •  Disadvantage: Harder to read

               6   web:

               7     environment:

               8       <<: *common-env

               9       HTTP_PORT: 8080

              10     volumes: &app-volumes

              11       ./data:/data:ro,z

              12   worker:

              13     environment: *common-env

              14     volumes: *app-volumes

                                                 Software Engineering II: (Docker/Podman) Compose                                          20 / 23

                                                        Compose: Profiles

                                                                    ██ Separating larger applications

               1 services:

               2   app:                                                •  Profiles can be used to divide complex setups

               3     image: python:alpine                              •  Each service can be added to multiple profiles

               4     profiles: ["app", "web"]                          •  Use docker-compose --profile db up to spawn db

               5   web:                                                   ◦  Use multiple --profile flags for web + app

               6     image: nginx:alpine                                  ◦  Or: Use COMPOSE_PROFILES=web,app environment

               7     profiles: ["web"]

               8   db:

               9     image: postgres:18-alpine

              10     profiles: ["app", "db"]

                                                 Software Engineering II: (Docker/Podman) Compose                                          21 / 23

                                           Compose: File based separation

            ██ Include

                                                                                   1 include:

               •  The include directive can be used to import content              2   - ./compose.simple.yml

                  from other compose files                                         3 services:

                  ◦  Simple, declarative way if files get too large                4   db:

                                                                                   5     image: postgres:alpine

            ██ Multiple files via CLI

                                                                                   file=../examples/compose/compose.include.yml

               •  We can also specify multiple files via CLI                       docker-compose -f $file config --services

               •  Useful if we use tooling around compose

                  ◦  E.g. conditionally include files

               •  docker-compose -f file1 -f file2                               ——————————————————— [finished] ———————————————————

               •  Each file overrides contents of previous files

                  ◦  Objects (lists, dicts) get merged

web

db

                                                 Software Engineering II: (Docker/Podman) Compose                                          22 / 23

            Thank you for your attention!

            Don't forget the feedback

                                                 Software Engineering II: (Docker/Podman) Compose                                          23 / 23