What is Docker Compose? A Complete Guide to Managing Multi-container Applications

The Problem with Running Multiple Containers

Imagine you're developing a complete web application with:

• Frontend: React running on port 3000

• Backend: Node.js API running on port 5000

• Database: PostgreSQL running on port 5432

• Cache: Redis running on port 6379

• Reverse Proxy: Nginx running on port 80

To run this application with plain Docker, you would need to:

bash

# Create network
docker network create my-app-network

# Run each container
docker run -d --name postgres --network my-app-network -e POSTGRES_PASSWORD=secret postgres:15
docker run -d --name redis --network my-app-network redis:7-alpine
docker run -d --name backend --network my-app-network -e DB_HOST=postgres my-backend:latest
docker run -d --name frontend --network my-app-network my-frontend:latest
docker run -d --name nginx --network my-app-network -p 80:80 my-nginx:latest

Problems:

• Must remember and type many long commands

• Difficult to manage startup order (database must run before backend)

• Hard to share configuration with the team

• No easy way to start/stop the entire application

Docker Compose was created to solve all these problems.

What is Docker Compose?

Docker Compose is a tool that allows you to define and run multi-container Docker applications. Instead of running individual docker run commands, you define the entire application in a single YAML file (docker-compose.yml), then use just one command to start everything.

yaml

# docker-compose.yml
services:
  frontend:
    image: my-frontend:latest
    ports:
      - "3000:3000"
  
  backend:
    image: my-backend:latest
    environment:
      - DB_HOST=postgres
    depends_on:
      - postgres
  
  postgres:
    image: postgres:15
    environment:
      - POSTGRES_PASSWORD=secret

bash

# Start the entire application
docker compose up -d

# Stop the entire application
docker compose down

One command replaces dozens of commands!

Why Use Docker Compose?

1. Simplify Multi-container Management

Instead of remembering dozens of Docker commands, you only need one configuration file and a few simple commands.

2. Infrastructure as Code (IaC)

The docker-compose.yml file is code, which can be:

• Stored in Git alongside source code

• Reviewed via Pull Requests

• Track change history

• Rollback when needed

3. Consistent Development Environment

Everyone on the team runs the same docker compose up command and gets an identical environment.

4. Easy Environment Recreation

Onboarding a new team member? Clone the repo and run docker compose up. Done.

5. Manage Dependencies Between Services

Docker Compose automatically handles startup order based on depends_on.

Installing Docker Compose

Docker Desktop (Windows/macOS)

Docker Compose is already integrated. No additional installation needed.

Linux

Method 1: Install via Docker Engine (recommended)

bash

# Docker Compose V2 is integrated into Docker CLI
sudo apt-get update
sudo apt-get install docker-compose-plugin

# Verify
docker compose version

Method 2: Install standalone

bash

# Download binary
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

# Make executable
sudo chmod +x /usr/local/bin/docker-compose

# Verify
docker-compose --version

Note on versions:

• V1 (legacy): Command is docker-compose (with hyphen)

• V2 (current): Command is docker compose (without hyphen)

This article uses V2 syntax.

docker-compose.yml File Structure

The docker-compose.yml file has this basic structure:

yaml

# Version (not required from Compose V2)
version: "3.9"

# Define services (containers)
services:
  service-name:
    # Service configuration
    ...

# Define networks (optional)
networks:
  network-name:
    # Network configuration
    ...

# Define volumes (optional)
volumes:
  volume-name:
    # Volume configuration
    ...

# Define configs (optional)
configs:
  config-name:
    # Config configuration
    ...

# Define secrets (optional)
secrets:
  secret-name:
    # Secret configuration
    ...

Services — The Core Component

Services is where you define the containers to run. Each service corresponds to one container.

Using Existing Images

yaml

services:
  database:
    image: postgres:15-alpine
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: admin
      POSTGRES_PASSWORD: secret

Building Images from Dockerfile

yaml

services:
  backend:
    build:
      context: ./backend          # Directory containing Dockerfile
      dockerfile: Dockerfile      # Dockerfile name (default is Dockerfile)
      args:                       # Build arguments
        NODE_ENV: production
    image: my-backend:latest      # Name for the built image

Shorthand syntax:

yaml

services:
  backend:
    build: ./backend    # If Dockerfile is at ./backend/Dockerfile

Essential Service Directives

image — Specify Docker Image

yaml

services:
  web:
    image: nginx:1.25-alpine

build — Build image from Dockerfile

yaml

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile.prod
      args:
        - BUILD_ENV=production
      target: production          # Multi-stage build target

container_name — Set container name

yaml

services:
  web:
    container_name: my-web-app    # Instead of auto-generated name

ports — Map ports

yaml

services:
  web:
    ports:
      - "80:80"           # HOST:CONTAINER
      - "443:443"
      - "3000-3005:3000-3005"   # Range of ports
      - "127.0.0.1:8080:80"    # Bind to localhost only

expose — Expose internal port (not mapped to host)

yaml

services:
  backend:
    expose:
      - "5000"    # Only other services in the same network can access

environment — Environment variables

yaml

services:
  app:
    environment:
      - NODE_ENV=production
      - DB_HOST=postgres
      - DB_PORT=5432
    # Or object format
    environment:
      NODE_ENV: production
      DB_HOST: postgres

env_file — Load environment variables from file

yaml

services:
  app:
    env_file:
      - .env
      - .env.local

# .env
NODE_ENV=production
DB_HOST=postgres
DB_PASSWORD=supersecret

volumes — Mount volumes

yaml

services:
  app:
    volumes:
      # Named volume
      - db-data:/var/lib/postgresql/data
      
      # Bind mount (host path : container path)
      - ./src:/app/src
      
      # Read-only bind mount
      - ./config:/app/config:ro
      
      # Anonymous volume
      - /app/node_modules

volumes:
  db-data:    # Declare named volume

networks — Connect to networks

yaml

services:
  frontend:
    networks:
      - frontend-network
  
  backend:
    networks:
      - frontend-network
      - backend-network
  
  database:
    networks:
      - backend-network

networks:
  frontend-network:
  backend-network:

depends_on — Define dependencies

yaml

services:
  backend:
    depends_on:
      - postgres
      - redis
  
  postgres:
    image: postgres:15
  
  redis:
    image: redis:7

Note: depends_on only ensures containers start in order, not that the service inside is ready. To check service readiness, use healthcheck:

yaml

services:
  backend:
    depends_on:
      postgres:
        condition: service_healthy
  
  postgres:
    image: postgres:15
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5

restart — Restart policy

yaml

services:
  web:
    restart: "no"              # Don't restart (default)
    restart: always            # Always restart
    restart: on-failure        # Restart when exit code is non-zero
    restart: unless-stopped    # Restart unless manually stopped

command — Override CMD in Dockerfile

yaml

services:
  app:
    image: node:18
    command: npm run dev
    # Or array format
    command: ["npm", "run", "dev"]

entrypoint — Override ENTRYPOINT in Dockerfile

yaml

services:
  app:
    entrypoint: /app/docker-entrypoint.sh

working_dir — Working directory

yaml

services:
  app:
    working_dir: /app

user — Run as specific user

yaml

services:
  app:
    user: "1000:1000"    # UID:GID

healthcheck — Health check

yaml

services:
  web:
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

deploy — Deployment configuration (Swarm mode)

yaml

services:
  web:
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: "0.5"
          memory: 512M
        reservations:
          cpus: "0.25"
          memory: 256M

logging — Logging configuration

yaml

services:
  app:
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

Networks — Connecting Services

By default, Docker Compose creates a network for all services in the file. Services can call each other by service name.

Default network

yaml

services:
  backend:
    image: my-backend
    # Can call database using hostname "postgres"
  
  postgres:
    image: postgres:15

Custom networks

yaml

services:
  frontend:
    networks:
      - public
  
  backend:
    networks:
      - public
      - private
  
  database:
    networks:
      - private

networks:
  public:
    driver: bridge
  private:
    driver: bridge
    internal: true    # No internet access

Using existing networks (external)

yaml

networks:
  existing-network:
    external: true
    name: my-existing-network

Volumes — Data Storage

Named volumes

yaml

services:
  database:
    volumes:
      - postgres-data:/var/lib/postgresql/data

volumes:
  postgres-data:
    driver: local

Bind mounts

yaml

services:
  app:
    volumes:
      - ./src:/app/src           # Development: sync code
      - ./logs:/app/logs         # Share logs with host

External volumes

yaml

volumes:
  shared-data:
    external: true
    name: my-shared-volume

Common Docker Compose Commands

Starting services

bash

# Start all services
docker compose up

# Run in background (detached mode)
docker compose up -d

# Start only specific services
docker compose up -d backend postgres

# Rebuild images before starting
docker compose up -d --build

# Force recreate containers
docker compose up -d --force-recreate

Stopping services

bash

# Stop all services
docker compose stop

# Stop and remove containers, networks
docker compose down

# Also remove volumes
docker compose down -v

# Also remove images
docker compose down --rmi all

Viewing status

bash

# List running containers
docker compose ps

# List all containers (including stopped)
docker compose ps -a

Viewing logs

bash

# Logs for all services
docker compose logs

# Logs for specific service
docker compose logs backend

# Follow logs (real-time)
docker compose logs -f

# Limit number of lines
docker compose logs --tail=100

Executing commands in containers

bash

# Run command in running container
docker compose exec backend bash
docker compose exec postgres psql -U postgres

# Run command in new container
docker compose run --rm backend npm test

Building images

bash

# Build all services
docker compose build

# Build specific service
docker compose build backend

# Build without cache
docker compose build --no-cache

Other commands

bash

# View merged configuration
docker compose config

# Pull latest images
docker compose pull

# Restart services
docker compose restart

# Scale services (run multiple instances)
docker compose up -d --scale backend=3

Environment Variables and Interpolation

Using variables in docker-compose.yml

yaml

services:
  app:
    image: myapp:${TAG:-latest}    # Default is "latest"
    environment:
      - DB_HOST=${DB_HOST}
      - DB_PASSWORD=${DB_PASSWORD:?DB_PASSWORD is required}  # Required

.env file

Docker Compose automatically reads the .env file in the same directory:

# .env
TAG=v1.2.3
DB_HOST=postgres
DB_PASSWORD=supersecret

Using multiple Compose files

bash

# Override configuration
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

yaml

# docker-compose.yml (base)
services:
  app:
    build: .
    environment:
      - NODE_ENV=development

# docker-compose.prod.yml (override)
services:
  app:
    environment:
      - NODE_ENV=production
    restart: always

Real-world Examples

Example 1: MERN Stack (MongoDB, Express, React, Node.js)

yaml

version: "3.9"

services:
  # Frontend - React
  frontend:
    build: ./frontend
    ports:
      - "3000:3000"
    volumes:
      - ./frontend/src:/app/src    # Hot reload
    environment:
      - REACT_APP_API_URL=http://localhost:5000
    depends_on:
      - backend

  # Backend - Node.js/Express
  backend:
    build: ./backend
    ports:
      - "5000:5000"
    volumes:
      - ./backend/src:/app/src
    environment:
      - NODE_ENV=development
      - MONGO_URI=mongodb://mongo:27017/myapp
      - REDIS_URL=redis://redis:6379
    depends_on:
      - mongo
      - redis

  # Database - MongoDB
  mongo:
    image: mongo:7
    volumes:
      - mongo-data:/data/db
    environment:
      - MONGO_INITDB_ROOT_USERNAME=admin
      - MONGO_INITDB_ROOT_PASSWORD=secret

  # Cache - Redis
  redis:
    image: redis:7-alpine
    volumes:
      - redis-data:/data

  # Admin UI - Mongo Express
  mongo-express:
    image: mongo-express
    ports:
      - "8081:8081"
    environment:
      - ME_CONFIG_MONGODB_ADMINUSERNAME=admin
      - ME_CONFIG_MONGODB_ADMINPASSWORD=secret
      - ME_CONFIG_MONGODB_URL=mongodb://admin:secret@mongo:27017/
    depends_on:
      - mongo

volumes:
  mongo-data:
  redis-data:

Example 2: Laravel + MySQL + Nginx

yaml

version: "3.9"

services:
  # Nginx
  nginx:
    image: nginx:1.25-alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
      - ./src:/var/www/html:ro
    depends_on:
      - php

  # PHP-FPM
  php:
    build:
      context: ./docker/php
      dockerfile: Dockerfile
    volumes:
      - ./src:/var/www/html
    environment:
      - DB_CONNECTION=mysql
      - DB_HOST=mysql
      - DB_PORT=3306
      - DB_DATABASE=laravel
      - DB_USERNAME=laravel
      - DB_PASSWORD=secret
    depends_on:
      mysql:
        condition: service_healthy

  # MySQL
  mysql:
    image: mysql:8.0
    volumes:
      - mysql-data:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD=rootsecret
      - MYSQL_DATABASE=laravel
      - MYSQL_USER=laravel
      - MYSQL_PASSWORD=secret
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      timeout: 5s
      retries: 5

  # Redis
  redis:
    image: redis:7-alpine
    volumes:
      - redis-data:/data

  # Queue Worker
  queue:
    build:
      context: ./docker/php
    volumes:
      - ./src:/var/www/html
    command: php artisan queue:work
    depends_on:
      - php
      - redis

volumes:
  mysql-data:
  redis-data:

Example 3: Microservices with API Gateway

yaml

version: "3.9"

services:
  # API Gateway - Nginx
  gateway:
    image: nginx:1.25-alpine
    ports:
      - "80:80"
    volumes:
      - ./gateway/nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - user-service
      - product-service
      - order-service

  # User Service
  user-service:
    build: ./services/user
    expose:
      - "3001"
    environment:
      - DATABASE_URL=postgresql://postgres:secret@user-db:5432/users
    depends_on:
      - user-db
    networks:
      - frontend
      - user-network

  user-db:
    image: postgres:15-alpine
    environment:
      - POSTGRES_DB=users
      - POSTGRES_PASSWORD=secret
    volumes:
      - user-db-data:/var/lib/postgresql/data
    networks:
      - user-network

  # Product Service
  product-service:
    build: ./services/product
    expose:
      - "3002"
    environment:
      - MONGO_URI=mongodb://product-db:27017/products
    depends_on:
      - product-db
    networks:
      - frontend
      - product-network

  product-db:
    image: mongo:7
    volumes:
      - product-db-data:/data/db
    networks:
      - product-network

  # Order Service
  order-service:
    build: ./services/order
    expose:
      - "3003"
    environment:
      - DATABASE_URL=postgresql://postgres:secret@order-db:5432/orders
      - USER_SERVICE_URL=http://user-service:3001
      - PRODUCT_SERVICE_URL=http://product-service:3002
    depends_on:
      - order-db
      - user-service
      - product-service
    networks:
      - frontend
      - order-network

  order-db:
    image: postgres:15-alpine
    environment:
      - POSTGRES_DB=orders
      - POSTGRES_PASSWORD=secret
    volumes:
      - order-db-data:/var/lib/postgresql/data
    networks:
      - order-network

  # Message Queue - RabbitMQ
  rabbitmq:
    image: rabbitmq:3-management-alpine
    ports:
      - "15672:15672"    # Management UI
    environment:
      - RABBITMQ_DEFAULT_USER=admin
      - RABBITMQ_DEFAULT_PASS=secret
    networks:
      - frontend

networks:
  frontend:
  user-network:
  product-network:
  order-network:

volumes:
  user-db-data:
  product-db-data:
  order-db-data:

Profiles — Run Services by Group

Profiles allow you to define service groups and run them only when needed:

yaml

services:
  frontend:
    image: my-frontend
    profiles:
      - frontend
  
  backend:
    image: my-backend
    # No profile = always runs
  
  debug-tools:
    image: debug-tools
    profiles:
      - debug
  
  test-runner:
    image: test-runner
    profiles:
      - test

bash

# Only run services without profiles
docker compose up -d

# Run with frontend profile
docker compose --profile frontend up -d

# Run multiple profiles
docker compose --profile frontend --profile debug up -d

Best Practices

1. Use .env for secrets

yaml

# ❌ Not recommended - hardcoded password
services:
  db:
    environment:
      - POSTGRES_PASSWORD=mysecretpassword

# ✅ Recommended - use environment variables
services:
  db:
    environment:
      - POSTGRES_PASSWORD=${DB_PASSWORD}

# .env (don't commit to git)
DB_PASSWORD=mysecretpassword

2. Use meaningful names for volumes and networks

yaml

# ✅ Meaningful names
volumes:
  postgres-data:
  redis-cache:
  app-uploads:

networks:
  frontend-network:
  backend-network:

3. Use depends_on with healthcheck

yaml

services:
  app:
    depends_on:
      db:
        condition: service_healthy
  
  db:
    healthcheck:
      test: ["CMD", "pg_isready"]
      interval: 5s
      timeout: 5s
      retries: 5

4. Separate files for different environments

├── docker-compose.yml          # Base configuration
├── docker-compose.override.yml # Development (auto-loaded)
├── docker-compose.prod.yml     # Production
└── docker-compose.test.yml     # Testing

bash

# Development (automatically loads override)
docker compose up -d

# Production
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

5. Limit resources in production

yaml

services:
  app:
    deploy:
      resources:
        limits:
          cpus: "1"
          memory: 1G
        reservations:
          cpus: "0.5"
          memory: 512M

6. Use restart policy

yaml

services:
  app:
    restart: unless-stopped
  
  db:
    restart: always

7. Configure logging

yaml

services:
  app:
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "5"

8. Use .dockerignore

Ensure you have a .dockerignore file to avoid copying unnecessary files into the image.

Common Errors and Solutions

Error 1: Port already in use

Error: Bind for 0.0.0.0:3000 failed: port is already allocated

Solution:

bash

# Find process using the port
lsof -i :3000

# Kill the process or change port in docker-compose.yml

Error 2: Volume permission denied

Permission denied: '/var/lib/postgresql/data'

Solution:

yaml

services:
  db:
    user: "1000:1000"    # Or
    volumes:
      - ./data:/var/lib/postgresql/data

bash

# Or fix permissions on host
sudo chown -R 1000:1000 ./data

Error 3: Container exits immediately after starting

Solution:

bash

# Check logs for the cause
docker compose logs service-name

# Check exit code
docker compose ps -a

Error 4: Services can't connect to each other

Solution:

yaml

# Ensure same network
services:
  app:
    networks:
      - backend
  db:
    networks:
      - backend

networks:
  backend:

bash

# Use service name as hostname
# app connects to db using hostname "db", not "localhost"

Error 5: Changes not being applied

bash

# Rebuild and recreate
docker compose up -d --build --force-recreate

# Or down then up again
docker compose down
docker compose up -d --build

Conclusion

Docker Compose is an essential tool when working with multi-container applications. It helps:

• Simplify management of multiple containers

• Standardize development environment for the entire team

• Automate dependency and networking setup

• Document infrastructure as code

Start with simple examples, then gradually apply advanced features like profiles, healthchecks, and multi-file configuration to build professional development and production environments.

References

1. Docker Compose Overview: https://docs.docker.com/compose/

2. Compose File Reference: https://docs.docker.com/compose/compose-file/

3. Compose CLI Reference: https://docs.docker.com/compose/reference/

4. Networking in Compose: https://docs.docker.com/compose/networking/

5. Environment Variables in Compose: https://docs.docker.com/compose/environment-variables/