Optimizing Storage

Docker stores images and containers in a graph driver (a pluggable storage backend), such as Device Mapper, Overlay, and Btrfs. Each have advantages and disadvantages. For example, Overlay is faster than Device Mapper at starting and stopping containers, but is not POSIX compliant because of the architectural limitations of a union file system, and does not yet support SELinux.

For more information about Overlay, including supportability and usage caveats, see the RHEL 7.3 Release Notes.

In production environments, using a LVM thin pool on top of regular block devices (not loop devices) for container images and container root file systems storage is recommended.

Using a Loop device back-end can affect performance issues. While you can still continue to use it, Docker logs a warning message. For example:

devmapper: Usage of loopback devices is strongly discouraged for production use.
Please use `--storage-opt dm.thinpooldev` or use `man docker` to refer to
dm.thinpooldev section.

To ease Docker backend storage configuration, use the docker-storage-setup utility, which automates much of the configuration details:

  1. If you had a separate disk drive dedicated to Docker storage (for example, /dev/xvdb), add the following to the /etc/sysconfig/docker-storage-setup file:

    DEVS=/dev/xvdb
    VG=docker_vg
  2. Restart the docker-storage-setup service:

    # systemctl restart docker-storage-setup

    After the restart, docker-storage-setup sets up a volume group named docker_vg and creates a thin pool logical volume. Documentation for thin provisioning on RHEL is available in the LVM Administrator Guide. View the newly created volumes with the lsblk command:

    # lsblk /dev/xvdb
    NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
    xvdb 202:16 0 20G 0 disk
    └─xvdb1 202:17 0 10G 0 part
      ├─docker_vg-docker--pool_tmeta 253:0 0 12M 0 lvm
      │ └─docker_vg-docker--pool 253:2 0 6.9G 0 lvm
      └─docker_vg-docker--pool_tdata 253:1 0 6.9G 0 lvm
      └─docker_vg-docker--pool 253:2 0 6.9G 0 lvm

    Thin-provisioned volumes are not mounted and have no file system (individual containers do have an XFS file system), thus they will not show up in “df” output.

  3. To verify that Docker is using a LVM thin pool, and to monitor disk space utilization, use the docker info command. The Pool Name will correspond with the VG you specified in /etc/sysconfig/docker-storage-setup:

    # docker info | egrep -i 'storage|pool|space|filesystem'
    Storage Driver: devicemapper
     Pool Name: docker_vg-docker--pool
     Pool Blocksize: 524.3 kB
     Backing Filesystem: xfs
     Data Space Used: 62.39 MB
     Data Space Total: 6.434 GB
     Data Space Available: 6.372 GB
     Metadata Space Used: 40.96 kB
     Metadata Space Total: 16.78 MB
     Metadata Space Available: 16.74 MB

By default, a thin pool is configured to use 40% of the underlying block device. As you use the storage, LVM automatically extends the thin pool up to 100%. This is why the Data Space Total value does not match the full size of the underlying LVM device. This auto-extend technique was used to unify the storage approach taken in both Red Hat Enterprise Linux and Red Hat Atomic Host, which only uses a single partition.

In development, Docker in Red Hat distributions defaults to a loopback mounted sparse file. To see if your system is using the loopback mode:

# docker info|grep loop0
 Data file: /dev/loop0
refarch-feedback@redhat.com 16 www.redhat.com

Red Hat strongly recommends using the Device Mapper storage driver in thin pool mode for production workloads.

Overlay is also supported for Docker use cases as of Red Hat Enterprise Linux 7.2, and provides faster start up time and page cache sharing, which can potentially improve density by reducing overall memory utilization.