Attach an Existing ZFS Pool to the Compose Runtime

Introduction

Some edge nodes already hold application data in a ZFS pool that was created by an earlier application instance, for example a virtual machine that ran Kubernetes with OpenEBS storage. When you move those workloads to Docker Compose, the data needs to come with them.

The ZFS variant of the Compose Runtime solves this. It attaches the existing ZFS pool volume as a second drive on the runtime, imports the pool when the runtime boots, discovers the datasets inside it, and mounts each one at a stable, friendly path inside the runtime. Docker Compose applications then bind those paths as ordinary volumes. The compose file never has to name the dataset's real identifier, which on Kubernetes-managed pools is often an opaque string such as pvc- followed by a UUID.

This article covers the three pieces of configuration the ZFS variant adds on top of the standard Compose Runtime: the second drive, the two configuration files in the runtime's custom configuration template, and the volume binding in each compose app. For everything else, the ZFS variant deploys exactly like the standard one.

NOTE: This capability is designed for migrating existing ZFS-backed workloads to Docker Compose on an edge node where the data already lives. If you are starting a new deployment and do not have an existing pool, use the standard Compose Runtime variant. Contact your ZEDEDA account team before adopting the ZFS variant for a new deployment.

This is a series of articles. You will likely follow them in this order.

1. Docker Compose Overview.
2. Deploy the Compose Runtime Application.
3. Attach an Existing ZFS Pool to the Compose Runtime (you are here).
4. Compose App Instance Control Overview.
5. Add a Docker Compose as an Edge App Type.
6. Troubleshooting Docker Compose.

How It Works

When the ZFS Compose Runtime boots, a service named zrun-zvol-mount runs before the Docker Engine starts. It reads two files that are written by the runtime's custom configuration template:

• /etc/zededa/zvol-mounts.sh names the pool to import and declares a list of logical dataset names. For each logical name, it defines a small detection function that inspects a dataset's contents and returns success when the dataset matches. This lets the runtime identify "the database dataset" or "the configuration dataset" by what is inside it rather than by its identifier, so the same template works across edge nodes where every dataset has a different identifier.
• /etc/zededa/zvol-mounts.conf sets the permission mode and numeric owner for each logical name's mount point.

The service imports the pool, mounts it under /imported_zpool_mount_root, runs the detection functions against a read-only staged copy of each dataset, and bind-mounts each matched dataset at /imported_zfs_vols/<logical name>. Compose apps reference that path.

As shipped, the pool name in zvol-mounts.sh is empty. With an empty pool name the service does nothing and exits successfully, so a ZFS runtime with an unedited template behaves like a standard runtime. You enable the feature by filling in the pool name and dataset configuration.

You configure the template once per edge app. Every runtime instance deployed from that edge app, on any edge node, uses the same detection logic.

Prerequisites

• A Compose Runtime edge app of the ZFS variant (zed-compose-runtime_<size>-zfs) imported into your local marketplace. See Deploy the Compose Runtime Application.
• EVE-OS (Edge Virtualization Engine) 14.5.0 LTS or greater on the target edge node. If the edge node uses ZFS persistent storage, a 16.0.x LTS release of EVE-OS or greater is required.
• The ZFS pool volume instance already exists on the target edge node. It was created by an earlier application instance and marked persistent, so it survived that instance's removal. You know the volume's Name, which you use as its label.
• The earlier application instance that owned the pool is stopped or deleted. A ZFS pool can be imported by only one consumer at a time.
• You know the ZFS pool name as it appears inside the pool itself (for example, zfspoolname), and you know at least one file or directory that uniquely identifies each dataset you want to mount.
• You have the SysManager role in your enterprise.

Add the ZFS Pool Volume as a Second Drive

The runtime edge app needs a second drive that points at the existing volume by label. Open the imported ZFS runtime edge app in the Marketplace, duplicate it, and edit the copy.

1. In the edge app editor, go to the Drives section.
2. Confirm the first drive references the Compose Runtime image.
3. Select Add Drive.
4. In the second drive, leave the Edge App Image empty and set the Volume Label to the Name of the existing ZFS pool volume instance. The label is how EVE-OS maps this drive to the volume instance that already exists on the edge node. A drive with a volume label and no image attaches the edge node's existing volume instead of creating a new one.
5. Set Max Size to the size of the existing volume.
6. Set Purge so that the volume is persistent. A label-attached drive is never purged, so the volume continues to belong to the edge node and is not deleted if the runtime instance is removed. See Add Persistent Volume Instances.
7. Select Next and continue to the custom configuration template.

WARNING: The volume must exist on the edge node before the runtime instance starts. If the label does not match a volume instance on the target edge node, the runtime instance fails to start. Verify the volume under Library > Volume Instances before deploying.

Configure Pool Import and Dataset Discovery

The custom configuration template on the ZFS variant already contains both files. You edit them in place in the template editor. Do not remove or reorder other parts of the template.

Edit zvol-mounts.sh

Locate the write_files entry whose path is /etc/zededa/zvol-mounts.sh. Its content defines four variables and a set of detection functions.

For every name in ZFS_LOGICAL_DATASET_NAMES, define a shell function named zfs_dataset_detect_<logical name>. The runtime calls each function once per dataset in the pool, passing three arguments: the logical name, the dataset name, and the path to a read-only staged copy of the dataset. The function must return 0 when the dataset matches and 1 otherwise. Use only simple file-system tests against the staged path. Each function runs under a timeout.

The following example declares two logical names and identifies each dataset by a file that exists only in that dataset.

ZPOOL_IMPORT_NAME="zfspoolname"
ZPOOL_MOUNT_ROOT="/imported_zpool_mount_root"
ZFS_VOL_ROOT="/imported_zfs_vols"
ZFS_LOGICAL_DATASET_NAMES="example_ds1,example_ds2"

zfs_dataset_detect_example_ds1() {
  [ -f "$3/root/example_ds1" ] && return 0
  return 1
}

zfs_dataset_detect_example_ds2() {
  [ -f "$3/data/example_ds2" ] && return 0
  return 1
}

Replace the pool name, logical names, and detection tests with values that match your data. Be specific in each test. A test that matches more than one dataset causes the wrong dataset to be mounted under that logical name. A logical name with no matching function is a configuration error, and the service reports it rather than skipping it.

Edit zvol-mounts.conf

Locate the write_files entry whose path is /etc/zededa/zvol-mounts.conf. This file sets permissions on each mount point. Permissions apply to the mount point only, not recursively to the files inside.

Each line has the format:

<logical name> [mode] [uid:gid] [options]

The following example matches the two logical names from the preceding section. The first mount is owned by root. The second is owned by UID and GID 999, which is a common default for database containers.

example_ds1 0770 0:0
example_ds2 0770 999:999

Blank lines and lines beginning with # are ignored.

Save the edge app

1. Continue through the remaining sections of the edge app editor.
2. Review the configuration. Confirm that two drives are listed and that the custom configuration template contains your edited zvol-mounts.sh and zvol-mounts.conf entries.
3. Select Add (or Save if editing an existing edge app).

Deploy the runtime as described in Deploy the Compose Runtime Application.

Bind an Imported Dataset in a Compose App

After the runtime is online, any Docker Compose application on that edge node can mount an imported dataset by binding its friendly path. Add a volumes entry to the service in your compose.yaml:

services:
  app:
    image: your-registry.example.com/your-app:latest
    volumes:
      - /imported_zfs_vols/example_ds1:/data

The left side is the path inside the runtime, formed from ZFS_VOL_ROOT and the logical name. The right side is the path inside the container. The container reads and writes the dataset directly.

If the container runs as a non-root user, make sure the uid:gid you set in zvol-mounts.conf for that logical name matches the container's user. Otherwise the container can see the mount but cannot write to it.

For the rest of the compose app definition, see Add a Docker Compose as an Edge App Type.

Verify the Configuration

1. Select Edge App Instances in the left panel.
2. Confirm the Compose Runtime instance shows a run state of Online.
3. Connect to the runtime over SSH as described in Troubleshooting Docker Compose.
4. Run ls /imported_zfs_vols/. Each logical name you declared should appear as a directory.
5. Run df -h /imported_zfs_vols/<logical name> for one of the mounts. The size and usage should reflect the dataset, not the runtime's own disk.
6. Deploy a compose app that binds one of the paths, write a test file from inside the container, and confirm the file appears under /imported_zfs_vols/<logical name> on the runtime.

If a logical name is missing from /imported_zfs_vols/, check the runtime logs for messages from zrun-zvol-mount. The most common causes are a pool name that does not match, a detection function that matched no dataset, or a logical name with no detection function.

Next Steps

This is a series of articles. You will likely follow them in this order.

1. Docker Compose Overview.
2. Deploy the Compose Runtime Application.
3. Attach an Existing ZFS Pool to the Compose Runtime (you are here).
4. Compose App Instance Control Overview.
5. Add a Docker Compose as an Edge App Type.
6. Troubleshooting Docker Compose.