# Docker Run

Run a Docker container.

This feature requires runflow[docker].

$ pip install runflow[docker]

Added in v0.3.0.

# Example Usage

  • Set task type to "docker_run".
  • Set docker image.
  • Set the command.
# File: docker-hello-world.hcl
flow "docker-hello-world" {
  variable "out" {
    default = ""
  }

  task "docker_run" "echo" {
    image   = "ubuntu:latest"
    command = "echo hello world"
  }

  task "file_write" "hello-world" {
    filename = var.out
    content = task.docker_run.echo.stdout
  }
}

Run:

$ runflow run docker-hello-world.hcl --var out=/tmp/out.txt
[2021-06-12 15:14:01,654] Task "echo" is started.
[2021-06-12 15:14:02,158] Task "echo" is successful.
[2021-06-12 15:14:02,160] Task "save" is started.
[2021-06-12 15:14:02,234] Task "save" is successful.

$ cat /tmp/out.txt
hello world

# Set Environment Variables

  • Set argument environment to key-value pairs.
# File: docker-env.hcl
flow "docker-env" {
  variable "out" {
    default = ""
  }

  task "docker_run" "this" {
    image   = "ubuntu:latest"
    command = "env"
    environment = {
      "greeter": "world"
    }
  }

  task "file_write" "this" {
    filename = var.out
    content = task.docker_run.this.stdout
  }
}

Run:

$ runflow run docker-env.hcl --var out=/tmp/out.txt
[2021-06-12 15:24:08,870] Task "echo" is started.
[2021-06-12 15:24:09,399] Task "echo" is successful.
[2021-06-12 15:24:09,401] Task "save" is started.
[2021-06-12 15:24:09,415] Task "save" is successful.

$ cat /tmp/out.txt
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=9389736c56f1
greeter=world
HOME=/root

# Set Entrypoint

  • Set argument entrypoint to a list of strings.
# File: docker-entrypoint.hcl
flow "docker-entrypoint" {
  variable "out" {
    default = ""
  }

  task "docker_run" "this" {
    image       = "ubuntu:latest"
    entrypoint  = ["/bin/echo"]
    command     = "runflow is awesome"
  }

  task "file_write" "this" {
    filename    = var.out
    content     = task.docker_run.this.stdout
  }
}

Run:

$ runflow run docker-entrypoint.hcl --var out=/tmp/out.txt
[2021-06-12 15:37:25,390] Task "setup" is started.
[2021-06-12 15:37:25,903] Task "setup" is successful.
[2021-06-12 15:37:25,906] Task "save" is started.
[2021-06-12 15:37:25,921] Task "save" is successful.

$ cat /tmp/out.txt
runflow is awesome

# Failed Execution

When the docker container exits with non-zero code, the task run is marked as failed.

# File: docker-failed-run.hcl
flow "docker-failed-run" {
  task "docker_run" "exit" {
    image       = "ubuntu:latest"
    command     = "/bin/bash -c 'exit 1'"
  }
}

Run:

$ runflow run examples/docker-failed-run.hcl
[2021-06-12 15:41:02,979] Task "exit" is started.
[2021-06-12 15:41:03,496] Task "exit" is failed.
Traceback (most recent call last):
... (truncated)
docker.errors.ContainerError: Command "/bin/bash -c 'exit 1'" in image 'ubuntu:latest' returned non-zero exit status 1: b''

If you want the task to keep running, please wrap up your script to recover the error.

# Argument Reference

The following arguments are supported:

  • image (str) – The image to run.

  • command (str or list) – The command to run in the container.

  • auto_remove (bool) – enable auto-removal of the container on daemon side when the container’s process exits.

  • blkio_weight_device – Block IO weight (relative device weight) in the form of: [{"Path": "device_path", "Weight": weight}].

  • blkio_weight – Block IO weight (relative weight), accepts a weight value between 10 and 1000.

  • cap_add (list of str) – Add kernel capabilities. For example, ["SYS_ADMIN", "MKNOD"].

  • cap_drop (list of str) – Drop kernel capabilities.

  • cgroup_parent (str) – Override the default parent cgroup.

  • cpu_count (int) – Number of usable CPUs (Windows only).

  • cpu_percent (int) – Usable percentage of the available CPUs (Windows only).

  • cpu_period (int) – The length of a CPU period in microseconds.

  • cpu_quota (int) – Microseconds of CPU time that the container can get in a CPU period.

  • cpu_rt_period (int) – Limit CPU real-time period in microseconds.

  • cpu_rt_runtime (int) – Limit CPU real-time runtime in microseconds.

  • cpu_shares (int) – CPU shares (relative weight).

  • cpuset_cpus (str) – CPUs in which to allow execution (0-3, 0,1).

  • cpuset_mems (str) – Memory nodes (MEMs) in which to allow execution (0-3, 0,1). Only effective on NUMA systems.

  • detach (bool) – Run container in the background and return a Container object.

  • device_cgroup_rules (list) – A list of cgroup rules to apply to the container.

  • device_read_bps – Limit read rate (bytes per second) from a device in the form of: [{“Path”: “device_path”, “Rate”: rate}]

  • device_read_iops – Limit read rate (IO per second) from a device.

  • device_write_bps – Limit write rate (bytes per second) from a device.

  • device_write_iops – Limit write rate (IO per second) from a device.

  • devices (list) –

    Expose host devices to the container, as a list of strings in the form <path_on_host>:<path_in_container>:<cgroup_permissions>.

    For example, /dev/sda:/dev/xvda:rwm allows the container to have read-write access to the host’s /dev/sda via a node named /dev/xvda inside the container.

  • device_requests (list) – Expose host resources such as GPUs to the container, as a list of docker.types.DeviceRequest instances.

  • dns (list) – Set custom DNS servers.

  • dns_opt (list) – Additional options to be added to the container’s resolv.conf file.

  • dns_search (list) – DNS search domains.

  • domainname (str or list) – Set custom DNS search domains.

  • entrypoint (str or list) – The entrypoint for the container.

  • environment (map or list) – Environment variables to set inside the container, as a map or a list of strings in the format ["SOMEVARIABLE=xxx"].

  • extra_hosts (map) – Additional hostnames to resolve inside the container, as a mapping of hostname to IP address.

  • group_add (list) – List of additional group names and/or IDs that the container process will run as.

  • healthcheck (map) – Specify a test to perform to check that the container is healthy.

  • hostname (str) – Optional hostname for the container.

  • init (bool) – Run an init inside the container that forwards signals and reaps processes

  • init_path (str) – Path to the docker-init binary

  • ipc_mode (str) – Set the IPC mode for the container.

  • isolation (str) – Isolation technology to use. Default: None.

  • kernel_memory (int or str) – Kernel memory limit

  • labels (map or list) – A map of name-value labels (e.g. {"label1": "value1", "label2": "value2"}) or a list of names of labels to set with empty values (e.g. ["label1", "label2"])

  • links (map) – Mapping of links using the {"container": "alias"} format. The alias is optional. Containers declared in this map will be linked to the new container using the provided alias. Default: None.

  • log_config (LogConfig) – Logging configuration.

  • lxc_conf (map) – LXC config.

  • mac_address (str) – MAC address to assign to the container.

  • mem_limit (int or str) – Memory limit. Accepts float values (which represent the memory limit of the created container in bytes) or a string with a units identification char (100000b, 1000k, 128m, 1g). If a string is specified without a units character, bytes are assumed as an intended unit.

  • mem_reservation (int or str) – Memory soft limit.

  • mem_swappiness (int) – Tune a container’s memory swappiness behavior. Accepts number between 0 and 100.

  • memswap_limit (str or int) – Maximum amount of memory + swap a container is allowed to consume.

  • mounts (list) – Specification for mounts to be added to the container. More powerful alternative to volumes. Each item in the list is expected to be a docker.types.Mount object.

  • name (str) – The name for this container.

  • nano_cpus (int) – CPU quota in units of 1e-9 CPUs.

  • network (str) – Name of the network this container will be connected to at creation time. You can connect to additional networks using Network.connect(). Incompatible with network_mode.

  • network_disabled (bool) – Disable networking.

  • network_mode (str) –

    One of:

    • bridge Create a new network stack for the container on on the bridge network.
    • none No networking for this container.
    • container:<name|id> Reuse another container’s network stack.
    • host Use the host network stack. This mode is incompatible with ports.

    Incompatible with network.

  • oom_kill_disable (bool) – Whether to disable OOM killer.

  • oom_score_adj (int) – An integer value containing the score given to the container in order to tune OOM killer preferences.

  • pid_mode (str) – If set to host, use the host PID namespace inside the container.

  • pids_limit (int) – Tune a container’s pids limit. Set -1 for unlimited.

  • platform (str) – Platform in the format os[/arch[/variant]]. Only used if the method needs to pull the requested image.

  • ports (map) –

    Ports to bind inside the container.

    The keys of the map are the ports to bind inside the container, either as an integer or a string in the form port/protocol, where the protocol is either tcp, udp, or sctp.

    The values of the map are the corresponding ports to open on the host, which can be either:

    • The port number, as an integer. For example, {"2222/tcp": 3333} will expose port 2222 inside the container as port 3333 on the host.
    • None, to assign a random host port. For example, {"2222/tcp": None}.
    • A tuple of (address, port) if you want to specify the host interface. For example, {"1111/tcp": ("127.0.0.1", 1111)}.
    • A list of integers, if you want to bind multiple host ports to a single container port. For example, {"1111/tcp": [1234, 4567]}.

    Incompatible with host network mode.

  • privileged (bool) – Give extended privileges to this container.

  • publish_all_ports (bool) – Publish all ports to the host.

  • read_only (bool) – Mount the container’s root filesystem as read only.

  • remove (bool) – Remove the container when it has finished running. Default: False.

  • restart_policy (map) –

    Restart the container when it exits. Configured as a map with keys:

    • Name One of on-failure, or always.
    • MaximumRetryCount Number of times to restart the container on failure.

    For example: {"Name": "on-failure", "MaximumRetryCount": 5}

  • runtime (str) – Runtime to use with this container.

  • security_opt (list) – A list of string values to customize labels for MLS systems, such as SELinux.

  • shm_size (str or int) – Size of /dev/shm (e.g. 1G).

  • stdin_open (bool) – Keep STDIN open even if not attached.

  • stdout (bool) – Return logs from STDOUT when detach=False. Default: True.

  • stderr (bool) – Return logs from STDERR when detach=False. Default: False.

  • stop_signal (str) – The stop signal to use to stop the container (e.g. SIGINT).

  • storage_opt (map) – Storage driver options per container as a key-value mapping.

  • stream (bool) – If true and detach is false, return a log generator instead of a string. Ignored if detach is true. Default: False.

  • sysctls (map) – Kernel parameters to set in the container.

  • tmpfs (map) –

    Temporary filesystems to mount, as a map mapping a path inside the container to options for that path.

    For example:

    {
        "/mnt/vol2": "",
        "/mnt/vol1": "size=3G,uid=1000"
    }
    
  • tty (bool) – Allocate a pseudo-TTY.

  • ulimits (list) – Ulimits to set inside the container, as a list of docker.types.Ulimit instances.

  • use_config_proxy (bool) – If True, and if the docker client configuration file (~/.docker/config.json by default) contains a proxy configuration, the corresponding environment variables will be set in the container being built.

  • user (str or int) – Username or UID to run commands as inside the container.

  • userns_mode (str) – Sets the user namespace mode for the container when user namespace remapping option is enabled. Supported values are: host

  • uts_mode (str) – Sets the UTS namespace mode for the container. Supported values are: host

  • version (str) – The version of the API to use. Set to auto to automatically detect the server’s version. Default: 1.35

  • volume_driver (str) – The name of a volume driver/plugin.

  • volumes (map or list) –

    A map to configure volumes mounted inside the container. The key is either the host path or a volume name, and the value is a map with the keys:

    • bind The path to mount the volume inside the container
    • mode Either rw to mount the volume read/write, or ro to mount it read-only.

    For example:

     {"/home/user1/": {"bind": "/mnt/vol2", "mode": "rw"},
     "/var/www": {"bind": "/mnt/vol1", "mode": "ro"}}
    
  • volumes_from (list) – List of container names or IDs to get volumes from.

  • working_dir (str) – Path to the working directory.

# Attributes Reference

The following attributes are supported:

  • stdout - String. The output of command run.