6.1. wc_env_manager package

6.1.2. Submodules

6.1.3. wc_env_manager.__main__ module

wc_env_manager command line interface

Author:Arthur Goldberg <Arthur.Goldberg@mssm.edu>
Date:2018-04-04
Copyright:2018, Karr Lab
License:MIT
class wc_env_manager.__main__.AllController(*args, **kw)[source]

Bases: cement.ext.ext_argparse.ArgparseController

Build, push, pull, and remove images and containers

class Meta[source]

Bases: object

arguments = [][source]
description = 'Build, push, pull, and remove images and containers'[source]
label = 'all'[source]
stacked_on = 'base'[source]
stacked_type = 'embedded'[source]
build()[source]
pull()[source]
push()[source]
remove()[source]
class wc_env_manager.__main__.App(label=None, **kw)[source]

Bases: cement.core.foundation.App

Command line application

class Meta[source]

Bases: object

base_controller = 'base'[source]
handlers = [<class 'wc_env_manager.__main__.BaseController'>, <class 'wc_env_manager.__main__.BaseImageController'>, <class 'wc_env_manager.__main__.ImageController'>, <class 'wc_env_manager.__main__.NetworkController'>, <class 'wc_env_manager.__main__.ContainerController'>, <class 'wc_env_manager.__main__.AllController'>][source]
label = 'wc_env_manager'[source]
class wc_env_manager.__main__.BaseController(*args, **kw)[source]

Bases: cement.ext.ext_argparse.ArgparseController

Base controller for command line application

class Meta[source]

Bases: object

arguments = [(['-v', '--version'], {'action': 'version', 'version': '0.0.1'})][source]
description = 'Whole-cell modeling environment manager'[source]
label = 'base'[source]
class wc_env_manager.__main__.BaseImageController(*args, **kw)[source]

Bases: cement.ext.ext_argparse.ArgparseController

Build, push, and pull the base image, wc_env_dependencies

class Meta[source]

Bases: object

arguments = [][source]
description = 'Build, push, pull, and remove the base image, `wc_env_dependencies`'[source]
label = 'base-image'[source]
stacked_on = 'base'[source]
stacked_type = 'nested'[source]
build()[source]
pull()[source]
push()[source]
remove()[source]
version()[source]
class wc_env_manager.__main__.ContainerController(*args, **kw)[source]

Bases: cement.ext.ext_argparse.ArgparseController

Build and remove containers of wc_env

class Meta[source]

Bases: object

arguments = [][source]
description = 'Build and remove containers of `wc_env`'[source]
label = 'container'[source]
stacked_on = 'base'[source]
stacked_type = 'nested'[source]
build()[source]
remove()[source]
class wc_env_manager.__main__.ImageController(*args, **kw)[source]

Bases: cement.ext.ext_argparse.ArgparseController

Build, push, and pull the image, wc_env

class Meta[source]

Bases: object

arguments = [][source]
description = 'Build, push, pull, and remove the image, `wc_env`'[source]
label = 'image'[source]
stacked_on = 'base'[source]
stacked_type = 'nested'[source]
build()[source]
pull()[source]
push()[source]
remove()[source]
version()[source]
class wc_env_manager.__main__.NetworkController(*args, **kw)[source]

Bases: cement.ext.ext_argparse.ArgparseController

Build and remove a Docker network

class Meta[source]

Bases: object

arguments = [][source]
description = 'Build and remove a Docker network'[source]
label = 'network'[source]
stacked_on = 'base'[source]
stacked_type = 'nested'[source]
build()[source]
remove()[source]
wc_env_manager.__main__.main()[source]

6.1.4. wc_env_manager.core module

Tools for managing computing environments for whole-cell modeling

  • Build the Docker image

    • wc_env: image with WC models and WC modeling tools and their dependencies
    • wc_env_dependencies: base image with third party dependencies
  • Remove the Docker images

  • Push/pull the Docker images

  • Create Docker containers

    1. Mount host directories into container
    2. Copy files (such as configuration files and authentication keys into container
    3. Install GitHub SSH key
    4. Verify access to GitHub
    5. Install Python packages in mounted directories from host
  • Copy files to/from Docker container

  • List Docker containers of the image

  • Get CPU, memory, network usage statistics of Docker containers

  • Stop Docker containers

  • Remove Docker containers

  • Login to DockerHub

Author:Jonathan Karr <jonrkarr@gmail.com>
Author:Arthur Goldberg <Arthur.Goldberg@mssm.edu>
Date:2018-08-23
Copyright:2018, Karr Lab
License:MIT
class wc_env_manager.core.WcEnvManager(config=None)[source]

Bases: object

Manage computing environments (Docker containers) for whole-cell modeling

config[source]

Dictionary of configuration options. See wc_env_manager/config/core.schema.cfg.

Type:configobj.ConfigObj
_docker_client[source]

client connected to the Docker daemon

Type:docker.client.DockerClient
_base_image_unsquashed[source]

unsquasehd version of the current base Docker image

Type:docker.models.images.Image
_base_image[source]

current base Docker image

Type:docker.models.images.Image
_image[source]

current Docker image

Type:docker.models.images.Image
_container[source]

current Docker container

Type:docker.models.containers.Container
build_base_image()[source]

Build base Docker image for WC modeling environment

Before executing this method, you must download CPLEX and obtain licenses for Gurobi, MINOS, Mosek, and XPRESS. See the documentation <building_images> for more information.

Returns:Docker image
Return type:docker.models.images.Image
build_container(tty=True)[source]

Create Docker container for WC modeling environmet

Parameters:tty (bool) – if True, allocate a pseudo-TTY
Returns:Docker container
Return type:docker.models.containers.Container
build_image()[source]

Build Docker image for WC modeling environment

Returns:Docker image
Return type:docker.models.images.Image
Raises:WcEnvManagerError – if a copied configuration file clashes with
build_network()[source]

Create Docker network

copy_path_from_container(container_path, local_path, overwrite=True)[source]

Copy file/directory from Docker container

Implemented using subprocess because docker-py does not (as 2018-08-22) provide a copy method.

Parameters:
  • container_path (str) – path to file/directory within container
  • local_path (str) – local path to copy file/directory from container
  • overwrite (bool, optional) – if True, overwrite file
Raises:

WcEnvManagerError – if the container_path already exists and overwrite is False

copy_path_to_container(local_path, container_path, overwrite=True, container_user=<WcEnvUser.root: 0>)[source]

Copy file or directory to Docker container

Implemented using subprocess because docker-py does not (as 2018-08-22) provide a copy method.

Parameters:
  • local_path (str) – path to local file/directory to copy to container
  • container_path (str) – path to copy file/directory within container
  • overwrite (bool, optional) – if True, overwrite file
Raises:

WcEnvManagerError – if the container_path already exists and overwrite is False

get_config_file_paths_to_copy_to_image()[source]

Get list of configuration file paths to copy from ~/.wc to Docker image

Returns:configuration file paths to copy from ~/.wc to Docker image
Return type:list of dict
get_container_stats()[source]

Get statistics about the CPU, io, memory, network performance of the Docker container

Returns:statistics about the CPU, io, memory, network performance of the Docker container
Return type:dict
get_containers(sort_by_read_time=False)[source]

Get list of Docker containers that are WC modeling environments

Parameters:sort_by_read_time (bool) – if True, sort by read time in descending order (latest first)
Returns:
list of Docker containers
that are WC modeling environments
Return type:list of docker.models.containers.Container
get_image_version(image)[source]

Get the version of the Docker image

Parameters:image (docker.models.images.Image) – image
Returns:docker image version
Return type:str
get_latest_container()[source]

Get current Docker container

Returns:Docker container
Return type:docker.models.containers.Container
get_latest_image(image_repo)[source]

Get the lastest version of the Docker image for the WC modeling environment

Parameters:image_repo (str) – image repository
Returns:Docker image
Return type:docker.models.images.Image
get_required_python_packages()[source]
Get Python packages required for the WC models and WC modeling
tools (config[‘image’][‘python_packages’])
Returns:
list of Python requirements in
requirements.txt format
Return type:list of str
login_docker_hub()[source]

Login to DockerHub

make_container_name()[source]

Create a timestamped name for a Docker container

Returns:container name
Return type:str
pull_image(image_repo, image_tags)[source]

Pull Docker image for WC modeling environment

Parameters:
  • image_repo (str) – image repository
  • image_tags (list of str) – list of tags
Returns:

Docker image

Return type:

docker.models.images.Image

push_image(image_repo, image_tags)[source]

Push Docker image to DockerHub

Parameters:
  • image_repo (str) – image repository
  • image_tags (list of str) – list of tags
remove_container(force=False)[source]

Remove current Docker container

Parameters:force (bool, optional) – if True, force removal of the container (e.g. remove container even if it is running)
remove_containers(force=False)[source]

Remove Docker all containers that are WC modeling environments

Parameters:force (bool, optional) – if True, force removal of the container (e.g. remove containers even if they are running)
remove_image(image_repo, image_tags, force=False)[source]

Remove version of Docker image

Parameters:
  • image_repo (str) – image repository
  • image_tags (list of str) – list of tags
  • force (bool, optional) – if True, force removal of the version of the image (e.g. even if a container with the image is running)
remove_network()[source]

Remove Docker network

run_process_in_container(cmd, work_dir=None, env=None, check=True, container_user=<WcEnvUser.root: 0>)[source]

Run a process in the current Docker container

Parameters:
  • cmd (list of str or str) – command to run
  • work_dir (str, optional) – path to working directory within container
  • env (dict, optional) – key/value pairs of environment variables
  • check (bool, optional) – if True, raise exception if exit code is not 0
  • container_user (WcEnvUser, optional) – user to run commands in container
Returns:

output of the process

Return type:

str

Raises:

WcEnvManagerError – if the command is not executed successfully

run_process_on_host(cmd)[source]

Run a process on the host

Parameters:cmd (list of str or str) – command to run
set_container(container)[source]

Set the Docker containaer

Parameters:container (docker.models.containers.Container or str) – Docker container or name of Docker container
set_image(image_repo, image)[source]

Set the Docker image for WC modeling environment

Parameters:
  • image_repo (str) – image repository
  • image (docker.models.images.Image or str) – Docker image or name of Docker image
setup_container(upgrade=False, process_dependency_links=True)[source]

Install Python packages into Docker container

Parameters:
  • upgrade (bool, optional) – if True, upgrade package
  • process_dependency_links (bool, optional) – if True, install packages from provided URLs
stop_container()[source]

Remove current Docker container

exception wc_env_manager.core.WcEnvManagerError(message=None)[source]

Bases: Exception

Base class for exceptions in wc_env_manager

message[source]

the exception’s message

Type:str
class wc_env_manager.core.WcEnvUser[source]

Bases: enum.Enum

WC environment users and their ids

container_user = 999[source]
root = 0[source]

6.1.5. Module contents