6.1. wc_env_manager package

6.1.1. Subpackages

• 6.1.1.1. wc_env_manager.config package
  • 6.1.1.1.1. Submodules
  • 6.1.1.1.2. wc_env_manager.config.core module
  • 6.1.1.1.3. Module contents

6.1.2. Submodules

6.1.3. wc_env_manager.__main__ module

wc_env_manager command line interface

Author

Arthur Goldberg <Arthur.Goldberg@mssm.edu>

Date

2018-04-04

Copyright

2018, Karr Lab

License

MIT

class wc_env_manager.__main__.AllController(*args, **kw)

Bases: cement.ext.ext_argparse.ArgparseController

Build, push, pull, and remove images and containers

class Meta

Bases: object

arguments = []

description = 'Build, push, pull, and remove images and containers'

help = 'Build, push, pull, and remove images and containers'

label = 'all'

stacked_on = 'base'

stacked_type = 'embedded'

build()

pull()

push()

remove()

class wc_env_manager.__main__.App(label=None, **kw)

Bases: cement.core.foundation.App

Command line application

class Meta

Bases: object

base_controller = 'base'

handlers = [<class 'wc_env_manager.__main__.BaseController'>, <class 'wc_env_manager.__main__.BaseImageController'>, <class 'wc_env_manager.__main__.ImageController'>, <class 'wc_env_manager.__main__.NetworkController'>, <class 'wc_env_manager.__main__.ContainerController'>, <class 'wc_env_manager.__main__.AllController'>]

label = 'wc-env-manager'

class wc_env_manager.__main__.BaseController(*args, **kw)

Bases: cement.ext.ext_argparse.ArgparseController

Base controller for command line application

class Meta

Bases: object

arguments = [(['-v', '--version'], {'action': 'version', 'version': '0.0.1'})]

description = 'Whole-cell modeling environment manager'

help = 'Whole-cell modeling environment manager'

label = 'base'

class wc_env_manager.__main__.BaseImageController(*args, **kw)

Bases: cement.ext.ext_argparse.ArgparseController

Build, push, and pull the base image, wc_env_dependencies

class Meta

Bases: object

arguments = []

description = 'Build, push, pull, and remove the base image, `wc_env_dependencies`'

help = 'Build, push, pull, and remove the base image, `wc_env_dependencies`'

label = 'base-image'

stacked_on = 'base'

stacked_type = 'nested'

build()

pull()

push()

remove()

version()

class wc_env_manager.__main__.ContainerController(*args, **kw)

Bases: cement.ext.ext_argparse.ArgparseController

Build and remove containers of wc_env

class Meta

Bases: object

arguments = []

description = 'Build and remove containers of `wc_env`'

help = 'Build and remove containers of `wc_env`'

label = 'container'

stacked_on = 'base'

stacked_type = 'nested'

build()

remove()

class wc_env_manager.__main__.ImageController(*args, **kw)

Bases: cement.ext.ext_argparse.ArgparseController

Build, push, and pull the image, wc_env

class Meta

Bases: object

arguments = []

description = 'Build, push, pull, and remove the image, `wc_env`'

help = 'Build, push, pull, and remove the image, `wc_env`'

label = 'image'

stacked_on = 'base'

stacked_type = 'nested'

build()

pull()

push()

remove()

version()

class wc_env_manager.__main__.NetworkController(*args, **kw)

Bases: cement.ext.ext_argparse.ArgparseController

Build and remove a Docker network

class Meta

Bases: object

arguments = []

description = 'Build and remove a Docker network'

help = 'Build and remove a Docker network'

label = 'network'

stacked_on = 'base'

stacked_type = 'nested'

build()

remove()

wc_env_manager.__main__.main()

6.1.4. wc_env_manager._version module

6.1.5. wc_env_manager.core module

Tools for managing computing environments for whole-cell modeling

• Build the Docker image

  • wc_env: image with WC models and WC modeling tools and their dependencies

  • wc_env_dependencies: base image with third party dependencies

• Remove the Docker images

• Push/pull the Docker images

• Create Docker containers

  1. Mount host directories into container

  2. Copy files (such as configuration files and authentication keys) into container

  3. Install GitHub SSH key

  4. Verify access to GitHub

  5. Install Python packages in mounted directories from host

• Copy files to/from Docker container

• List Docker containers of the image

• Get CPU, memory, network usage statistics of Docker containers

• Stop Docker containers

• Remove Docker containers

• Login to DockerHub

Author

Jonathan Karr <jonrkarr@gmail.com>

Author

Arthur Goldberg <Arthur.Goldberg@mssm.edu>

Date

2018-08-23

Copyright

2018, Karr Lab

License

MIT

class wc_env_manager.core.WcEnvManager(config=None)

Bases: object

Manage computing environments (Docker containers) for whole-cell modeling

config

Dictionary of configuration options. See wc_env_manager/config/core.schema.cfg.

Type

configobj.ConfigObj

_docker_client

client connected to the Docker daemon

Type

docker.client.DockerClient

_base_image_unsquashed

unsquasehd version of the current base Docker image

Type

docker.models.images.Image

_base_image

current base Docker image

Type

docker.models.images.Image

_image

current Docker image

Type

docker.models.images.Image

_container

current Docker container

Type

docker.models.containers.Container

IMAGE_OS_SEP = '/'

build_base_image()

Build base Docker image for WC modeling environment

Before executing this method, you must download CPLEX and obtain licenses for Gurobi, MINOS, Mosek, and XPRESS. See the documentation <building_images> for more information.

Returns

Docker image

Return type

docker.models.images.Image

build_container(tty=True)

Create Docker container for WC modeling environmet

Parameters

tty (bool) – if True, allocate a pseudo-TTY

Returns

Docker container

Return type

docker.models.containers.Container

build_image()

Build Docker image for WC modeling environment

Returns

Docker image

Return type

docker.models.images.Image

Raises

WcEnvManagerError – if a copied configuration file clashes with

build_network()

Create Docker network

copy_path_from_container(container_path, local_path, overwrite=True)

Copy file/directory from Docker container

Implemented using subprocess because docker-py does not (as 2018-08-22) provide a copy method.

Parameters

• container_path (str) – path to file/directory within container

• local_path (str) – local path to copy file/directory from container

• overwrite (bool, optional) – if True, overwrite file

Raises

WcEnvManagerError – if the container_path already exists and overwrite is False

copy_path_to_container(local_path, container_path, overwrite=True, container_user=<WcEnvUser.root: 0>)

Copy file or directory to Docker container

Implemented using subprocess because docker-py does not (as 2018-08-22) provide a copy method.

Parameters

• local_path (str) – path to local file/directory to copy to container

• container_path (str) – path to copy file/directory within container

• overwrite (bool, optional) – if True, overwrite file

Raises

WcEnvManagerError – if the container_path already exists and overwrite is False

get_config_file_paths_to_copy_to_image()

Get list of configuration file paths to copy from ~/.wc to Docker image

Returns

configuration file paths to copy from ~/.wc to Docker image

Return type

list of dict

get_container_stats()

Get statistics about the CPU, io, memory, network performance of the Docker container

Returns

statistics about the CPU, io, memory, network performance of the Docker container

Return type

dict

get_containers(sort_by_read_time=False)

Get list of Docker containers that are WC modeling environments

Parameters

sort_by_read_time (bool) – if True, sort by read time in descending order (latest first)

Returns

list of Docker containers

that are WC modeling environments

Return type

list of docker.models.containers.Container

get_image_version(image)

Get the version of the Docker image

Parameters

image (docker.models.images.Image) – image

Returns

docker image version

Return type

str

get_latest_container()

Get current Docker container

Returns

Docker container

Return type

docker.models.containers.Container

get_latest_image(image_repo)

Get the lastest version of the Docker image for the WC modeling environment

Parameters

image_repo (str) – image repository

Returns

Docker image

Return type

docker.models.images.Image

get_required_python_packages()

Get Python packages required for the WC models and WC modeling

tools (config[‘image’][‘python_packages’])

Returns

list of Python requirements in

requirements.txt format

Return type

list of str

login_docker_hub()

Login to DockerHub

make_container_name()

Create a timestamped name for a Docker container

Returns

container name

Return type

str

pull_image(image_repo, image_tags)

Pull Docker image for WC modeling environment

Parameters

• image_repo (str) – image repository

• image_tags (list of str) – list of tags

Returns

Docker image

Return type

docker.models.images.Image

push_image(image_repo, image_tags)

Push Docker image to DockerHub

Parameters

• image_repo (str) – image repository

• image_tags (list of str) – list of tags

remove_container(force=False)

Remove current Docker container

Parameters

force (bool, optional) – if True, force removal of the container (e.g. remove container even if it is running)

remove_containers(force=False)

Remove Docker all containers that are WC modeling environments

Parameters

force (bool, optional) – if True, force removal of the container (e.g. remove containers even if they are running)

remove_image(image_repo, image_tags, force=False)

Remove version of Docker image

Parameters

• image_repo (str) – image repository

• image_tags (list of str) – list of tags

• force (bool, optional) – if True, force removal of the version of the image (e.g. even if a container with the image is running)

remove_network()

Remove Docker network

run_process_in_container(cmd, work_dir=None, env=None, check=True, container_user=<WcEnvUser.root: 0>)

Run a process in the current Docker container

Parameters

• cmd (list of str or str) – command to run

• work_dir (str, optional) – path to working directory within container

• env (dict, optional) – key/value pairs of environment variables

• check (bool, optional) – if True, raise exception if exit code is not 0

• container_user (WcEnvUser, optional) – user to run commands in container

Returns

output of the process

Return type

str

Raises

WcEnvManagerError – if the command is not executed successfully

run_process_on_host(cmd)

Run a process on the host

Parameters

cmd (list of str or str) – command to run

set_container(container)

Set the Docker containaer

Parameters

container (docker.models.containers.Container or str) – Docker container or name of Docker container

set_image(image_repo, image)

Set the Docker image for WC modeling environment

Parameters

• image_repo (str) – image repository

• image (docker.models.images.Image or str) – Docker image or name of Docker image

setup_container(upgrade=False)

Install Python packages into Docker container

Parameters

upgrade (bool, optional) – if True, upgrade package

stop_container()

Remove current Docker container

exception wc_env_manager.core.WcEnvManagerError(message=None)

Bases: Exception

Base class for exceptions in wc_env_manager

message

the exception’s message

Type

str

class wc_env_manager.core.WcEnvUser

Bases: enum.Enum

WC environment users and their ids

container_user = 999

root = 0

6.1.6. Module contents