The DevOps Series Using Docker with Ansible

Tagged: ,

This topic contains 1 reply, has 2 voices, and was last updated by  Parison 4 months, 2 weeks ago.

  • Author
    Posts
  • #19920
     gracylayla 
    Participant

    Docker provides operating system level virtualisation in the form of containers. These containers allow you to run standalone applications in an isolated environment. The three important features of Docker containers are isolation, portability and repeatability. All along we have used Parabola GNU/Linux-libre as the host system, and executed Ansible scripts on target virtual machines (VM) such as CentOS and Ubuntu.

    Docker containers are extremely lightweight and fast to launch. You can also specify the amount of resources that you need such as the CPU, memory and network. The Docker technology was launched in 2013, and released under the Apache 2.0 licence. It is implemented using the Go programming language. A number of frameworks have been built on top of Docker for managing these clusters of servers. The Apache Mesos project, Google’s Kubernetes, and the Docker Swarm project are popular examples. These are ideal for running stateless applications and help you to easily scale horizontally.

    Setting it up

    The Ansible version used on the host system (Parabola GNU/Linux-libre x86_64) is 2.3.0.0. Internet access should be available on the host system. The ansible/ folder contains the following file:

    ansible/playbooks/configuration/docker.yml
    Installation

    The following playbook is used to install Docker on the host system:

    ---
     
    - name: Setup Docker
     
    hosts: localhost
     
    gather_facts: true
     
    become: true
     
    tags: [setup]
     
    tasks:
     
    - name: Update the software package repository
     
    pacman:
     
    update_cache: yes
     
    - name: Install dependencies
     
    package:
     
    name: “{{ item }}”
     
    state: latest
     
    with_items:
     
    - python2-docker
     
    - docker
     
    - service:
     
    name: docker
     
    state: started
     
    - name: Run the hello-world container
     
    docker_container:
     
    name: hello-world
     
    image: library/hello-world

    The Parabola package repository is updated before proceeding to install the dependencies. The python2-docker package is required for use with Ansible. Hence, it is installed along with the docker package. The Docker daemon service is then started and the library/hello-world container is fetched and executed. A sample invocation and execution of the above playbook is shown below:

    $ ansible-playbook playbooks/configuration/docker.yml -K –tags=setup

    SUDO password:

    PLAY [Setup Docker] *****************************************

    TASK [Gathering Facts] **************************************

    ok: [localhost]

    TASK [Update the software package repository] ***************

    changed: [localhost]

    TASK [Install dependencies] *********************************

    ok: [localhost] => (item=python2-docker)

    ok: [localhost] => (item=docker)

    TASK [service] **********************************************

    ok: [localhost]

    TASK [Run the hello-world container] ************************

    changed: [localhost]

    PLAY RECAP **************************************************

    localhost : ok=5 changed=2 unreachable=0 failed=0
    With the verbose ‘-v’ option to ansible-playbook, you will see an entry for LogPath, such as /var/lib/docker/containers/<container-id>/<container-id>-json.log. In this log file, you will see the output of the execution of the hello-world container. This output is the same when you run the container manually as shown below:

    $ sudo docker run hello-world

    Hello from Docker!
    This message shows that your installation appears to be working correctly.

    To generate this message, Docker took the following steps:

    1. The Docker client contacted the Docker daemon.

    2. The Docker daemon pulled the hello-world image from the Docker Hub.

    3. The Docker daemon created a new container from that image, which runs the executable that produces the output you are currently reading.

    4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal.

    To try something more ambitious, you can run an Ubuntu container with:

    $ docker run -it ubuntu bash
    You can share images, automate workflows, and more with a free Docker ID at https://cloud.docker.com/.

    An example

    A deep learning (DL) Docker project is available with support for frameworks, libraries and software tools. We can use Ansible to build the entire DL container from the source code of the tools. The base OS of the container is Ubuntu 14.04, and will include the following software packages:

    TensorFlow
    Caffe
    Theano
    Keras
    Lasagne
    Torch
    iPython/Jupyter Notebook
    Numpy
    SciPy
    Pandas
    Scikit Learn
    Matplotlib
    OpenCV

    The playbook to build the DL Docker image is given below:

    – name: Build the dl-docker image

    hosts: localhost

    gather_facts: true

    become: true

    tags: [deep-learning]

    vars:

    DL_BUILD_DIR: “/tmp/dl-docker”

    DL_DOCKER_NAME: “floydhub/dl-docker”

    tasks:

    – name: Download dl-docker

    git:

    repo: https://github.com/saiprashanths/dl-docker.git

    dest: “{{ DL_BUILD_DIR }}”

    – name: Build image and with buildargs

    docker_image:

    path: “{{ DL_BUILD_DIR }}”

    name: “{{ DL_DOCKER_NAME }}”

    dockerfile: Dockerfile.cpu

    buildargs:

    tag: “{{ DL_DOCKER_NAME }}:cpu”

    We first clone the deep learning Docker project sources. The docker_image module in Ansible helps us to build, load and pull images. We then use the Dockerfile.cpu file to build a Docker image targeting the CPU. If you have a GPU in your system, you can use the Dockerfile.gpu file. The above playbook can be invoked using the following command:

    $ ansible-playbook playbooks/configuration/docker.yml -K –tags=deep-learning
    Depending on the CPU and RAM you have, it will take a considerable amount of time to build the image with all the software. So be patient!

    Jupyter Notebook

    The built dl-docker image contains Jupyter Notebook, which can be launched when you start the container. An Ansible playbook for the same is provided below:

    - name: Start Jupyter notebook
     
    hosts: localhost
     
    gather_facts: true
     
    become: true
     
    tags: [notebook]
     
    vars:
     
    DL_DOCKER_NAME: “floydhub/dl-docker”
     
    tasks:
     
    - name: Run container for Jupyter notebook
     
    docker_container:
     
    name: “dl-docker-notebook”
     
    image: “{{ DL_DOCKER_NAME }}:cpu”
     
    state: started
     
    command: sh run_jupyter.sh

    You can invoke the playbook using the following command:

    1. $ ansible-playbook playbooks/configuration/docker.yml -K --tags=notebook

    The Dockerfile already exposes the port 8888, and hence you do not need to specify the same in the above docker_container configuration. After you run the playbook, using the ‘docker ps’ command on the host system, you can obtain the container ID as indicated below:

    $ sudo docker ps

    CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

    a876ad5af751 floydhub/dl-docker:cpu “sh run_jupyter.sh” 11 minutes ago Up 4 minutes 6006/tcp, 8888/tcp dl-docker-notebook
    You can now log in to the running container using the following command:

    $ sudo docker exec -it a876 /bin/bash
    You can then run an ‘ifconfig’ command to find the local IP address (‘172.17.0.2’ in this case), and then open http://172.17.0.2:8888 in a browser on your host system to see the Jupyter Notebook. A screenshot is shown in Figure 1.

    TensorBoard

    TensorBoard consists of a suite of visualisation tools to understand the TensorFlow programs. It is installed and available inside the Docker container. After you log in to the Docker container, at the root prompt, you can start TensorBoard by passing it a log directory as shown below:

    # tensorboard –logdir=./log
    You can then open http://172.17.0.2:6006/ in a browser on your host system to see the TensorBoard dashboard as shown in Figure 2.

    Docker image facts

    The docker_image_facts Ansible module provides useful information about a Docker image. We can use it to obtain the image facts for our dl-docker container as shown below:

    – name: Get Docker image facts

    hosts: localhost

    gather_facts: true

    become: true

    tags: [facts]

    vars:

    DL_DOCKER_NAME: “floydhub/dl-docker”

    tasks:

    – name: Get image facts

    docker_image_facts:

    name: “{{ DL_DOCKER_NAME }}:cpu”
    The above playbook can be invoked as follows:

    $ ANSIBLE_STDOUT_CALLBACK=json ansible-playbook playbooks/configuration/docker.yml -K –tags=facts
    The ANSIBLE_STDOUT_CALLBACK environment variable is set to ‘json’ to produce a JSON output for readability. Some important image facts from the invocation of the above playbook are shown below:

    “Architecture”: “amd64”,
     
    “Author”: “Sai Soundararaj <saip@outlook.com>”,
     
    “Config”: {
     
    “Cmd”: [
     
    “/bin/bash”
     
    ],
     
    “Env”: [
     
    “PATH=/root/torch/install/bin:/root/caffe/build/tools:/root/caffe/python:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin”,
     
    “CAFFE_ROOT=/root/caffe”,
     
    “PYCAFFE_ROOT=/root/caffe/python”,
     
    “PYTHONPATH=/root/caffe/python:”,
     
    “LUA_PATH=/root/.luarocks/share/lua/5.1/?.lua;/root/.luarocks/share/lua/5.1/?/init.lua;/root/torch/install/share/lua/5.1/?.lua;/root/torch/install/share/lua/5.1/?/init.lua;./?.lua;/root/torch/install/share/luajit-2.1.0-beta1/?.lua;/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua”,
     
    “LUA_CPATH=/root/torch/install/lib/?.so;/root/.luarocks/lib/lua/5.1/?.so;/root/torch/install/lib/lua/5.1/?.so;./?.so;/usr/local/lib/lua/5.1/?.so;/usr/local/lib/lua/5.1/loadall.so”,
     
    “LD_LIBRARY_PATH=/root/torch/install/lib:”,
     
    “DYLD_LIBRARY_PATH=/root/torch/install/lib:”
     
    ],
     
    “ExposedPorts”: {
     
    “6006/tcp”: {},
     
    “8888/tcp”: {}
     
    },
     
    “Created”: “2016-06-13T18:13:17.247218209Z”,
     
    “DockerVersion”: “1.11.1”,
     
    “Os”: “linux”,
     
    “task”: { “name”: “You aGet image facts” }
    • This topic was modified 1 year, 8 months ago by  gracylayla.
    #21177
     Parison 
    Participant

    Thanks! Good explanation! Is it interesting for you to participate in DevOps training?

Viewing 2 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.

©2019 ForgeRock - we provide an identity and access platform to secure every online relationship for the enterprise market, educational sector and even entire countries. Click to view our privacy policy and terms of use.

Log in with your credentials

Forgot your details?