Docker refresh: simplified & update to 16.04, cuda8, cudnn5, nccl

Author: cypof

docker/Makefile

@@ -1,52 +1,48 @@
-# Caffe standalone Dockerfiles.
+### Running an official image
 
-The `standalone` subfolder contains docker files for generating both CPU and GPU executable images for Caffe. The images can be built using make, or by running:
+You can run one of the automatic [builds](https://hub.docker.com/r/bvlc/caffe)
+like this:
 
-```
-docker build -t caffe:cpu standalone/cpu
-```
-for example. (Here `gpu` can be substituted for `cpu`, but to keep the readme simple, only the `cpu` case will be discussed in detail).
+`docker run -ti bvlc/caffe caffe --version`
 
-Note that the GPU standalone requires a CUDA 7.5 capable driver to be installed on the system and [nvidia-docker] for running the Docker containers. Here it is generally sufficient to use `nvidia-docker` instead of `docker` in any of the commands mentioned.
+or for GPU support (You need a CUDA 8.0 capable driver and
+[nvidia-docker](https://github.com/NVIDIA/nvidia-docker)):
 
-# Running Caffe using the docker image
+`nvidia-docker run -ti bvlc/caffe:gpu caffe --version`
 
-In order to test the Caffe image, run:
-```
-docker run -ti caffe:cpu caffe --version
-```
-which should show a message like:
-```
-libdc1394 error: Failed to initialize libdc1394
-caffe version 1.0.0-rc3
-```
+You might see an error about libdc1394, ignore it.
 
-One can also build and run the Caffe tests in the image using:
-```
-docker run -ti caffe:cpu bash -c "cd /opt/caffe/build; make runtest"
-```
+### Docker run options
 
-In order to get the most out of the caffe image, some more advanced `docker run` options could be used. For example, running:
-```
-docker run -ti --volume=$(pwd):/workspace caffe:cpu caffe train --solver=example_solver.prototxt
-```
-will train a network defined in the `example_solver.prototxt` file in the current directory (`$(pwd)` is maped to the container volume `/workspace` using the `--volume=` Docker flag).
+By default caffe runs as root, thus any output files, e.g. snapshots, will be owned
+by root. It also runs by default in a container-private folder.
 
-Note that docker runs all commands as root by default, and thus any output files (e.g. snapshots) generated will be owned by the root user. In order to ensure that the current user is used instead, the following command can be used:
-```
-docker run -ti --volume=$(pwd):/workspace -u $(id -u):$(id -g) caffe:cpu caffe train --solver=example_solver.prototxt
-```
-where the `-u` Docker command line option runs the commands in the container as the specified user, and the shell command `id` is used to determine the user and group ID of the current user. Note that the Caffe docker images have `/workspace` defined as the default working directory. This can be overridden using the `--workdir=` Docker command line option.
+You can change this using flags, like user (-u), current directory, and volumes (-w and -v).
+E.g. this behaves like the usual caffe executable:
 
-# Other use-cases
+`docker run --rm -u $(id -u):$(id -g) -v $(pwd):$(pwd) -w $(pwd) bvlc/caffe caffe train --solver=example_solver.prototxt`
 
-Although running the `caffe` command in the docker containers as described above serves many purposes, the container can also be used for more interactive use cases. For example, specifying `bash` as the command instead of `caffe` yields a shell that can be used for interactive tasks. (Since the caffe build requirements are included in the container, this can also be used to build and run local versions of caffe).
+Containers can also be used interactively, specifying e.g. `bash` or `ipython`
+instead of `caffe`.
 
-Another use case is to run python scripts that depend on `caffe`'s Python modules. Using the `python` command instead of `bash` or `caffe` will allow this, and an interactive interpreter can be started by running:
 ```
-docker run -ti caffe:cpu python
+docker run -ti bvlc/caffe ipython
+import caffe
+...
 ```
-(`ipython` is also available in the container).
 
-Since the `caffe/python` folder is also added to the path, the utility executable scripts defined there can also be used as executables. This includes `draw_net.py`, `classify.py`, and `detect.py`
+The caffe build requirements are included in the container, so this can be used to
+build and run custom versions of caffe. Also, `caffe/python` is in PATH, so python
+utilities can be used directly, e.g. `draw_net.py`, `classify.py`, or `detect.py`.
+
+### Building images yourself
+
+Examples:
+
+`docker build -t caffe cpu`
+
+`docker build -t caffe:gpu gpu`
+
+You can also build Caffe and run the tests in the image:
 
+`docker run -ti caffe bash -c "cd /opt/caffe/build; make runtest"`

docker/README.md

@@ -1,5 +1,5 @@
-FROM ubuntu:14.04
-MAINTAINER [email protected]
+FROM ubuntu:16.04
+LABEL maintainer [email protected]
 
 RUN apt-get update && apt-get install -y --no-install-recommends \
         build-essential \
@@ -20,17 +20,19 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
         python-dev \
         python-numpy \
         python-pip \
+        python-setuptools \
         python-scipy && \
     rm -rf /var/lib/apt/lists/*
 
 ENV CAFFE_ROOT=/opt/caffe
 WORKDIR $CAFFE_ROOT
 
-# FIXME: clone a specific git tag and use ARG instead of ENV once DockerHub supports this.
-ENV CLONE_TAG=master
+# FIXME: use ARG instead of ENV once DockerHub supports this
+ENV CLONE_TAG=1.0.0-rc4
 
 RUN git clone -b ${CLONE_TAG} --depth 1 https://github.com/BVLC/caffe.git . && \
-    for req in $(cat python/requirements.txt) pydot; do pip install $req; done && \
+    pip install --upgrade pip && \
+    cd python && for req in $(cat requirements.txt) pydot; do pip install $req; done && cd .. && \
     mkdir build && cd build && \
     cmake -DCPU_ONLY=1 .. && \
     make -j"$(nproc)"

docker/standalone/cpu/Dockerfile renamed to docker/cpu/Dockerfile

@@ -1,5 +1,5 @@
-FROM nvidia/cuda:7.5-cudnn5-devel-ubuntu14.04
-MAINTAINER [email protected]
+FROM nvidia/cuda:8.0-cudnn5-devel-ubuntu16.04
+LABEL maintainer [email protected]
 
 RUN apt-get update && apt-get install -y --no-install-recommends \
         build-essential \
@@ -20,19 +20,22 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
         python-dev \
         python-numpy \
         python-pip \
+        python-setuptools \
         python-scipy && \
     rm -rf /var/lib/apt/lists/*
 
 ENV CAFFE_ROOT=/opt/caffe
 WORKDIR $CAFFE_ROOT
 
-# FIXME: clone a specific git tag and use ARG instead of ENV once DockerHub supports this.
-ENV CLONE_TAG=master
+# FIXME: use ARG instead of ENV once DockerHub supports this
+ENV CLONE_TAG=1.0.0-rc4
 
 RUN git clone -b ${CLONE_TAG} --depth 1 https://github.com/BVLC/caffe.git . && \
-    for req in $(cat python/requirements.txt) pydot; do pip install $req; done && \
+    pip install --upgrade pip && \
+    cd python && for req in $(cat requirements.txt) pydot; do pip install $req; done && cd .. && \
+    git clone https://github.com/NVIDIA/nccl.git && cd nccl && make -j install && cd .. && rm -rf nccl && \
     mkdir build && cd build && \
-    cmake -DUSE_CUDNN=1 .. && \
+    cmake -DUSE_CUDNN=1 -DUSE_NCCL=1 .. && \
     make -j"$(nproc)"
 
 ENV PYCAFFE_ROOT $CAFFE_ROOT/python