Index of /examples/machine_learning/singularity

	Name	Last modified	Size	Description
	Parent Directory		-

Singularity Example

To run a Docker container, we use a similar software called Singularity. Singularity is more suitable than Docker for High Performance-Computing (HPC) clusters like the SCC. That is why we use it instead of Docker. It offers much the same functionality.

Set the environment variables

This is to ensure that the building of the container will not occur in your home directory. The building files consume a lot of space, typically. Since the home directory is limited, we don’t want to put those files there. Otherwise, you could run out of space.

1. To set the environment variables run the following commands:

cd /path/to/my/project/

mkdir .singularity

export SINGULARITY_CACHEDIR=/path/to/my/project/.singularity/

export SINGULARITY_TMPDIR=/path/to/my/project/.singularity/

export SINGULARITY_LOCALCACHEDIR=/path/to/my/project/.singularity/

Pull the Docker container

To pull a Docker container please follow these steps:

1. Find the link to the Docker container. For example, let’s consider if we are interested in NVIDIA pre-configured Docker containers for Machine Learning frameworks like TensorFlow.
   1. Go to the NVIDIA NGC containers website here (https://ngc.nvidia.com/catalog)
   2. Search for the container you’re interested in. For example, you will find the TensorFlow container here (https://ngc.nvidia.com/catalog/containers/nvidia:tensorflow)
   3. You will find the link listed in that webpage as nvcr.io/nvidia/tensorflow:21.06-tf1-py3
2. Add the prefix docker:// to the link. Thus, the link will be docker://nvcr.io/nvidia/tensorflow:21.06-tf1-py3
3. Use the command “singularity pull” followed by the complete link. The full command will be

singularity pull docker:// nvcr.io/nvidia/tensorflow:21.06-tf1-py3

The process of pulling the container and building it will take 15-20 minutes, depending on the container size. The output will be a .sif file.

Run the Singularity container

To run the container, use the following commands:

cd path/to/singularity/file

singularity shell --nv container_name.sif

The command singularity shell will put the user inside an interactive shell inside the container’s environment. The –-nv option gives the user GPU support inside the container. This is important for Deep Learning workloads if a GPU is required. Inside the shell you can run any command or program. For example, you can machine learning Python code You can run a python file using the command:

          python my_tensorflow_code.py

Contact Information

Help: help@scv.bu.edu
 
Note: RCS example programs are provided "as is" without any warranty of any kind. The user assumes the entire risk of quality, performance, and repair of any defect. You are welcome to copy and modify any of the given examples for your own use.