When prompted, enter the username and password associated with your Dockerhub account. Once the interactive job has begun, use podman to login to Dockerhub: podman login docker.io Submit the job to HTCondor using the -i flag to indicate that it will be an interactive build job:
The amount of disk used to build the container may be significantly larger than the total size of the container when it is finished. Building containers can use a surprising amount of disk, so if your build job continues to exit back to the submit node while the build is running, try increasing these values incrementally until the build succeeds. Some of the specified values, such as request_memory and request_disk might have to be updated to reflect the particular memory and disk requirements of your container. Transfer_input_files = requirements.txt, Dockerfile
Future updates to Colab may require an updated version of Python.Ĭreate a file called build.sub that looks like # Software build file Note that Python 3.7 is used in this example’s Dockerfile because at the time of writing, Colab runs on 3.7. If you know you require specific versions of CUDA/CuDNN, the base container can be adjusted accordingly. Using the Nvidia base container ensures that CUDA/CuDNN, two libraries often used by popular ML frameworks, are imported properly.
RUN apt-get update & apt-get install python3.7 The previous step to your home directory on the HTC system.Ĭopy the requirements.txt file to CHTC, and create a file called Dockerfile (there is no extension for this type of file) that looks like this: FROM nvidia/cuda:11.4.2-cudnn8-runtime-ubuntu20.04 Upload the requirements.txt file and the Python. If you don’t already have a Docker Hub account before starting this section, create one: Docker Hub The rest of the instructions occur on the CHTC system. py files in the directory, their imports will also be included in requirements.txt.ĭownload requirements.txt generated in the previous step and save it to the local directory containing the converted. Double check that the file contains the expected packages. This will generate a file named requirements.txt. These packages are determined by the program’s imports. Install and use pipreqs with the -use-local flag to generate a list of all python packages and versions used by Colab in the notebook. !jupyter nbconvert -to=python 'notebook_name.ipynb' Navigate to the working directory of the notebook: %cd 'path/to/directory'
Going from Google Colab to CHTC's GPU Lab