Alphafoldは docker にて稼働させてますが、もうひとつのコンテナ技術 Singularity で行う場合のお話
dockerを使わない alphafold_non_docker 版もあるhttps://github.com/kalininalab/alphafold_non_docker

一応https://cloud.sylabs.io/library/prehensilecode/alphafold_singularity/alphafoldにて
alphafold :2.2.4 の Singularity イメージが配布されているが、ここでは一から作ってみている.

dockerにはレシピのDockerfileがあって、Singularity にもdefファイルがある.
両者を比べてみたのが下記になります.

左は alphafld のDokerfile、右は alphafold_singularity のSingularity.def

alphafoldhttps://github.com/deepmind/alphafold
alphafold_singularityhttps://github.com/prehensilecode/alphafold_singularity

# Copyright 2021 DeepMind Technologies Limited                                                      Bootstrap: docker
#                                                                                                   From: nvidia/cuda:11.1.1-cudnn8-runtime-ubuntu18.04
# Licensed under the Apache License, Version 2.0 (the "License");                                   Stage: spython-base
# you may not use this file except in compliance with the License.                                  
# You may obtain a copy of the License at                                                           %files
#                                                                                                   . /app/alphafold
#      http://www.apache.org/licenses/LICENSE-2.0                                                   %post
#                                                                                                   # Copyright 2021 DeepMind Technologies Limited
# Unless required by applicable law or agreed to in writing, software                               #
# distributed under the License is distributed on an "AS IS" BASIS,                                 # Licensed under the Apache License, Version 2.0 (the "License");
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                          # you may not use this file except in compliance with the License.
# See the License for the specific language governing permissions and                               # You may obtain a copy of the License at
# limitations under the License.                                                                    #
                                                                                                    #      http://www.apache.org/licenses/LICENSE-2.0
ARG CUDA=11.1.1                                                                                     #
FROM nvidia/cuda:${CUDA}-cudnn8-runtime-ubuntu18.04                                                 # Unless required by applicable law or agreed to in writing, software
# FROM directive resets ARGS, so we specify again (the value is retained if                         # distributed under the License is distributed on an "AS IS" BASIS,
# previously set).                                                                                  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
ARG CUDA                                                                                            # See the License for the specific language governing permissions and
                                                                                                    # limitations under the License.
# Use bash to support string substitution.                                                          
SHELL ["/bin/bash", "-o", "pipefail", "-c"]                                                         # FROM directive resets ARGS, so we specify again (the value is retained if
                                                                                                    # previously set).
RUN apt-get update \                                                                                
    && DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \                  apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
        build-essential \                                                                           build-essential \
        cmake \                                                                                     cmake \
        cuda-command-line-tools-$(cut -f1,2 -d- <<< ${CUDA//./-}) \                                 cuda-command-line-tools-11-1 \
        git \                                                                                       git \
        hmmer \                                                                                     hmmer \
        kalign \                                                                                    kalign \
        tzdata \                                                                                    tzdata \
        wget \                                                                                      wget \
    && rm -rf /var/lib/apt/lists/* \                                                                && rm -rf /var/lib/apt/lists/* \
    && apt-get autoremove -y \                                                                      && apt-get autoremove -y \
    && apt-get clean                                                                                && apt-get clean
                                                                                                    
# Compile HHsuite from source.                                                                      # Compile HHsuite from source.
RUN git clone --branch v3.3.0 https://github.com/soedinglab/hh-suite.git /tmp/hh-suite \            /bin/rm -rf /tmp/hh-suite \
    && mkdir /tmp/hh-suite/build \                                                                  && git clone --branch v3.3.0 https://github.com/soedinglab/hh-suite.git /tmp/hh-suite \
    && pushd /tmp/hh-suite/build \                                                                  && mkdir /tmp/hh-suite/build \
    && cmake -DCMAKE_INSTALL_PREFIX=/opt/hhsuite .. \                                               && cd /tmp/hh-suite/build \
    && make -j 4 && make install \                                                                  && cmake -DCMAKE_INSTALL_PREFIX=/opt/hhsuite .. \
    && ln -s /opt/hhsuite/bin/* /usr/bin \                                                          && make -j 4 && make install \
    && popd \                                                                                       && ln -s /opt/hhsuite/bin/* /usr/bin \
    && rm -rf /tmp/hh-suite                                                                         && cd / \
                                                                                                    && /bin/rm -rf /tmp/hh-suite
# Install Miniconda package manager.                                                                
RUN wget -q -P /tmp \                                                                               # Install Miniconda package manager.
  https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \                           wget -q -P /tmp \
    && bash /tmp/Miniconda3-latest-Linux-x86_64.sh -b -p /opt/conda \                               https://repo.anaconda.com/miniconda/Miniconda3-py37_4.12.0-Linux-x86_64.sh \
    && rm /tmp/Miniconda3-latest-Linux-x86_64.sh                                                    && bash /tmp/Miniconda3-py37_4.12.0-Linux-x86_64.sh -b -p /opt/conda \
                                                                                                    && rm /tmp/Miniconda3-py37_4.12.0-Linux-x86_64.sh
# Install conda packages.                                                                           
ENV PATH="/opt/conda/bin:$PATH"                                                                     # Install conda packages.
RUN conda install -qy conda==4.13.0 \                                                               PATH="/opt/conda/bin:/usr/local/cuda-11.1/bin:$PATH"
    && conda install -y -c conda-forge \                                                            conda install -qy conda==4.13.0 \
      openmm=7.5.1 \                                                                                && conda install -y -c conda-forge \
      cudatoolkit==${CUDA_VERSION} \                                                                openmm=7.5.1 \
      pdbfixer \                                                                                    cudatoolkit==11.1.1 \
      pip \                                                                                         pdbfixer \
      python=3.7 \                                                                                  pip \
      && conda clean --all --force-pkgs-dirs --yes                                                  python=3.7 \
                                                                                                    && conda clean --all --force-pkgs-dirs --yes
COPY . /app/alphafold                                                                               
RUN wget -q -P /app/alphafold/alphafold/common/ \                                                   ### /bin/cp -r . /app/alphafold
  https://git.scicore.unibas.ch/schwede/openstructure/-/raw/7102c63615b64735c494(略                 
                                                                                                    wget -q -P /app/alphafold/alphafold/common/ \
# Install pip packages.                                                                             https://git.scicore.unibas.ch/schwede/openstructure/-/raw/7102c63615b64735c494(略
RUN pip3 install --upgrade pip --no-cache-dir \                                                     
    && pip3 install -r /app/alphafold/requirements.txt --no-cache-dir \                             # Install pip packages.
    && pip3 install --upgrade --no-cache-dir \                                                      # N.B. The URL specifies the list of jaxlib releases.
      jax==0.3.17 \                                                                                 pip3 install --upgrade pip  --no-cache-dir \
      jaxlib==0.3.15+cuda11.cudnn805 \                                                              && pip3 install -r /app/alphafold/requirements.txt --no-cache-dir \
      -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html                         && pip3 install --upgrade --no-cache-dir \
                                                                                                    jax==0.3.17 \
# Apply OpenMM patch.                                                                               jaxlib==0.3.15+cuda11.cudnn805 \
WORKDIR /opt/conda/lib/python3.7/site-packages                                                      -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
RUN patch -p0 < /app/alphafold/docker/openmm.patch                                                  
                                                                                                    # Apply OpenMM patch.
# Add SETUID bit to the ldconfig binary so that non-root users can run it.                          cd /opt/conda/lib/python3.7/site-packages
RUN chmod u+s /sbin/ldconfig.real                                                                   patch -p0 < /app/alphafold/docker/openmm.patch
                                                                                                    
# We need to run `ldconfig` first to ensure GPUs are visible, due to some quirk                     # Add SETUID bit to the ldconfig binary so that non-root users can run it.
# with Debian. See https://github.com/NVIDIA/nvidia-docker/issues/1399 for                          chmod u+s /sbin/ldconfig.real
# details.                                                                                          
# ENTRYPOINT does not support easily running multiple commands, so instead we                       %environment
# write a shell script to wrap them up.                                                             export PATH="/opt/conda/bin:/usr/local/cuda-11.1/bin:$PATH"
WORKDIR /app/alphafold                                                                              %runscript
RUN echo $'#!/bin/bash\n\                                                                           cd /app/alphafold
ldconfig\n\                                                                                         ldconfig
python /app/alphafold/run_alphafold.py "$@"' > /app/run_alphafold.sh \                              exec python /app/alphafold/run_alphafold.py "$@"
  && chmod +x /app/run_alphafold.sh                                                                 # %startscript
ENTRYPOINT ["/app/run_alphafold.sh"]                                                                # cd /app/alphafold
                                                                                                    # exec python /app/alphafold/run_alphafold.py "$@"

では作成してみます.

[root@rockylinux ~]# cat /etc/redhat-release
Rocky Linux release 8.6 (Green Obsidian)
 
[root@rockylinux ~]# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module  520.56.06  Thu Oct  6 21:38:55 UTC 2022
GCC version:  gcc version 8.5.0 20210514 (Red Hat 8.5.0-10) (GCC)
 
[root@rockylinux ~]# /usr/local/cuda/bin/nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
 
[root@rockylinux ~]#

まずは実行環境を pyenv/anaconda で作ります
もしtopazらで既に anaconda 環境があるのならそれを使います

git clone https://github.com/yyuu/pyenv.git /apps/pyenv
export PYENV_ROOT=/apps/pyenv
export PATH=$PYENV_ROOT/bin:$PATH
eval "$(pyenv init - --no-rehash)"
pyenv install anaconda3-2022.05
pyenv global anaconda3-2022.05
 
export PATH=$PYENV_ROOT/versions/anaconda3-2022.05/bin/:$PATH
conda update conda
 
(既にanaconda環境があるなら)
export PYENV_ROOT=/apps/pyenv
export PATH=$PYENV_ROOT/bin:$PATH
export PATH=$PYENV_ROOT/versions/anaconda3-2022.05/bin/:$PATH

作成する環境は alphafold_singularity としてます

[root@rockylinux ~]# conda create -n alphafold_singularity pip
 
[root@rockylinux ~]# source activate alphafold_singularity
 
(alphafold_singularity) [root@rockylinux ~]#

この状態で構築を進める

(alphafold_singularity) [root@rockylinux ~]# cd /apps/src/
(alphafold_singularity) [root@rockylinux src]# git clone https://github.com/deepmind/alphafold
 
(alphafold_singularity) [root@rockylinux src]# cd alphafold/
 
(alphafold_singularity) [root@rockylinux alphafold]# git tag
v2.0.0
v2.0.1
v2.1.0
v2.1.1
v2.1.2
v2.2.0
v2.2.1
v2.2.2
v2.2.3
v2.2.4
(alphafold_singularity) [root@rockylinux alphafold]# git checkout v2.2.4
 
(alphafold_singularity) [root@rockylinux alphafold]# git branch
* (HEAD detached at v2.2.4)
  main
(alphafold_singularity) [root@rockylinux alphafold]# git clone https://github.com/prehensilecode/alphafold_singularity singularity
 
(alphafold_singularity) [root@rockylinux alphafold]# cat singularity/requirements.txt
# Dependencies necessary to execute run_singularity.py
absl-py==0.13.0
spython==0.1.16
 
(alphafold_singularity) [root@rockylinux alphafold]# which pip
/apps/pyenv/versions/anaconda3-2022.05/envs/alphafold_singularity/bin/pip
 
(alphafold_singularity) [root@rockylinux alphafold]# pip install -r singularity/requirements.txt
 
(alphafold_singularity) [root@rockylinux alphafold]# singularity build alphafold.sif singularity/Singularity.def
(alphafold_singularity) [root@rockylinux alphafold]# source deactivate
[root@rockylinux alphafold]#

出来上がった alphafold.sif がこちら

[root@rockylinux alphafold]# pwd
/apps/src/alphafold
[root@rockylinux alphafold]# ls -lh alphafold.sif
-rwxr-xr-x. 1 root root 4.5G Nov 11 03:49 alphafold.sif
 
[root@rockylinux alphafold]#

っでこれらを/appsに配置してみる. 使いやすいようにするために.

[root@rockylinux alphafold]# mkdir /apps/alphafold_singularity
[root@rockylinux alphafold]# mv alphafold.sif /apps/alphafold_singularity/
[root@rockylinux alphafold]# cp singularity/run_singularity.py /apps/alphafold_singularity/
 
[root@rockylinux alphafold]# cd /apps/alphafold_singularity
 
[root@rockylinux alphafold_singularity]# pwd
/apps/alphafold_singularity
[root@rockylinux alphafold_singularity]# ls -l
total 4694588
-rwxr-xr-x. 1 root root 4807245824 Nov 11 03:49 alphafold.sif
-rw-r--r--. 1 root root      10504 Nov 11 03:58 run_singularity.py
 
[root@rockylinux alphafold_singularity]#

EnvironmentModulesとしては

[root@rockylinux ~]# vi /apps/modulefiles/alphafold_singularity
#%Module1.0
#
set          root /apps/pyenv/versions/anaconda3-2022.05/envs/alphafold_singularity
set          sif  /apps/alphafold_singularity
prepend-path PATH $sif:$root/bin
 
setenv ALPHAFOLD_DIR $sif
 
[root@rockylinux ~]#

っでテスト実行

[illya@rockylinux ~]$ module use --append /apps/modulefiles/
 
[illya@rockylinux ~]$ module load alphafold_singularity
 
[illya@rockylinux ~]$ cat query.fasta
>dummy_sequence
GWSTELEKHREELKEFLKKEGITNVEIRIDNGRLEVRVEGGTERLKRFLEELRQKLEKKGYTVDIKIE
 
[illya@rockylinux ~]$ which python
/apps/pyenv/versions/anaconda3-2022.05/envs/alphafold_singularity/bin/python
 
[illya@rockylinux ~]$ echo $ALPHAFOLD_DIR
/apps/alphafold_singularity
 
[illya@rockylinux ~]$ python /apps/alphafold_singularity/run_singularity.py --fasta_paths=/home/illya/query.fasta --max_template_date=2020-05-14 --data_dir=/Public/alphafold
 
(結果は /tmp/alphafold-xxxx/ に置かれる)
(次にdimerでテスト)
 
[illya@rockylinux ~]$ cat dimer.fasta
>XP_009313165.1.1
MRAAFAEARAALAEGEVPVGCVLVPVDASCAANAQLAADDDDDENKSKGSSNSNNSKKNDAVERLIAARG
RNATNREHHALAHAEFVAVEALLRELAANGQQRPASLAGYVLYVVVEPCIMCAAMLLYNRVQKVFFGCGN
PRFGGNGTVLAVHTAAGCSAPGYESSGGHRADEAVALLQEFYRHENTNAPGHKRRRKCECLNN
>XP_009313165.1.2
MRAAFAEARAALAEGEVPVGCVLVPVDASCAANAQLAADDDDDENKSKGSSNSNNSKKNDAVERLIAARG
RNATNREHHALAHAEFVAVEALLRELAANGQQRPASLAGYVLYVVVEPCIMCAAMLLYNRVQKVFFGCGN
PRFGGNGTVLAVHTAAGCSAPGYESSGGHRADEAVALLQEFYRHENTNAPGHKRRRKCECLNN
 
[illya@rockylinux ~]$ mkdir out_dimer    <-- 出力先を作成
 
[illya@rockylinux ~]$ python /apps/alphafold_singularity/run_singularity.py --fasta_paths=$HOME/dimer.fasta --max_template_date=2020-05-14 --data_dir=/Public/alphafold \
--db_preset=reduced_dbs --model_preset=multimer --output_dir=$HOME/out_dimer
 
[illya@rockylinux ~]$ ls -l out_dimer/
total 0
 
[illya@rockylinux ~]$
(output出力先には何もない... どうも /tmp/alphafold-xxxxxx に置かれる...

めも

Singularityイメージファイルで計算できるのでクラスターマシンとかでも使えるのかも.

ただ、Alphafoldの更新に伴い入力パラメータの追加があると、run_singularity.pyも追随して修正する必要がある.
この辺が面倒なところかなと思ってる.

ならオリジナルのdockerを使えば?なのでしょうけどね.


トップ   編集 添付 複製 名前変更     ヘルプ   最終更新のRSS
Last-modified: 2022-11-11 (金) 08:10:18 (17d)