本家様 https://github.com/google-deepmind/alphafold3

12月にリリースかなと思ったのですが11月上旬にリリース.

ここでは rockylinux9での構築をしてみます.

AlphaFold2と同じようにdockerコンテナで実行できるようにするのがオリジナルみたい.
まずは下記記載のドキュメント通りに作ってみます
https://github.com/google-deepmind/alphafold3/blob/main/docs/installation.md

Toolkit for alphafold3 input and output files https://github.com/cddlab/alphafold3_tools

対象マシン†

インストール対象マシンは下記の感じです

[root@rockylinux9 ~]# cat /etc/redhat-release
Rocky Linux release 9.4 (Blue Onyx)
 
[root@rockylinux9 ~]# getenforce
Enforcing
 
[root@rockylinux9 ~]# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module  550.127.05  Tue Oct  8 03:22:07 UTC 2024
GCC version:  gcc version 11.4.1 20231218 (Red Hat 11.4.1-3) (GCC)
 
[root@rockylinux9 ~]# nvidia-smi -L
GPU 0: NVIDIA GeForce GTX 1070 (UUID: GPU-a49de51b-de1e-52f3-1e3f-ce704e159713)
 
[root@rockylinux9 ~]# ls -l /usr/local/cuda
ls: cannot access '/usr/local/cuda': No such file or directory
 
[root@rockylinux9 ~]#
↑

docker構築†

[root@rockylinux9 ~]# dnf -y install dnf-plugins-core
[root@rockylinux9 ~]# dnf config-manager --add-repo https://download.docker.com/linux/rhel/docker-ce.repo
Adding repo from: https://download.docker.com/linux/rhel/docker-ce.repo
[root@rockylinux9 ~]# ls /etc/yum.repos.d/
docker-ce.repo  rocky-addons.repo  rocky-devel.repo  rocky-extras.repo  rocky.repo    <--「docker-ce.repo」が追加される
 
[root@rockylinux9 ~]#
[root@rockylinux9 ~]# sed -i s'/enabled=1/enabled=0/' /etc/yum.repos.d/docker-ce.repo

使用するリポジトリは「docker-ce-stable」となる. 詳細は「dnf repolist -v docker-ce-stable」. 中身は「dnf list available --disablerepo=* --enablerepo=docker-ce-stable」で見れる.

[root@rockylinux9 ~]# dnf --enablerepo=docker-ce-stable install docker-ce
 
docker-ce, docker-ce-cli, docker-buildx-plugin, docker-ce-rootless-extras, docker-compose-plugin, containerd.io がインストールされる
 
っで起動
[root@rockylinux9 ~]# systemctl enable docker --now

「systemctl status docker」の中身を見ると「/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock」で実行なので「/var/lib/docker」にイメージが保存されます.
別の場所に変えたいなら「--data-root」を使って変更します. 「/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --data-root /docker-images 」とかで.

次にNVIDIA Container Toolkitをインストールします

[root@rockylinux9 ~]# curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | tee /etc/yum.repos.d/nvidia-container-toolkit.repo
[root@rockylinux9 ~]# sed -i s'/enabled=1/enabled=0/' /etc/yum.repos.d/nvidia-container-toolkit.repo

追加したリポジトリ「nvidia-container-toolkit」の詳細は「dnf repolist -v nvidia-container-toolkit」. 中身は「dnf list available --disablerepo=* --enablerepo=nvidia-container-toolkit」で見れる
っでインストールします

[root@rockylinux9 ~]# dnf --enablerepo=nvidia-container-toolkit install nvidia-container-toolkit
 
nvidia-container-toolkit, libnvidia-container-tools, libnvidia-container1, nvidia-container-toolkit-base がインストールされる
 
dockerを再起動させて有効にします
 
[root@rockylinux9 ~]# systemctl restart docker

っでテスト

[root@rockylinux9 ~]# nvidia-container-cli info
NVRM version:   550.127.05
CUDA version:   12.4
 
Device Index:   0
Device Minor:   0
Model:          NVIDIA GeForce GTX 1070
Brand:          GeForce
GPU UUID:       GPU-a49de51b-de1e-52f3-1e3f-ce704e159713
Bus Location:   00000000:06:10.0
Architecture:   6.1
 
[root@rockylinux9 ~]#
[root@rockylinux9 ~]# docker run --gpus all --rm nvidia/cuda:11.8.0-base-ubuntu22.04 nvidia-smi -L
 :
GPU 0: NVIDIA GeForce GTX 1070 (UUID: GPU-a49de51b-de1e-52f3-1e3f-ce704e159713)
[root@rockylinux9 ~]#

これでdockerの構築が完了.

そのままでは一般ユーザがdockerを使えない. なので「dockerd-rootless-setuptool.sh --skip-iptables install」でrootlessを有効にさせる.

[saber@rockylinux9 ~]$ dockerd-rootless-setuptool.sh --skip-iptables install
 :
[INFO] Installed docker.service successfully.
[INFO] To control docker.service, run: `systemctl --user (start|stop|restart) docker.service`
[INFO] To run docker.service on system startup, run: `sudo loginctl enable-linger saber`
 :
[INFO] Some applications may require the following environment variable too:
export DOCKER_HOST=unix:///run/user/1000/docker.sock
 :
[saber@rockylinux9 ~]$ ps -ef |grep docker
(この段階でdockerが動いてます. ログアウトするとdockerは止まります. 再度ログインするとdockerは自動的に動きます)

その後にrootで「/etc/nvidia-container-runtime/config.toml」にある「#no-cgroups = false」を「no-cgroups = true」に変更する. これでgpuが一般ユーザでもrootlessで使える.
systemのbootに合わせて、ログインの有無にかかわらずユーザのdockerを動かすには「sudo loginctl enable-linger saber」を有効にすればいいみたい

っで本題へ.

↑

AlphaFold3構築†

せっかくdockerをrootlessで作ったので一般ユーザでalphafold3を作ってみます.

[saber@rockylinux9 ~]$ git clone https://github.com/google-deepmind/alphafold3.git
 
[saber@rockylinux9 ~]$ cd alphafold3/
[saber@rockylinux9 alphafold3]$ ls -CF
CMakeLists.txt        docs/               OUTPUT_TERMS_OF_USE.md  requirements.txt       src/
dev-requirements.txt  fetch_databases.py  pyproject.toml          run_alphafold.py       WEIGHTS_PROHIBITED_USE_POLICY.md
docker/               LICENSE             README.md               run_alphafold_test.py  WEIGHTS_TERMS_OF_USE.md
 
[saber@rockylinux9 alphafold3]$　
[saber@rockylinux9 alphafold3]$ docker build -t alphafold3 --network host -f docker/Dockerfile .      <--「apt update」が失敗するので「--network host」を追加してます
 :
 :
[saber@rockylinux9 alphafold3]$ docker images
REPOSITORY   TAG       IMAGE ID       CREATED         SIZE
alphafold3   latest    59219a3b2f9b   2 minutes ago   9.03GB
 
[saber@rockylinux9 alphafold3]$

docker imageの作成に成功しました.

↑

データベースの準備†

以前とは違ってpythonツール「fetch_databases.py」が提供されている. 中身を読むと「storage.googleapis.com/alphafold-databases/v3.0」にあるファイルのダウンロードと解凍ですね.

[saber@rockylinux9 alphafold3]$ python ./fetch_databases.py --help
usage: fetch_databases.py [-h] [--download_destination DOWNLOAD_DESTINATION]
 
Downloads AlphaFold databases.
 
optional arguments:
  -h, --help            show this help message and exit
  --download_destination DOWNLOAD_DESTINATION
                        The directory to download the databases to.
 
[saber@rockylinux9 alphafold3]$
 
[saber@rockylinux9 alphafold3]$ python ./fetch_databases.py --download_destination /Public/alphafold3
 :
(ティータイム)
 :
All databases have been downloaded and decompressed.
 
[saber@rockylinux9 alphafold3]$
[saber@rockylinux9 alphafold3]$ ls -l /Public/alphafold3
total 656822124
-rw-r--r--. 1 saber saber  18171626364 Nov 17 08:32 bfd-first_non_consensus_sequences.fasta
-rw-r--r--. 1 saber saber 128579703018 Nov 17 10:08 mgy_clusters_2022_05.fa
-rw-r--r--. 1 saber saber  80977012680 Nov 17 08:57 nt_rna_2023_02_23_clust_seq_id_90_cov_80_rep_seq.fasta
-rw-r--r--. 1 saber saber 250266275840 Nov 17 10:05 pdb_2022_09_28_mmcif_files.tar
-rw-r--r--. 1 saber saber    232899463 Nov 17 08:12 pdb_seqres_2022_09_28.fasta
-rw-r--r--. 1 saber saber    228433680 Nov 17 08:12 rfam_14_9_clust_seq_id_90_cov_80_rep_seq.fasta
-rw-r--r--. 1 saber saber  13860314914 Nov 17 08:20 rnacentral_active_seq_id_90_cov_80_linclust.fasta
-rw-r--r--. 1 saber saber 108447942931 Nov 17 09:54 uniprot_all_2021_04.fa
-rw-r--r--. 1 saber saber  71821260491 Nov 17 09:19 uniref90_2022_05.fa
 
[saber@rockylinux9 alphafold3]$
↑

モデルパラメータの取得†

下記申請フォームに記載して取得します

https://forms.gle/svvpY4u2jsHEwWYS6

Google DeepMind 側の裁量があるようです. 申請しても貰えない可能性がある
「$HOME/af3-models」にでも配置しておきます

↑

実行†

未実施

mkdir  $HOME/af_input  $HOME/af_output
 
docker run -it \
    --volume $HOME/af_input:/root/af_input \
    --volume $HOME/af_output:/root/af_output \
    --volume $HOME/af3-models:/root/models \
    --volume /Public/alphafold3:/root/public_databases \
    --gpus all \
    alphafold3 \
    python run_alphafold.py \
    --json_path=/root/af_input/fold_input.json \
    --model_dir=/root/models \
    --output_dir=/root/af_output
↑

Singularityへ†

作ったdokcerイメージを Singularity image file に変換してdocker不要で実行できるようにしてみます

singularityのインストール

[root@rockylinux9 ~]# dnf install epel-release -y
[root@rockylinux9 ~]# dnf install singularity-ce -y
[saber@rockylinux9 alphafold3]$ docker ps -a
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 
[saber@rockylinux9 alphafold3]$ docker images
REPOSITORY   TAG       IMAGE ID       CREATED        SIZE
alphafold3   latest    59219a3b2f9b   13 hours ago   9.03GB
 
[saber@rockylinux9 alphafold3]$ docker run -d -p 5000:5000 --restart=always --name registry registry:2
Unable to find image 'registry:2' locally
2: Pulling from library/registry
dc0decf4841d: Pull complete
6cb0aa443e23: Pull complete
813676e291ef: Pull complete
dc2fb7dcec61: Pull complete
916205650bfe: Pull complete
Digest: sha256:543dade69668e02e5768d7ea2b0aa4fae6aa7384c9a5a8dbecc2be5136079ddb
Status: Downloaded newer image for registry:2
3c355497cde6f5959b956e03533aa2d84fc5e760c674318c6ced5129fb2ab49d
 
[saber@rockylinux9 alphafold3]$ 
[saber@rockylinux9 alphafold3]$ lsof -i
COMMAND     PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
rootlessk 16914 saber   13u  IPv4  70840      0t0  TCP *:commplex-main (LISTEN)
rootlessk 16914 saber   16u  IPv6  69250      0t0  TCP *:commplex-main (LISTEN)
 
[saber@rockylinux9 alphafold3]$
[saber@rockylinux9 alphafold3]$ docker ps -a
CONTAINER ID   IMAGE        COMMAND                  CREATED          STATUS          PORTS                                       NAMES
3c355497cde6   registry:2   "/entrypoint.sh /etc…"   47 seconds ago   Up 47 seconds   0.0.0.0:5000->5000/tcp, :::5000->5000/tcp   registry
[saber@rockylinux9 alphafold3]$
[saber@rockylinux9 alphafold3]$ docker images
REPOSITORY   TAG       IMAGE ID       CREATED         SIZE
alphafold3   latest    59219a3b2f9b   13 hours ago    9.03GB
registry     2         c18a86d35e98   13 months ago   25.4MB
[saber@rockylinux9 alphafold3]$
 
[saber@rockylinux9 alphafold3]$ docker tag alphafold3 localhost:5000/alphafold3
[saber@rockylinux9 alphafold3]$ docker ps -a
CONTAINER ID   IMAGE        COMMAND                  CREATED         STATUS         PORTS                                       NAMES
3c355497cde6   registry:2   "/entrypoint.sh /etc…"   3 minutes ago   Up 3 minutes   0.0.0.0:5000->5000/tcp, :::5000->5000/tcp   registry
[saber@rockylinux9 alphafold3]$
[saber@rockylinux9 alphafold3]$ docker images
REPOSITORY                  TAG       IMAGE ID       CREATED         SIZE
alphafold3                  latest    59219a3b2f9b   13 hours ago    9.03GB
localhost:5000/alphafold3   latest    59219a3b2f9b   13 hours ago    9.03GB
registry                    2         c18a86d35e98   13 months ago   25.4MB
[saber@rockylinux9 alphafold3]$
 
[saber@rockylinux9 alphafold3]$ docker push localhost:5000/alphafold3
Using default tag: latest
The push refers to repository [localhost:5000/alphafold3]
d96d8841e4b4: Pushed
9cbe0a162b7b: Pushed
ef0973ada226: Pushed
5f70bf18a086: Pushed
ae59d8945f10: Pushed
b257f8f438cc: Pushed
d0eca21f61dc: Pushed
3f00387e422f: Pushed
437449bb2387: Pushed
3b6bc3c2c74b: Pushed
cc51bf61b66d: Pushed
2591292aa496: Pushed
1e9c40c384ef: Pushed
931b7ff0cb6f: Pushed
latest: digest: sha256:9d3eb15f3e87f755d1ea9bdbdbae0b009f0e3b1487d739043aba6b895de595f9 size: 3274
[saber@rockylinux9 alphafold3]$
 
[saber@rockylinux9 alphafold3]$ docker ps -a
CONTAINER ID   IMAGE        COMMAND                  CREATED         STATUS         PORTS                                       NAMES
3c355497cde6   registry:2   "/entrypoint.sh /etc…"   9 minutes ago   Up 9 minutes   0.0.0.0:5000->5000/tcp, :::5000->5000/tcp   registry
[saber@rockylinux9 alphafold3]$
[saber@rockylinux9 alphafold3]$ docker images
REPOSITORY                  TAG       IMAGE ID       CREATED         SIZE
localhost:5000/alphafold3   latest    59219a3b2f9b   13 hours ago    9.03GB
alphafold3                  latest    59219a3b2f9b   13 hours ago    9.03GB
registry                    2         c18a86d35e98   13 months ago   25.4MB
[saber@rockylinux9 alphafold3]$
[saber@rockylinux9 alphafold3]$ lsof  -i
COMMAND     PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
rootlessk 16914 saber   13u  IPv4  70840      0t0  TCP *:commplex-main (LISTEN)
rootlessk 16914 saber   16u  IPv6  69250      0t0  TCP *:commplex-main (LISTEN)
[saber@rockylinux9 alphafold3]$
 
 
[saber@rockylinux9 alphafold3]$ SINGULARITY_NOHTTPS=1 singularity build alphafold3.sif docker://localhost:5000/alphafold3:latest
INFO:    Starting build...
INFO:    Fetching OCI image...
82.1MiB / 82.1MiB [================================================================================================================================================================================] 100 % 161.3 MiB/s 0s
7.1MiB / 7.1MiB [==================================================================================================================================================================================] 100 % 161.3 MiB/s 0s
12.3MiB / 12.3MiB [================================================================================================================================================================================] 100 % 161.3 MiB/s 0s
54.6MiB / 54.6MiB [================================================================================================================================================================================] 100 % 161.3 MiB/s 0s
119.3MiB / 119.3MiB [==============================================================================================================================================================================] 100 % 161.3 MiB/s 0s
233.2MiB / 233.2MiB [==============================================================================================================================================================================] 100 % 161.3 MiB/s 0s
4.4MiB / 4.4MiB [==================================================================================================================================================================================] 100 % 161.3 MiB/s 0s
11.2MiB / 11.2MiB [================================================================================================================================================================================] 100 % 161.3 MiB/s 0s
29.0MiB / 29.0MiB [================================================================================================================================================================================] 100 % 161.3 MiB/s 0s
99.4MiB / 99.4MiB [================================================================================================================================================================================] 100 % 161.3 MiB/s 0s
4.3GiB / 4.3GiB [==================================================================================================================================================================================] 100 % 161.3 MiB/s 0s
INFO:    Extracting OCI image...
2024/11/17 21:25:02  warn rootless{usr/lib/x86_64-linux-gnu/gstreamer1.0/gstreamer-1.0/gst-ptp-helper} ignoring (usually) harmless EPERM on setxattr "security.capability"
INFO:    Inserting Singularity configuration...
INFO:    Creating SIF file...
INFO:    Build complete: alphafold3.sif
[saber@rockylinux9 alphafold3]$
 
[saber@rockylinux9 alphafold3]$ ls -lh alphafold3.sif
-rwxr-xr-x. 1 saber saber 4.9G Nov 17 21:28 alphafold3.sif
[saber@rockylinux9 alphafold3]$
 
 
[saber@rockylinux9 alphafold3]$ docker images
REPOSITORY                  TAG       IMAGE ID       CREATED         SIZE
alphafold3                  latest    59219a3b2f9b   14 hours ago    9.03GB
localhost:5000/alphafold3   latest    59219a3b2f9b   14 hours ago    9.03GB
registry                    2         c18a86d35e98   13 months ago   25.4MB
[saber@rockylinux9 alphafold3]$
[saber@rockylinux9 alphafold3]$ docker ps -a
CONTAINER ID   IMAGE        COMMAND                  CREATED          STATUS          PORTS                                       NAMES
3c355497cde6   registry:2   "/entrypoint.sh /etc…"   22 minutes ago   Up 22 minutes   0.0.0.0:5000->5000/tcp, :::5000->5000/tcp   registry
[saber@rockylinux9 alphafold3]$

後始末

[saber@rockylinux9 alphafold3]$ docker stop registry
[saber@rockylinux9 alphafold3]$ docker ps -a
CONTAINER ID   IMAGE        COMMAND                  CREATED          STATUS                     PORTS     NAMES
3c355497cde6   registry:2   "/entrypoint.sh /etc…"   23 minutes ago   Exited (2) 3 seconds ago             registry
[saber@rockylinux9 alphafold3]$ docker rm 3c355497cde6
3c355497cde6
[saber@rockylinux9 alphafold3]$ docker ps -a
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
[saber@rockylinux9 alphafold3]$
[saber@rockylinux9 alphafold3]$ docker rmi localhost:5000/alphafold3
[saber@rockylinux9 alphafold3]$ docker rmi c18a86d35e98   
[saber@rockylinux9 alphafold3]$ docker rmi 59219a3b2f9b 
 
[saber@rockylinux9 alphafold3]$ docker images
REPOSITORY   TAG       IMAGE ID   CREATED   SIZE
[saber@rockylinux9 alphafold3]$
↑

job†

配列構造データベースが「/Public/alphafold3」にあって
取得したmodelファイルが「/Public/af3-model」にあるとして

slurmな環境なら今いるディレクトリに「af_input」と「af_output」を掘って
「af_input/fold_input.json」のインプットファイルを作って
下記batchを動かす

#!/bin/bash
#SBATCH -J af3-test
#SBATCH -o %j.out
#SBATCH -e %j.err
#SBATCH -p workq
#SBATCH -n 8
#SBATCH --gres=gpu:1
 
cd $SLURM_SUBMIT_DIR
 
MODEL_DIR=/Public/af3-model
DB_DIR=/Public/alphafold3
 
singularity exec \
     --nv \
     --bind $SLURM_SUBMIT_DIR/af_input:/root/af_input \
     --bind $SLURM_SUBMIT_DIR/af_output:/root/af_output \
     --bind $MODEL_DIR:/root/models \
     --bind $DB_DIR:/root/public_databases \
     /apps/alphafold3.sif \
     python /app/alphafold/run_alphafold.py \
     --json_path=/root/af_input/fold_input.json \
     --model_dir=/root/models \
     --db_dir=/root/public_databases \
     --output_dir=/root/af_output
最新の60件
2025-01-23 2025-01-22 2025-01-21 2025-01-20 2025-01-13 2025-01-12 2025-01-08 2024-12-30 2024-12-29 2024-12-22
  • AlphaFold3
2024-12-20 2024-12-17 2024-12-15 2024-12-14 2024-12-12 2024-12-11 2024-12-10 2024-12-09 2024-12-08 2024-12-04 2024-11-28 2024-11-22 2024-11-15 2024-11-14 2024-11-12 2024-11-06 2024-11-05 2024-11-04 2024-11-02 2024-11-01 2024-10-29 2024-10-28 2024-10-27 2024-10-23 2024-10-18 2024-10-17 2024-10-15 2024-10-14 2024-10-13 2024-10-11 2024-10-10 2024-10-09 2024-10-08 2024-10-05

edit

トップ   編集 差分 履歴 添付 複製 名前変更 リロード   新規 一覧 検索 最終更新   ヘルプ   最終更新のRSS
Last-modified: 2024-12-22 (日) 02:52:48