cryoCARE をテンプレートにして作成
[
トップ
] [
新規
|
一覧
|
検索
|
最終更新
|
ヘルプ
|
ログイン
]
開始行:
本家様
[[https://github.com/juglab/cryoCARE_T2T>+https://github....
[[https://github.com/juglab/cryoCARE_T2T/wiki/Cryo-CARE>+...
ノイズ除去の一種(GPUを使います)
使い方はSingularityのイメージを作成して、この中で使って解...
あとメモリー消費を抑えたエディションも用意されていている...
[[https://github.com/juglab/cryoCARE_pip>+https://github....
まずはSingularity版を用意してみる.
***cryoCARE_T2T Singularity [#b2ac7883]
&color(red){*};&size(10){既にbuildされたimage fileも提供...
計算機に搭載のnvidiaドライバが 430.40 以上であることが必...
&color(red){*};&size(10){cudaライブラリは入れてないです. ...
#code(nonumber){{
[root@centos7 ~]# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 515.76 M...
GCC version: gcc version 4.8.5 20150623 (Red Hat 4.8.5-4...
[root@centos7 ~]# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
[root@centos7 ~]#
}}
と大丈夫みたい.
っで次. Singularityのimage fileをユーザに作らせたいのでfa...
#code(nonumber){{
[root@centos7 ~]# echo "user.max_user_namespaces=10000" >...
[root@centos7 ~]# sysctl -p
user.max_user_namespaces = 10000
[root@centos7 ~]# singularity config fakeroot --add illya
}}
っでfakerootを与えられたユーザにて構築
#code(nonumber){{
[illya@centos7 ~]$ git clone https://github.com/juglab/cr...
[illya@centos7 ~]$ cd cryoCARE_T2T/
[illya@centos7 cryoCARE_T2T]$ ls -CF
cryoCARE.Singularity example/ LICENSE preview.png REA...
[illya@centos7 cryoCARE_T2T]$
}}
少々「cryoCARE.Singularity」を修正する必要がありました.
#code(diff,nonumber){{
diff --git a/cryoCARE.Singularity b/cryoCARE.Singularity
index 8fd173d..ed22e4d 100644
--- a/cryoCARE.Singularity
+++ b/cryoCARE.Singularity
@@ -4,6 +4,8 @@ BootStrap: docker
From: tensorflow/tensorflow:1.15.0-gpu-py3
%post
+ apt-key adv --fetch-keys https://developer.download....
+ apt-key adv --fetch-keys https://developer.download....
apt-get -y update
# libtiff5 is needed for MotionCor2
@@ -39,14 +41,14 @@ From: tensorflow/tensorflow:1.15.0-gp...
# Download IMOD
mkdir /imod
- wget -P /imod https://bio3d.colorado.edu/ftp/latestI...
- sh /imod/imod_4.10.52_RHEL6-64_CUDA8.0.sh -extract
+ wget -P /imod https://bio3d.colorado.edu/ftp/latestI...
+ sh /imod/imod_4.12.27_RHEL6-64_CUDA8.0.sh -extract
ls /
- tar -xzf /IMODtempDir/imod_4.10.52_RHEL6-64_CUDA8.0....
- ln -s /usr/local/imod_4.10.52 /usr/local/IMOD
+ tar -xzf /IMODtempDir/imod_4.12.27_RHEL6-64_CUDA8.0....
+ ln -s /usr/local/imod_4.12.27 /usr/local/IMOD
cp /usr/local/IMOD/IMOD-linux.* /etc/profile.d
rm -r /imod
- rm -r /IMODtempDir/imod_4.10.52_RHEL6-64_CUDA8.0.tar...
+ rm -r /IMODtempDir/imod_4.12.27_RHEL6-64_CUDA8.0.tar...
# Install required Python packages
pip install keras==2.2.4
}}
その上で下記コマンドでSingularityのimage fileを作ります
#code(nonumber){{
[illya@centos7 cryoCARE_T2T]$ singularity build --fakeroo...
}}
っで動作テスト
作成した「cryoCARE_v0.1.1.simg」は「/home/illya/cryoCARE_...
MotionCor2を入手して、これを「/home/illya/cryoCARE_T2T/ex...
#code(nonumber){{
[illya@centos7 ~]$ cd cryoCARE_T2T/example/
[illya@centos7 example]$ ls -l
total 7268
-rw-rw-r--. 1 illya illya 13890 Oct 9 12:59 01_Split_F...
:
drwxrwxr-x. 2 illya illya 22 Oct 9 12:59 data
-rw-rw-r--. 1 illya illya 6528 Oct 9 12:59 generate_t...
-rwxrwxr-x. 1 illya illya 3058240 Oct 9 13:24 MotionCor2...
:
[illya@centos7 example]$ cd data/
[illya@centos7 data]$ unzip -q ~/Tomo110.zip
[illya@centos7 data]$ ls -l
total 0
drwxr-xr-x. 4 illya illya 32 Jul 12 2019 Tomo110
[illya@centos7 data]$
}}
参照 [[https://github.com/juglab/cryoCARE_T2T#example-dat...
テスト準備が完了したら起動します
#code(nonumber){{
[illya@centos7 ~]$ cd cryoCARE_T2T/
[illya@centos7 cryoCARE_T2T]$ singularity run --nv -B use...
INFO: underlay of /etc/localtime required more than 50...
INFO: underlay of /usr/bin/nvidia-smi required more th...
Starting notebook...
Open browser to localhost:8888
[I 14:03:38.314 NotebookApp] Writing notebook server cook...
[I 14:03:38.663 NotebookApp] Serving notebooks from local...
[I 14:03:38.663 NotebookApp] Jupyter Notebook 6.4.10 is r...
[I 14:03:38.663 NotebookApp] http://centos7:8888/?token=8...
[I 14:03:38.663 NotebookApp] or http://127.0.0.1:8888/?t...
[I 14:03:38.663 NotebookApp] Use Control-C to stop this s...
[C 14:03:38.665 NotebookApp]
To access the notebook, open this file in a browser:
file:///home/illya/.local/share/jupyter/runtime/n...
Or copy and paste one of these URLs:
http://centos7:8888/?token=823f4924266938676c4399...
or http://127.0.0.1:8888/?token=823f4924266938676c43...
}}
notebookですね。他のPCでブラウザを立ち上げて、明記された...
&ref(2022y10m09d_140802161.png,nolink);
だが、、テスト実行が動かない...
nvidia-smiがそもそもsingularity内で動かなかった. これはホ...
結局、MotionCor2は本家指定の「MotionCor2_1.3.0-Cuda101」...
だが、「515.76」だと無理でした.
よくよく「cryoCARE.Singularity」を見たら....「From: tenso...
***メモリー消費を抑えたエディション [#e21fd020]
[[https://github.com/juglab/cryoCARE_pip>+https://github....
Singularity経由でなく pip で構築となっているので、ここで...
#code(nonumber){{
[root@rockylinux9 ~]# cat /etc/redhat-release
Rocky Linux release 9.5 (Blue Onyx)
[root@rockylinux9 ~]# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 570.133.0...
GCC version: gcc version 11.5.0 20240719 (Red Hat 11.5.0...
[root@rockylinux9 ~]#
}}
anaconda環境は[[crYOLO]]とか[[topaz]]のように構築しますが...
#code(nonumber){{
git clone https://github.com/yyuu/pyenv.git /apps/pyenv
export PYENV_ROOT=/apps/pyenv
export PATH=$PYENV_ROOT/bin:$PATH
pyenv install anaconda3-2024.10-1
pyenv global anaconda3-2024.10-1
export PATH=$PYENV_ROOT/versions/anaconda3-2024.10-1/bin/...
}}
として
#code(nonumber){{
conda create -n cryocare_11 python=3.8 cudatoolkit=11.0 c...
conda activate cryocare_11
pip install tensorflow==2.4
pip install cryoCARE
conda deactivate
}}
最後にいつもの [[EnvironmentModules]]
#code(nonumber){{
[root@centos7 ~]# vi /apps/modulefiles/cryoCARE
#%Module
set root /apps/pyenv/versions/anaconda3-2024.10-...
prepend-path PATH $root/bin
prepend-path LD_LIBRARY_PATH $root/lib
[root@centos7 ~]#
}}
***blackwell対応 [#qb230f25]
&color(red){未確認品};
#code(nonumber){{
conda create -n cryocare_12 python=3.9 cuda-toolkit=12 te...
conda activate cryocare_12
pip install cryoCARE
conda install "ml_dtypes<0.4.0"
pip install cryoCARE
conda list
:
cryocare 0.3.0 pypi_0 ...
tensorflow 2.16.2 cuda120py...
:
}}
***cryocare-from-movies [#jb08eb7f]
本家様 [[https://github.com/SBC-Utrecht/cryocare-from-mov...
終了行:
本家様
[[https://github.com/juglab/cryoCARE_T2T>+https://github....
[[https://github.com/juglab/cryoCARE_T2T/wiki/Cryo-CARE>+...
ノイズ除去の一種(GPUを使います)
使い方はSingularityのイメージを作成して、この中で使って解...
あとメモリー消費を抑えたエディションも用意されていている...
[[https://github.com/juglab/cryoCARE_pip>+https://github....
まずはSingularity版を用意してみる.
***cryoCARE_T2T Singularity [#b2ac7883]
&color(red){*};&size(10){既にbuildされたimage fileも提供...
計算機に搭載のnvidiaドライバが 430.40 以上であることが必...
&color(red){*};&size(10){cudaライブラリは入れてないです. ...
#code(nonumber){{
[root@centos7 ~]# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 515.76 M...
GCC version: gcc version 4.8.5 20150623 (Red Hat 4.8.5-4...
[root@centos7 ~]# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
[root@centos7 ~]#
}}
と大丈夫みたい.
っで次. Singularityのimage fileをユーザに作らせたいのでfa...
#code(nonumber){{
[root@centos7 ~]# echo "user.max_user_namespaces=10000" >...
[root@centos7 ~]# sysctl -p
user.max_user_namespaces = 10000
[root@centos7 ~]# singularity config fakeroot --add illya
}}
っでfakerootを与えられたユーザにて構築
#code(nonumber){{
[illya@centos7 ~]$ git clone https://github.com/juglab/cr...
[illya@centos7 ~]$ cd cryoCARE_T2T/
[illya@centos7 cryoCARE_T2T]$ ls -CF
cryoCARE.Singularity example/ LICENSE preview.png REA...
[illya@centos7 cryoCARE_T2T]$
}}
少々「cryoCARE.Singularity」を修正する必要がありました.
#code(diff,nonumber){{
diff --git a/cryoCARE.Singularity b/cryoCARE.Singularity
index 8fd173d..ed22e4d 100644
--- a/cryoCARE.Singularity
+++ b/cryoCARE.Singularity
@@ -4,6 +4,8 @@ BootStrap: docker
From: tensorflow/tensorflow:1.15.0-gpu-py3
%post
+ apt-key adv --fetch-keys https://developer.download....
+ apt-key adv --fetch-keys https://developer.download....
apt-get -y update
# libtiff5 is needed for MotionCor2
@@ -39,14 +41,14 @@ From: tensorflow/tensorflow:1.15.0-gp...
# Download IMOD
mkdir /imod
- wget -P /imod https://bio3d.colorado.edu/ftp/latestI...
- sh /imod/imod_4.10.52_RHEL6-64_CUDA8.0.sh -extract
+ wget -P /imod https://bio3d.colorado.edu/ftp/latestI...
+ sh /imod/imod_4.12.27_RHEL6-64_CUDA8.0.sh -extract
ls /
- tar -xzf /IMODtempDir/imod_4.10.52_RHEL6-64_CUDA8.0....
- ln -s /usr/local/imod_4.10.52 /usr/local/IMOD
+ tar -xzf /IMODtempDir/imod_4.12.27_RHEL6-64_CUDA8.0....
+ ln -s /usr/local/imod_4.12.27 /usr/local/IMOD
cp /usr/local/IMOD/IMOD-linux.* /etc/profile.d
rm -r /imod
- rm -r /IMODtempDir/imod_4.10.52_RHEL6-64_CUDA8.0.tar...
+ rm -r /IMODtempDir/imod_4.12.27_RHEL6-64_CUDA8.0.tar...
# Install required Python packages
pip install keras==2.2.4
}}
その上で下記コマンドでSingularityのimage fileを作ります
#code(nonumber){{
[illya@centos7 cryoCARE_T2T]$ singularity build --fakeroo...
}}
っで動作テスト
作成した「cryoCARE_v0.1.1.simg」は「/home/illya/cryoCARE_...
MotionCor2を入手して、これを「/home/illya/cryoCARE_T2T/ex...
#code(nonumber){{
[illya@centos7 ~]$ cd cryoCARE_T2T/example/
[illya@centos7 example]$ ls -l
total 7268
-rw-rw-r--. 1 illya illya 13890 Oct 9 12:59 01_Split_F...
:
drwxrwxr-x. 2 illya illya 22 Oct 9 12:59 data
-rw-rw-r--. 1 illya illya 6528 Oct 9 12:59 generate_t...
-rwxrwxr-x. 1 illya illya 3058240 Oct 9 13:24 MotionCor2...
:
[illya@centos7 example]$ cd data/
[illya@centos7 data]$ unzip -q ~/Tomo110.zip
[illya@centos7 data]$ ls -l
total 0
drwxr-xr-x. 4 illya illya 32 Jul 12 2019 Tomo110
[illya@centos7 data]$
}}
参照 [[https://github.com/juglab/cryoCARE_T2T#example-dat...
テスト準備が完了したら起動します
#code(nonumber){{
[illya@centos7 ~]$ cd cryoCARE_T2T/
[illya@centos7 cryoCARE_T2T]$ singularity run --nv -B use...
INFO: underlay of /etc/localtime required more than 50...
INFO: underlay of /usr/bin/nvidia-smi required more th...
Starting notebook...
Open browser to localhost:8888
[I 14:03:38.314 NotebookApp] Writing notebook server cook...
[I 14:03:38.663 NotebookApp] Serving notebooks from local...
[I 14:03:38.663 NotebookApp] Jupyter Notebook 6.4.10 is r...
[I 14:03:38.663 NotebookApp] http://centos7:8888/?token=8...
[I 14:03:38.663 NotebookApp] or http://127.0.0.1:8888/?t...
[I 14:03:38.663 NotebookApp] Use Control-C to stop this s...
[C 14:03:38.665 NotebookApp]
To access the notebook, open this file in a browser:
file:///home/illya/.local/share/jupyter/runtime/n...
Or copy and paste one of these URLs:
http://centos7:8888/?token=823f4924266938676c4399...
or http://127.0.0.1:8888/?token=823f4924266938676c43...
}}
notebookですね。他のPCでブラウザを立ち上げて、明記された...
&ref(2022y10m09d_140802161.png,nolink);
だが、、テスト実行が動かない...
nvidia-smiがそもそもsingularity内で動かなかった. これはホ...
結局、MotionCor2は本家指定の「MotionCor2_1.3.0-Cuda101」...
だが、「515.76」だと無理でした.
よくよく「cryoCARE.Singularity」を見たら....「From: tenso...
***メモリー消費を抑えたエディション [#e21fd020]
[[https://github.com/juglab/cryoCARE_pip>+https://github....
Singularity経由でなく pip で構築となっているので、ここで...
#code(nonumber){{
[root@rockylinux9 ~]# cat /etc/redhat-release
Rocky Linux release 9.5 (Blue Onyx)
[root@rockylinux9 ~]# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 570.133.0...
GCC version: gcc version 11.5.0 20240719 (Red Hat 11.5.0...
[root@rockylinux9 ~]#
}}
anaconda環境は[[crYOLO]]とか[[topaz]]のように構築しますが...
#code(nonumber){{
git clone https://github.com/yyuu/pyenv.git /apps/pyenv
export PYENV_ROOT=/apps/pyenv
export PATH=$PYENV_ROOT/bin:$PATH
pyenv install anaconda3-2024.10-1
pyenv global anaconda3-2024.10-1
export PATH=$PYENV_ROOT/versions/anaconda3-2024.10-1/bin/...
}}
として
#code(nonumber){{
conda create -n cryocare_11 python=3.8 cudatoolkit=11.0 c...
conda activate cryocare_11
pip install tensorflow==2.4
pip install cryoCARE
conda deactivate
}}
最後にいつもの [[EnvironmentModules]]
#code(nonumber){{
[root@centos7 ~]# vi /apps/modulefiles/cryoCARE
#%Module
set root /apps/pyenv/versions/anaconda3-2024.10-...
prepend-path PATH $root/bin
prepend-path LD_LIBRARY_PATH $root/lib
[root@centos7 ~]#
}}
***blackwell対応 [#qb230f25]
&color(red){未確認品};
#code(nonumber){{
conda create -n cryocare_12 python=3.9 cuda-toolkit=12 te...
conda activate cryocare_12
pip install cryoCARE
conda install "ml_dtypes<0.4.0"
pip install cryoCARE
conda list
:
cryocare 0.3.0 pypi_0 ...
tensorflow 2.16.2 cuda120py...
:
}}
***cryocare-from-movies [#jb08eb7f]
本家様 [[https://github.com/SBC-Utrecht/cryocare-from-mov...
ページ名:
1
![[PukiWiki]](image/picc.png "[PukiWiki]")
