DeepEMhancer をテンプレートにして作成
[
トップ
] [
新規
|
一覧
|
検索
|
最終更新
|
ヘルプ
|
ログイン
]
開始行:
本家様 [[https://github.com/rsanchezgarc/deepEMhancer>+ht...
gitには
#code(nonumber){{
Simply speaking, DeepEMhancer performs a non-linear post-...
1. Local sharpening-like post-processing.
2. Automatic masking/denoising of cryo-EM maps.
}}
とある. cryoSPARCからもコールされるようになっている.
インストール環境はこんな感じ
#code(nonumber){{
[root@rockylinux9 ~]# cat /etc/redhat-release
Rocky Linux release 9.4 (Blue Onyx)
[root@rockylinux9 ~]# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 535.179 ...
GCC version: gcc version 11.4.1 20231218 (Red Hat 11.4.1...
[root@rockylinux9 ~]#
}}
「/usr/local/cuda」は用意していないです
***インストール [#j5b5561d]
DeepEMhancerはpythonアプリなので、ここでは[[crYOLO]]のよ...
pyenv/anaconda環境がないなら下記のように敷設して
#code(nonumber){{
git clone https://github.com/yyuu/pyenv.git /apps/pyenv
export PYENV_ROOT=/apps/pyenv
export PATH=$PYENV_ROOT/bin:$PATH
pyenv install anaconda3-2024.02-1
pyenv global anaconda3-2024.02-1
source /apps/pyenv/versions/anaconda3-2024.02-1/etc/profi...
}}
もし既にpyenv/anaconda環境があるならsourceで環境を呼び込...
#code(nonumber){{
source /apps/pyenv/versions/anaconda3-2024.02-1/etc/profi...
}}
っで、DeepEMhancer を入れていきます.
#code(nonumber){{
[root@rockylinux9 ~]# cd /apps/
[root@rockylinux9 apps]# git clone https://github.com/rsa...
[root@rockylinux9 apps]# cd deepEMhancer
[root@rockylinux9 deepEMhancer]# ls -CF
alternative_installation/ condaDeepEMHancer/ conda_ymls...
[root@rockylinux deepEMhancer]# conda env create -f deepE...
[root@rockylinux deepEMhancer]#
}}
作った deepEMhancer 実行環境に対して
#code(nonumber){{
[root@rockylinux deepEMhancer]# conda activate deepEMhancer
(deepEMhancer) [root@rockylinux9 deepEMhancer]# conda list
:
cudatoolkit 11.8.0 h4ba93d1_13...
:
h5py 3.9.0 nompi_py39h87ca...
:
nvidia-cublas-cu11 11.11.3.6 pypi_0...
nvidia-cudnn-cu11 8.6.0.163 pypi_0...
:
tensorflow 2.12.0 pypi_0...
:
(deepEMhancer) [root@rockylinux9 deepEMhancer]# which pyt...
/apps/pyenv/versions/anaconda3-2024.02-1/envs/deepEMhance...
(deepEMhancer) [root@rockylinux9 deepEMhancer]# pwd
/apps/deepEMhancer
(deepEMhancer) [root@rockylinux9 deepEMhancer]# python -m...
(deepEMhancer) [root@rockylinux9 deepEMhancer]# conda list
:
deepemhancer 0.16 pypi_0...
:
(deepEMhancer) [root@rockylinux9 deepEMhancer]#
}}
これで一応インストールは完了です
***動作確認 [#e0d8fbf1]
GPUを利用して計算速度を向上できます
実際にGPUが使えるかのテストをしてみます
#code(nonumber){{
(deepEMhancer) [root@rockylinux9 deepEMhancer]# deepemhan...
0.16
(deepEMhancer) [root@rockylinux9 deepEMhancer]# python
>>>
>>> import tensorflow as tf
2024-06-02 02:00:18.071439: I tensorflow/tsl/cuda/cudart_...
2024-06-02 02:00:18.097133: I tensorflow/tsl/cuda/cudart_...
2024-06-02 02:00:18.097475: I tensorflow/core/platform/cp...
To enable the following instructions: AVX2 FMA, in other ...
2024-06-02 02:00:18.545798: W tensorflow/compiler/tf2tens...
/apps/pyenv/versions/anaconda3-2024.02-1/envs/deepEMhance...
_warn(("h5py is running against HDF5 {0} when it was bu...
>>>
>>>
}}
と検証前に「UserWarning: h5py is running against HDF5 1.1...
h5pyに関しては現状 h5py 3.9.0 を h5py 3.10.0 にすることで...
TensorRTに関しては新規に tensorrt 8.6.1 と LD_LIBRARY_PAT...
「Could not find cuda drivers...」も LD_LIBRARY_PATH で抑...
#code(nonumber){{
(deepEMhancer) [root@rockylinux9 deepEMhancer]# pip insta...
(deepEMhancer) [root@rockylinux9 deepEMhancer]# export LD...
(deepEMhancer) [root@rockylinux9 deepEMhancer]# export LD...
(deepEMhancer) [root@rockylinux9 deepEMhancer]# export LD...
(deepEMhancer) [root@rockylinux9 deepEMhancer]# python
>>>
>>> import tensorflow as tf
2024-06-02 02:11:02.483195: I tensorflow/core/platform/cp...
To enable the following instructions: AVX2 FMA, in other ...
>>>
(っでGPU確認)
>>>
>>> tf.test.is_gpu_available()
(略)
True
>>> quit();
(deepEMhancer) [root@rockylinux9 deepEMhancer]# conda dea...
[root@rockylinux9 deepEMhancer]#
}}
***EnvironmentModules [#r45f11d0]
#code(nonumber){{
[root@rockylinux9 ~]# vi /apps/modulefiles/deepEMhancer
#%Module1.0
set root /apps/pyenv/versions/anaconda3-20...
prepend-path PATH $root/bin
prepend-path LD_LIBRARY_PATH $root/lib:$root/lib/python3....
[root@rockylinux9 ~]#
}}
***つかう [#v940e17c]
モデルファイルが必要ですが、/appsとかに配置しておきます
#code(nonumber){{
[root@rockylinux9 ~]# module load /apps/modulefiles
[root@rockylinux9 ~]# module load deepEMhancer
[root@rockylinux9 ~]# deepemhancer --download /apps/deepE...
[root@rockylinux9 ~]# ls -l /apps/deepEMhancerModels/
total 0
drwxr-xr-x. 2 root root 160 Jun 2 02:27 production_check...
[root@rockylinux9 ~]# ls -l /apps/deepEMhancerModels/prod...
total 904376
-rw-r--r--. 1 root root 311505312 Jun 2 02:27 deepEMhanc...
-rw-r--r--. 1 root root 204777344 Jun 2 02:27 deepEMhanc...
-rw-r--r--. 1 root root 204886472 Jun 2 02:27 deepEMhanc...
-rw-r--r--. 1 root root 204897936 Jun 2 02:27 deepEMhanc...
-rw-r--r--. 1 root root 74 Jun 2 02:27 versions.txt
[root@rockylinux9 ~]# cat /apps/deepEMhancerModels/produc...
highRes: g2_3
tightTarget: v3_L_c_2
wideModel: v3t_28_c+
masked: vt2_m_28
[root@rockylinux9 ~]#
}}
っでテストrunです.
#code(nonumber){{
[illya@rockylinux9 ~]$ module use /apps/modulefiles/
[illya@rockylinux9 ~]$ module load deepEMhancer
[illya@rockylinux9 ~]$ cd test/
[illya@rockylinux9 test]$ deepemhancer -h
usage: deepemhancer -i INPUTMAP -o OUTPUTMAP [-p {wideTar...
[--cleaningStrengh CLEANINGSTRENGH] [...
DeepEMHancer. Deep post-processing of cryo-EM maps. https...
optional arguments:
-h, --help show this help message and exit
（略
[illya@rockylinux9 test]$
}}
これが正しいか不明ですが、relionのデータを使って流してみた.
#code(nonumber){{
[illya@rockylinux9 test]$ mkdir deepEMhancer
[illya@rockylinux9 test]$ cd deepEMhancer/
[illya@rockylinux9 deepEMhancer]$ deepemhancer -i ../reli...
../relion40_tutorial_precalculated_results/Refine3D/j...
-p tightTarget -o model.mrc --deepLearningModelPath ...
updating environment to select gpu: [0]
loading model /apps/deepEMhancerModels/production_checkpo...
Automatic radial noise detected beyond 36 % of volume side
DONE!. Shape at 1.00 A/voxel after padding-> (320, 320, ...
:
[illya@rockylinux9 deepEMhancer]$
}}
#code(nonumber){{
deepEMhancerCS.sh
#!/bin/bash
d=/apps/pyenv/versions/miniforge3-24.9.2-0/envs/deepEMhan...
export PATH=$d/bin:$PATH
export LD_LIBRARY_PATH=$d/lib:$d/lib/python3.9/site-packa...
$d/bin/deepemhancer $@
unset d
}}
終了行:
本家様 [[https://github.com/rsanchezgarc/deepEMhancer>+ht...
gitには
#code(nonumber){{
Simply speaking, DeepEMhancer performs a non-linear post-...
1. Local sharpening-like post-processing.
2. Automatic masking/denoising of cryo-EM maps.
}}
とある. cryoSPARCからもコールされるようになっている.
インストール環境はこんな感じ
#code(nonumber){{
[root@rockylinux9 ~]# cat /etc/redhat-release
Rocky Linux release 9.4 (Blue Onyx)
[root@rockylinux9 ~]# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 535.179 ...
GCC version: gcc version 11.4.1 20231218 (Red Hat 11.4.1...
[root@rockylinux9 ~]#
}}
「/usr/local/cuda」は用意していないです
***インストール [#j5b5561d]
DeepEMhancerはpythonアプリなので、ここでは[[crYOLO]]のよ...
pyenv/anaconda環境がないなら下記のように敷設して
#code(nonumber){{
git clone https://github.com/yyuu/pyenv.git /apps/pyenv
export PYENV_ROOT=/apps/pyenv
export PATH=$PYENV_ROOT/bin:$PATH
pyenv install anaconda3-2024.02-1
pyenv global anaconda3-2024.02-1
source /apps/pyenv/versions/anaconda3-2024.02-1/etc/profi...
}}
もし既にpyenv/anaconda環境があるならsourceで環境を呼び込...
#code(nonumber){{
source /apps/pyenv/versions/anaconda3-2024.02-1/etc/profi...
}}
っで、DeepEMhancer を入れていきます.
#code(nonumber){{
[root@rockylinux9 ~]# cd /apps/
[root@rockylinux9 apps]# git clone https://github.com/rsa...
[root@rockylinux9 apps]# cd deepEMhancer
[root@rockylinux9 deepEMhancer]# ls -CF
alternative_installation/ condaDeepEMHancer/ conda_ymls...
[root@rockylinux deepEMhancer]# conda env create -f deepE...
[root@rockylinux deepEMhancer]#
}}
作った deepEMhancer 実行環境に対して
#code(nonumber){{
[root@rockylinux deepEMhancer]# conda activate deepEMhancer
(deepEMhancer) [root@rockylinux9 deepEMhancer]# conda list
:
cudatoolkit 11.8.0 h4ba93d1_13...
:
h5py 3.9.0 nompi_py39h87ca...
:
nvidia-cublas-cu11 11.11.3.6 pypi_0...
nvidia-cudnn-cu11 8.6.0.163 pypi_0...
:
tensorflow 2.12.0 pypi_0...
:
(deepEMhancer) [root@rockylinux9 deepEMhancer]# which pyt...
/apps/pyenv/versions/anaconda3-2024.02-1/envs/deepEMhance...
(deepEMhancer) [root@rockylinux9 deepEMhancer]# pwd
/apps/deepEMhancer
(deepEMhancer) [root@rockylinux9 deepEMhancer]# python -m...
(deepEMhancer) [root@rockylinux9 deepEMhancer]# conda list
:
deepemhancer 0.16 pypi_0...
:
(deepEMhancer) [root@rockylinux9 deepEMhancer]#
}}
これで一応インストールは完了です
***動作確認 [#e0d8fbf1]
GPUを利用して計算速度を向上できます
実際にGPUが使えるかのテストをしてみます
#code(nonumber){{
(deepEMhancer) [root@rockylinux9 deepEMhancer]# deepemhan...
0.16
(deepEMhancer) [root@rockylinux9 deepEMhancer]# python
>>>
>>> import tensorflow as tf
2024-06-02 02:00:18.071439: I tensorflow/tsl/cuda/cudart_...
2024-06-02 02:00:18.097133: I tensorflow/tsl/cuda/cudart_...
2024-06-02 02:00:18.097475: I tensorflow/core/platform/cp...
To enable the following instructions: AVX2 FMA, in other ...
2024-06-02 02:00:18.545798: W tensorflow/compiler/tf2tens...
/apps/pyenv/versions/anaconda3-2024.02-1/envs/deepEMhance...
_warn(("h5py is running against HDF5 {0} when it was bu...
>>>
>>>
}}
と検証前に「UserWarning: h5py is running against HDF5 1.1...
h5pyに関しては現状 h5py 3.9.0 を h5py 3.10.0 にすることで...
TensorRTに関しては新規に tensorrt 8.6.1 と LD_LIBRARY_PAT...
「Could not find cuda drivers...」も LD_LIBRARY_PATH で抑...
#code(nonumber){{
(deepEMhancer) [root@rockylinux9 deepEMhancer]# pip insta...
(deepEMhancer) [root@rockylinux9 deepEMhancer]# export LD...
(deepEMhancer) [root@rockylinux9 deepEMhancer]# export LD...
(deepEMhancer) [root@rockylinux9 deepEMhancer]# export LD...
(deepEMhancer) [root@rockylinux9 deepEMhancer]# python
>>>
>>> import tensorflow as tf
2024-06-02 02:11:02.483195: I tensorflow/core/platform/cp...
To enable the following instructions: AVX2 FMA, in other ...
>>>
(っでGPU確認)
>>>
>>> tf.test.is_gpu_available()
(略)
True
>>> quit();
(deepEMhancer) [root@rockylinux9 deepEMhancer]# conda dea...
[root@rockylinux9 deepEMhancer]#
}}
***EnvironmentModules [#r45f11d0]
#code(nonumber){{
[root@rockylinux9 ~]# vi /apps/modulefiles/deepEMhancer
#%Module1.0
set root /apps/pyenv/versions/anaconda3-20...
prepend-path PATH $root/bin
prepend-path LD_LIBRARY_PATH $root/lib:$root/lib/python3....
[root@rockylinux9 ~]#
}}
***つかう [#v940e17c]
モデルファイルが必要ですが、/appsとかに配置しておきます
#code(nonumber){{
[root@rockylinux9 ~]# module load /apps/modulefiles
[root@rockylinux9 ~]# module load deepEMhancer
[root@rockylinux9 ~]# deepemhancer --download /apps/deepE...
[root@rockylinux9 ~]# ls -l /apps/deepEMhancerModels/
total 0
drwxr-xr-x. 2 root root 160 Jun 2 02:27 production_check...
[root@rockylinux9 ~]# ls -l /apps/deepEMhancerModels/prod...
total 904376
-rw-r--r--. 1 root root 311505312 Jun 2 02:27 deepEMhanc...
-rw-r--r--. 1 root root 204777344 Jun 2 02:27 deepEMhanc...
-rw-r--r--. 1 root root 204886472 Jun 2 02:27 deepEMhanc...
-rw-r--r--. 1 root root 204897936 Jun 2 02:27 deepEMhanc...
-rw-r--r--. 1 root root 74 Jun 2 02:27 versions.txt
[root@rockylinux9 ~]# cat /apps/deepEMhancerModels/produc...
highRes: g2_3
tightTarget: v3_L_c_2
wideModel: v3t_28_c+
masked: vt2_m_28
[root@rockylinux9 ~]#
}}
っでテストrunです.
#code(nonumber){{
[illya@rockylinux9 ~]$ module use /apps/modulefiles/
[illya@rockylinux9 ~]$ module load deepEMhancer
[illya@rockylinux9 ~]$ cd test/
[illya@rockylinux9 test]$ deepemhancer -h
usage: deepemhancer -i INPUTMAP -o OUTPUTMAP [-p {wideTar...
[--cleaningStrengh CLEANINGSTRENGH] [...
DeepEMHancer. Deep post-processing of cryo-EM maps. https...
optional arguments:
-h, --help show this help message and exit
（略
[illya@rockylinux9 test]$
}}
これが正しいか不明ですが、relionのデータを使って流してみた.
#code(nonumber){{
[illya@rockylinux9 test]$ mkdir deepEMhancer
[illya@rockylinux9 test]$ cd deepEMhancer/
[illya@rockylinux9 deepEMhancer]$ deepemhancer -i ../reli...
../relion40_tutorial_precalculated_results/Refine3D/j...
-p tightTarget -o model.mrc --deepLearningModelPath ...
updating environment to select gpu: [0]
loading model /apps/deepEMhancerModels/production_checkpo...
Automatic radial noise detected beyond 36 % of volume side
DONE!. Shape at 1.00 A/voxel after padding-> (320, 320, ...
:
[illya@rockylinux9 deepEMhancer]$
}}
#code(nonumber){{
deepEMhancerCS.sh
#!/bin/bash
d=/apps/pyenv/versions/miniforge3-24.9.2-0/envs/deepEMhan...
export PATH=$d/bin:$PATH
export LD_LIBRARY_PATH=$d/lib:$d/lib/python3.9/site-packa...
$d/bin/deepemhancer $@
unset d
}}
ページ名:
1
![[PukiWiki]](image/picc.png "[PukiWiki]")
