![[PukiWiki]](image/picc.png "[PukiWiki]")
本家様 https://github.com/rsanchezgarc/deepEMhancer
gitには
Simply speaking, DeepEMhancer performs a non-linear post-processing of cryo-EM maps that produces two main effects:
1. Local sharpening-like post-processing.
2. Automatic masking/denoising of cryo-EM maps.とある. cryoSPARCからもコールされるようになっている.
インストール環境はこんな感じ
[root@rockylinux9 ~]# cat /etc/redhat-release
Rocky Linux release 9.4 (Blue Onyx)
[root@rockylinux9 ~]# cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 535.179 Fri Apr 26 21:43:18 UTC 2024
GCC version: gcc version 11.4.1 20231218 (Red Hat 11.4.1-3) (GCC)
[root@rockylinux9 ~]#「/usr/local/cuda」は用意していないです
DeepEMhancerはpythonアプリなので、ここではcrYOLOのようにpyenv/anacondaの環境にDeepEMhancer実行環境を用意していきます
pyenv/anaconda環境がないなら下記のように敷設して
git clone https://github.com/yyuu/pyenv.git /apps/pyenv
export PYENV_ROOT=/apps/pyenv
export PATH=$PYENV_ROOT/bin:$PATH
pyenv install anaconda3-2024.02-1
pyenv global anaconda3-2024.02-1
source /apps/pyenv/versions/anaconda3-2024.02-1/etc/profile.d/conda.shもし既にpyenv/anaconda環境があるならsourceで環境を呼び込みます.
source /apps/pyenv/versions/anaconda3-2024.02-1/etc/profile.d/conda.shっで、DeepEMhancer を入れていきます.
[root@rockylinux9 ~]# cd /apps/
[root@rockylinux9 apps]# git clone https://github.com/rsanchezgarc/deepEMhancer
[root@rockylinux9 apps]# cd deepEMhancer
[root@rockylinux9 deepEMhancer]# ls -CF
alternative_installation/ condaDeepEMHancer/ conda_ymls/ deepEMhancer/ deepEMhancer_env.yml deepEMhancer_tutorial.pdf LICENSE README.md setup.py
[root@rockylinux deepEMhancer]# conda env create -f deepEMhancer_env.yml -n deepEMhancer
[root@rockylinux deepEMhancer]#作った deepEMhancer 実行環境に対して
[root@rockylinux deepEMhancer]# conda activate deepEMhancer
(deepEMhancer) [root@rockylinux9 deepEMhancer]# conda list
:
cudatoolkit 11.8.0 h4ba93d1_13 conda-forge
:
h5py 3.9.0 nompi_py39h87cadad_103 conda-forge
:
nvidia-cublas-cu11 11.11.3.6 pypi_0 pypi
nvidia-cudnn-cu11 8.6.0.163 pypi_0 pypi
:
tensorflow 2.12.0 pypi_0 pypi
:
(deepEMhancer) [root@rockylinux9 deepEMhancer]# which python
/apps/pyenv/versions/anaconda3-2024.02-1/envs/deepEMhancer/bin/python
(deepEMhancer) [root@rockylinux9 deepEMhancer]# pwd
/apps/deepEMhancer
(deepEMhancer) [root@rockylinux9 deepEMhancer]# python -m pip install . --no-deps
(deepEMhancer) [root@rockylinux9 deepEMhancer]# conda list
:
deepemhancer 0.16 pypi_0 pypi
:
(deepEMhancer) [root@rockylinux9 deepEMhancer]#これで一応インストールは完了です
GPUを利用して計算速度を向上できます
実際にGPUが使えるかのテストをしてみます
(deepEMhancer) [root@rockylinux9 deepEMhancer]# deepemhancer --version バージョン確認
0.16
(deepEMhancer) [root@rockylinux9 deepEMhancer]# python
>>>
>>> import tensorflow as tf
2024-06-02 02:00:18.071439: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2024-06-02 02:00:18.097133: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2024-06-02 02:00:18.097475: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-06-02 02:00:18.545798: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/apps/pyenv/versions/anaconda3-2024.02-1/envs/deepEMhancer/lib/python3.9/site-packages/h5py/__init__.py:36: UserWarning: h5py is running against HDF5 1.14.3 when it was built against 1.14.2, this may cause problems
_warn(("h5py is running against HDF5 {0} when it was built against {1}, "
>>>
>>>と検証前に「UserWarning: h5py is running against HDF5 1.14.3 when it was built against 1.14.2」や「TF-TRT Warning: Could not find TensorRT」「Could not find cuda drivers on your machine, GPU will not be used」と言われる.
h5pyに関しては現状 h5py 3.9.0 を h5py 3.10.0 にすることで抑えられる.
TensorRTに関しては新規に tensorrt 8.6.1 と LD_LIBRARY_PATH の定義で抑えられる
「Could not find cuda drivers...」も LD_LIBRARY_PATH で抑えられる
(deepEMhancer) [root@rockylinux9 deepEMhancer]# pip install h5py==3.10.0 tensorrt==8.6.1
(deepEMhancer) [root@rockylinux9 deepEMhancer]# export LD_LIBRARY_PATH=/apps/pyenv/versions/anaconda3-2024.02-1/envs/deepEMhancer/lib
(deepEMhancer) [root@rockylinux9 deepEMhancer]# export LD_LIBRARY_PATH=/apps/pyenv/versions/anaconda3-2024.02-1/envs/deepEMhancer/lib/python3.9/site-packages/nvidia/cudnn/lib:$LD_LIBRARY_PATH
(deepEMhancer) [root@rockylinux9 deepEMhancer]# export LD_LIBRARY_PATH=/apps/pyenv/versions/anaconda3-2024.02-1/envs/deepEMhancer/lib/python3.9/site-packages/tensorrt_libs:$LD_LIBRARY_PATH
(deepEMhancer) [root@rockylinux9 deepEMhancer]# python
>>>
>>> import tensorflow as tf
2024-06-02 02:11:02.483195: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
>>>
(っでGPU確認)
>>>
>>> tf.test.is_gpu_available()
(略)
True
>>> quit();
(deepEMhancer) [root@rockylinux9 deepEMhancer]# conda deactivate
[root@rockylinux9 deepEMhancer]#[root@rockylinux9 ~]# vi /apps/modulefiles/deepEMhancer
#%Module1.0
set root /apps/pyenv/versions/anaconda3-2024.02-1/envs/deepEMhancer
prepend-path PATH $root/bin
prepend-path LD_LIBRARY_PATH $root/lib:$root/lib/python3.9/site-packages/nvidia/cudnn/lib:$root/lib/python3.9/site-packages/tensorrt_libs
[root@rockylinux9 ~]#モデルファイルが必要ですが、/appsとかに配置しておきます
[root@rockylinux9 ~]# module load /apps/modulefiles
[root@rockylinux9 ~]# module load deepEMhancer
[root@rockylinux9 ~]# deepemhancer --download /apps/deepEMhancerModels
[root@rockylinux9 ~]# ls -l /apps/deepEMhancerModels/
total 0
drwxr-xr-x. 2 root root 160 Jun 2 02:27 production_checkpoints
[root@rockylinux9 ~]# ls -l /apps/deepEMhancerModels/production_checkpoints/
total 904376
-rw-r--r--. 1 root root 311505312 Jun 2 02:27 deepEMhancer_highRes.hd5
-rw-r--r--. 1 root root 204777344 Jun 2 02:27 deepEMhancer_masked.hd5
-rw-r--r--. 1 root root 204886472 Jun 2 02:27 deepEMhancer_tightTarget.hd5
-rw-r--r--. 1 root root 204897936 Jun 2 02:27 deepEMhancer_wideTarget.hd5
-rw-r--r--. 1 root root 74 Jun 2 02:27 versions.txt
[root@rockylinux9 ~]# cat /apps/deepEMhancerModels/production_checkpoints/versions.txt
highRes: g2_3
tightTarget: v3_L_c_2
wideModel: v3t_28_c+
masked: vt2_m_28
[root@rockylinux9 ~]#っでテストrunです.
[illya@rockylinux9 ~]$ module use /apps/modulefiles/
[illya@rockylinux9 ~]$ module load deepEMhancer
[illya@rockylinux9 ~]$ cd test/
[illya@rockylinux9 test]$ deepemhancer -h
usage: deepemhancer -i INPUTMAP -o OUTPUTMAP [-p {wideTarget,tightTarget,highRes}] [-i2 HALFMAP2] [-s SAMPLINGRATE] [--noiseStats NOISE_MEAN NOISE_STD] [-m BINARYMASK] [--deepLearningModelPath PATH_TO_MODELS_DIR]
[--cleaningStrengh CLEANINGSTRENGH] [-g GPUIDS] [-b BATCH_SIZE] [--version] [-h] [--download [DOWNLOAD_DEST]]
DeepEMHancer. Deep post-processing of cryo-EM maps. https://github.com/rsanchezgarc/deepEMhancer
optional arguments:
-h, --help show this help message and exit
（略
[illya@rockylinux9 test]$これが正しいか不明ですが、relionのデータを使って流してみた.
[illya@rockylinux9 test]$ mkdir deepEMhancer
[illya@rockylinux9 test]$ cd deepEMhancer/
[illya@rockylinux9 deepEMhancer]$ deepemhancer -i ../relion40_tutorial_precalculated_results/Refine3D/job029/run_it000_half1_class001.mrc -i2 \
../relion40_tutorial_precalculated_results/Refine3D/job029/run_it000_half2_class001.mrc \
-p tightTarget -o model.mrc --deepLearningModelPath /apps/deepEMhancerModels/production_checkpoints/ -g 0
updating environment to select gpu: [0]
loading model /apps/deepEMhancerModels/production_checkpoints/deepEMhancer_tightTarget.hd5 ... DONE!
Automatic radial noise detected beyond 36 % of volume side
DONE!. Shape at 1.00 A/voxel after padding-> (320, 320, 320)
:
[illya@rockylinux9 deepEMhancer]$deepEMhancerCS.sh
#!/bin/bash
d=/apps/pyenv/versions/miniforge3-24.9.2-0/envs/deepEMhancer
export PATH=$d/bin:$PATH
export LD_LIBRARY_PATH=$d/lib:$d/lib/python3.9/site-packages/nvidia/cudnn/lib:$LD_LIBRARY_PATH
$d/bin/deepemhancer $@
unset d
