本家様 https://github.com/rsanchezgarc/deepEMhancer
gitには
Simply speaking, DeepEMhancer performs a non-linear post-processing of cryo-EM maps that produces two main effects:
1. Local sharpening-like post-processing.
2. Automatic masking/denoising of cryo-EM maps.
とある.
以前は Tensorflow version 1.14 ベースだったのが、Tensorflow version 2.10 に変更された模様.
pythonアプリなので、ここではcrYOLOのようにpyenv/anacondaの環境にDeepEMhancer 実行環境を用意していきます
pyenv/anaconda環境がないなら下記のように敷設して
git clone https://github.com/yyuu/pyenv.git /apps/pyenv
export PYENV_ROOT=/apps/pyenv
export PATH=$PYENV_ROOT/bin:$PATH
pyenv install anaconda3-2023.03
pyenv global anaconda3-2023.03
export PATH=$PYENV_ROOT/versions/anaconda3-2023.03/bin/:$PATH
もし既にpyenv/anaconda環境があるなら環境変数を整備して
export PYENV_ROOT=/apps/pyenv
export PATH=$PYENV_ROOT/bin:$PATH
export PATH=$PYENV_ROOT/versions/anaconda3-2023.03/bin/:$PATH
っで、DeepEMhancer を入れていきます.
[root@rockylinux ~]# cd /apps/
[root@rockylinux apps]# git clone https://github.com/rsanchezgarc/deepEMhancer
[root@rockylinux apps]# cd deepEMhancer
[root@rockylinux deepEMhancer]# conda env create -f deepEMhancer_env.yml -n deepEMhancer_env
Collecting package metadata (repodata.json): done
Solving environment: \
:
:
done
#
# To activate this environment, use
#
# $ conda activate deepEMhancer_env
#
# To deactivate an active environment, use
#
# $ conda deactivate
[root@rockylinux deepEMhancer]#
と環境構築が完了します. 引き続きその作った環境に移行して、deepEMhancerのプログラムをインストールします.
[root@rockylinux deepEMhancer]# source activate deepEMhancer_env
(deepEMhancer_env) [root@rockylinux deepEMhancer]# pwd
/apps/deepEMhancer
(deepEMhancer_env) [root@rockylinux deepEMhancer]#
/apps/deepEMhancer
(deepEMhancer_env) [root@rockylinux deepEMhancer]# ls -CF
alternative_installation/ condaDeepEMHancer/ deepEMhancer/ deepEMhancer_env.yml LICENSE README.md setup.py
(deepEMhancer_env) [root@rockylinux deepEMhancer]# which python
/apps/pyenv/versions/anaconda3-2023.03/envs/deepEMhancer_env/bin/python
(deepEMhancer_env) [root@rockylinux deepEMhancer]# python -m pip install . --no-deps
Processing /apps/deepEMhancer
Preparing metadata (setup.py) ... done
Building wheels for collected packages: deepEMhancer
Building wheel for deepEMhancer (setup.py) ... done
Created wheel for deepEMhancer: filename=deepEMhancer-0.15-py3-none-any.whl size=35349 sha256=c71c076c62fb0b4647fb49d8b2dec266fbaf44e9d621ee672316e1d9dc004815
Stored in directory: /tmp/pip-ephem-wheel-cache-1duggyi2/wheels/44/71/57/cf1894ae6e78993bf56a43c925286af06a88b16e8013d3536e
Successfully built deepEMhancer
Installing collected packages: deepEMhancer
Successfully installed deepEMhancer-0.15
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
(deepEMhancer_env) [root@rockylinux deepEMhancer]# conda deactivate
[root@rockylinux deepEMhancer]#
EnvironmentModulesで環境作成 †
[root@rockylinux ~]# vi /apps/modulefiles/deepEMhancer
#%Module1.0
set root /apps/pyenv/versions/anaconda3-2023.03/envs/deepEMhancer_env/
prepend-path PATH $root/bin
[root@rockylinux ~]#
つかう †
[saber@rockylinux ~]$ module use /apps/modulefiles/
[saber@rockylinux ~]$ module load deepEMhancer
[saber@rockylinux ~]$ deepemhancer --download
DOWNLAODING MODELs from https://zenodo.org/record/7432763/files/deepEMhancerModels_tf2.zip to /home/saber/.local/share/deepEMhancerModels
It may take a while...
100%|???????????????????????????????????????????????????????????????????????????????????????????????????????????| 180462/180464 [01:50<00:00, 1630.69it/s]
Total size downloaded: 739172352.
Unzipping...
DONE!!
[saber@rockylinux ~]$ ls -lh .local/share/deepEMhancerModels/production_checkpoints/
total 884M
-rw-rw-r--. 1 saber saber 298M May 22 02:22 deepEMhancer_highRes.hd5
-rw-rw-r--. 1 saber saber 196M May 22 02:22 deepEMhancer_masked.hd5
-rw-rw-r--. 1 saber saber 196M May 22 02:22 deepEMhancer_tightTarget.hd5
-rw-rw-r--. 1 saber saber 196M May 22 02:22 deepEMhancer_wideTarget.hd5
-rw-rw-r--. 1 saber saber 74 May 22 02:22 versions.txt
[saber@rockylinux ~]$
[saber@rockylinux ~]$ cat .local/share/deepEMhancerModels/production_checkpoints/versions.txt
highRes: g2_3
tightTarget: v3_L_c_2
wideModel: v3t_28_c+
masked: vt2_m_28
[saber@rockylinux ~]$
一応「deepemhancer --download ./DeepEMhancerData」と特定の場所にダウンロード可能です
っでテストrunです.
[saber@rockylinux ~]$ deepemhancer -h
usage: deepemhancer -i INPUTMAP -o OUTPUTMAP [-p {wideTarget,tightTarget,highRes}] [-i2 HALFMAP2] [-s SAMPLINGRATE] [--noiseStats NOISE_MEAN NOISE_STD]
[-m BINARYMASK] [--deepLearningModelPath PATH_TO_MODELS_DIR] [--cleaningStrengh CLEANINGSTRENGH] [-g GPUIDS] [-b BATCH_SIZE]
[--version] [-h] [--download [DOWNLOAD_DEST]]
DeepEMHancer. Deep post-processing of cryo-EM maps. https://github.com/rsanchezgarc/deepEMhancer
options:
-h, --help show this help message and exit
--download [DOWNLOAD_DEST]
download default DeepEMhancer models. They will be saved at /home/saber/.local/share/deepEMhancerModels/production_checkpoints
if no path provided
Main options:
-i INPUTMAP, --inputMap INPUTMAP
Input map to process or half map number 1. This map should be unmasked and not sharpened (Do not use post-processed maps, only
maps directly obtained from refinement). If half map 1 used, do not forget to also provide the half map 2 using -i2
-o OUTPUTMAP, --outputMap OUTPUTMAP
Output fname where post-processed map will be saved
-p {wideTarget,tightTarget,highRes}, --processingType {wideTarget,tightTarget,highRes}
Select the deep learning model you want to use. WideTarget will produce less sharp results than tightTarget. HighRes is only
recommended for overal FSC resolution < 4 A This option is igonred if normalization mode 2 is selected
-i2 HALFMAP2, --halfMap2 HALFMAP2
(Optional) Input half map 2 to process
-s SAMPLINGRATE, --samplingRate SAMPLINGRATE
(Optional) Sampling rate (A/voxel) of the input map. If not provided, the sampling rate will be read from mrc file header
Normalization options (auto normalization is applied if no option selected):
--noiseStats NOISE_MEAN NOISE_STD
(Optional) Normalization mode 1: The statisitcs of the noise to normalize (mean and standard deviation) the input. Preferred
over binaryMask but ignored if binaryMask provided. If not --noiseStats nor --binaryMask provided, nomralization params will be
automatically estimated, although, in some rare cases, estimation may fail or be less accurate
-m BINARYMASK, --binaryMask BINARYMASK
(Optional) Normalization mode 2: A binaryMask (1 protein, 0 no protein) used to normalize the input. If no normalization mode
provided, automatic normalization will be carried out. Supresses --precomputedModel option
Alternative options:
--deepLearningModelPath PATH_TO_MODELS_DIR
(Optional) Directory where a non default deep learning model is located (model is selected using --precomputedModel) or a path
to hd5 file containing the model
--cleaningStrengh CLEANINGSTRENGH
(Optional) Post-processing step to remove small connected components (hide dust). Max relative size of connected components to
remove 0<s<1 or -1 to deactivate. Default: -1
Computing devices options:
-g GPUIDS, --gpuIds GPUIDS
The gpu(s) where the program will be executed. If more that 1, comma seppared. E.g -g 1,2,3. Set to -1 to use only cpu (very
slow). Default: 0
-b BATCH_SIZE, --batch_size BATCH_SIZE
Number of cubes to process simultaneously. Lower it if CUDA Out Of Memory error happens and increase it if low GPU performance
observed. Warning, for some inputs it may crash if --gpus > . Use only 1 gpus in that case. 1Default: 8
--version show program's version number and exit
examples:
+ Download deep learning models
deepemhancer --download
+ Post-process input map path/to/inputVol.mrc and save it at path/to/outputVol.mrc using default deep model tightTarget
deepemhancer -i path/to/inputVol.mrc -o path/to/outputVol.mrc
+ Post-process input map path/to/inputVol.mrc and save it at path/to/outputVol.mrc using high resolution deep model
deepemhancer -p highRes -i path/to/inputVol.mrc -o path/to/outputVol.mrc
+ Post-process input map path/to/inputVol.mrc and save it at path/to/outputVol.mrc using a deep learning model located in path/to/deep/learningModel
deepemhancer -c path/to/deep/learningModel -i path/to/inputVol.mrc -o path/to/outputVol.mrc
+ Post-process input map path/to/inputVol.mrc and save it at path/to/outputVol.mrc using high resolution deep model and providing normalization information (mean
and standard deviation of the noise)
deepemhancer -p highRes -i path/to/inputVol.mrc -o path/to/outputVol.mrc --noiseStats 0.12 0.03
[saber@rockylinux ~]$
これが正しいか不明ですが、relionのデータを使って流してみた.
[saber@rockylinux test]$ deepemhancer -i ../relion40_tutorial_precalculated_results/Refine3D/job029/run_it000_half1_class001.mrc -i2 ../relion40_tutorial_precalculated_results/Refine3D/job029/run_it000_half2_class001.mrc -p tightTarget -o model.mrc
updating environment to select gpu: [0]
loading model /home/saber/.local/share/deepEMhancerModels/production_checkpoints/deepEMhancer_tightTarget.hd5 ... DONE!
Automatic radial noise detected beyond 36 % of volume side
DONE!. Shape at 1.00 A/voxel after padding-> (320, 320, 320)
Neural net inference
100%|?????????????????????????????????????????????????????????????????????????????????????????????????????????????????| 289/289 [4:58:14<00:00, 61.92s/it]
[saber@rockylinux test]$