![[PukiWiki]](image/picc.png "[PukiWiki]")
#author("2026-09-26T17:30:25+00:00","default:sysosa","sysosa") #author("2026-09-26T17:31:09+00:00","default:sysosa","sysosa") 本家様 [[https://github.com/rsanchezgarc/deepEMhancer>+https://github.com/rsanchezgarc/deepEMhancer]] gitには #code(nonumber){{ Simply speaking, DeepEMhancer performs a non-linear post-processing of cryo-EM maps that produces two main effects: 1. Local sharpening-like post-processing. 2. Automatic masking/denoising of cryo-EM maps. }} とある. cryoSPARCからもコールされるようになっている. インストール環境はこんな感じ #code(nonumber){{ [root@rockylinux9 ~]# cat /etc/redhat-release Rocky Linux release 9.8 (Blue Onyx) [root@rockylinux9 ~]# cat /proc/driver/nvidia/version NVRM version: NVIDIA UNIX Open Kernel Module for x86_64 595.104.02 Release Build (dvs-builder@U22-I3-H06-02-6) Thu Sep 17 17:31:15 UTC 2026 GCC version: gcc version 11.5.0 20240719 (Red Hat 11.5.0-14) (GCC) [root@rockylinux9 ~]# nvidia-smi -L GPU 0: NVIDIA RTX PRO 2000 Blackwell (UUID: GPU-40660e37-0d35-d4f4-294a-eee3fe83049e) [root@rockylinux9 ~]# }} 「/usr/local/cuda」は用意していないです ***インストール [#j5b5561d] DeepEMhancerはpythonアプリなので、pyenv/miniforgeの環境にDeepEMhancer実行環境を用意していきます pyenv/miniforge環境がないなら下記のように敷設して #code(nonumber){{ git clone https://github.com/yyuu/pyenv.git /apps/pyenv export PYENV_ROOT=/apps/pyenv export PATH=$PYENV_ROOT/bin:$PATH pyenv install miniforge3-26.7.2-0 source /apps/pyenv/versions/miniforge3-26.7.2-0/etc/profile.d/conda.sh conda update -n base -c conda-forge conda (既に環境があるなら) source /apps/pyenv/versions/miniforge3-26.7.2-0/etc/profile.d/conda.sh }} っで、DeepEMhancer を入れていきます. #code(nonumber){{ [root@rockylinux9 ~]# cd /apps [root@rockylinux9 apps]# git clone https://github.com/rsanchezgarc/deepEMhancer [root@rockylinux9 apps]# cd deepEMhancer/ [root@rockylinux9 deepEMhancer]# ls -CF alternative_installation/ condaDeepEMHancer/ conda_ymls/ deepEMhancer/ deepEMhancer_env.yml deepEMhancer_tutorial.pdf LICENSE README.md setup.py tests/ [root@rockylinux9 deepEMhancer]# cat deepEMhancer_env.yml channels: - conda-forge - defaults dependencies: - pip=26.2 - python=3.11 - pip: - h5py==3.14.0 - mrcfile==1.5.4 - numpy==2.2.6 - requests==2.34.2 - scikit-image==0.26.0 - scipy==1.17.1 - tensorflow[and-cuda]==2.21.0 - tqdm==4.70.1 [root@rockylinux9 deepEMhancer]# [root@rockylinux9 deepEMhancer]# conda env create -f deepEMhancer_env.yml -n deepEMhancer }} 作った deepEMhancer 実行環境に対して #code(nonumber){{ [root@rockylinux9 deepEMhancer]# conda activate deepEMhancer (deepEMhancer) [root@rockylinux9 deepEMhancer]# (deepEMhancer) [root@rockylinux9 deepEMhancer]# which python /apps/pyenv/versions/miniforge3-26.7.2-0/envs/deepEMhancer/bin/python (deepEMhancer) [root@rockylinux9 deepEMhancer]# python -m pip install . --no-deps : Successfully built deepEMhancer Installing collected packages: deepEMhancer Successfully installed deepEMhancer-0.18 (deepEMhancer) [root@rockylinux9 deepEMhancer]# (deepEMhancer) [root@rockylinux9 deepEMhancer]# conda list # packages in environment at /apps/pyenv/versions/miniforge3-26.7.2-0/envs/deepEMhancer: # # Name Version Build Channel _openmp_mutex 4.5 20_gnu conda-forge absl-py 2.5.0 pypi_0 pypi astunparse 1.6.3 pypi_0 pypi bzip2 1.0.8 hda65f42_10 conda-forge ca-certificates 2026.7.22 hbd8a1cb_0 conda-forge certifi 2026.7.22 pypi_0 pypi charset-normalizer 3.5.1 pypi_0 pypi deepemhancer 0.18 pypi_0 pypi flatbuffers 25.12.19 pypi_0 pypi gast 0.7.0 pypi_0 pypi google-pasta 0.2.0 pypi_0 pypi grpcio 1.84.0 pypi_0 pypi h5py 3.14.0 pypi_0 pypi icu 78.3 py310h44b86e0_2 conda-forge idna 3.20 pypi_0 pypi imageio 2.37.4 pypi_0 pypi keras 3.15.1 pypi_0 pypi lazy-loader 0.6 pypi_0 pypi ld_impl_linux-64 2.46.1 default_hbd61a6d_102 conda-forge libclang 18.1.1 pypi_0 pypi libexpat 2.8.5 hd2095e1_0 conda-forge libffi 3.7.0 h81df57d_1 conda-forge libgcc 16.2.0 ha9f2e26_7 conda-forge libgomp 16.2.0 he0feb66_7 conda-forge liblzma 5.8.3 hb03c661_1 conda-forge libnsl 2.0.1 hb9d3cd8_1 conda-forge libpython 3.11.16 h0c77377_2_cpython conda-forge libsqlite 3.53.4 h13e7031_1 conda-forge libstdcxx 16.2.0 h934c35e_7 conda-forge libuuid 2.42.4 hcfc3c73_0 conda-forge libxcrypt 4.4.38 h280c20c_0 conda-forge libzlib 1.3.2 h25fd6f3_3 conda-forge markdown-it-py 4.2.0 pypi_0 pypi mdurl 0.1.2 pypi_0 pypi ml-dtypes 0.6.0 pypi_0 pypi mrcfile 1.5.4 pypi_0 pypi namex 0.1.0 pypi_0 pypi ncurses 6.6 hdb14827_1 conda-forge networkx 3.6.1 pypi_0 pypi numpy 2.2.6 pypi_0 pypi nvidia-cublas-cu12 12.9.2.10 pypi_0 pypi nvidia-cuda-cupti-cu12 12.9.79 pypi_0 pypi nvidia-cuda-nvcc-cu12 12.9.86 pypi_0 pypi nvidia-cuda-nvrtc-cu12 12.9.86 pypi_0 pypi nvidia-cuda-runtime-cu12 12.9.79 pypi_0 pypi nvidia-cudnn-cu12 9.26.0.51 pypi_0 pypi nvidia-cufft-cu12 11.4.1.4 pypi_0 pypi nvidia-curand-cu12 10.3.10.19 pypi_0 pypi nvidia-cusolver-cu12 11.7.5.82 pypi_0 pypi nvidia-cusparse-cu12 12.5.10.65 pypi_0 pypi nvidia-nccl-cu12 2.32.3 pypi_0 pypi nvidia-nvjitlink-cu12 12.9.86 pypi_0 pypi openssl 3.6.4 h781a0a9_0 conda-forge opt-einsum 3.4.0 pypi_0 pypi optree 0.20.0 pypi_0 pypi packaging 26.3 pyhc364b38_0 conda-forge pillow 12.3.0 pypi_0 pypi pip 26.2.1 pyh8b19718_0 conda-forge protobuf 7.36.2 pypi_0 pypi pygments 2.21.0 pypi_0 pypi python 3.11.16 h5f976f7_2_cpython conda-forge readline 8.3 hd6e31c0_1 conda-forge requests 2.34.2 pypi_0 pypi rich 15.0.0 pypi_0 pypi scikit-image 0.26.0 pypi_0 pypi scipy 1.17.1 pypi_0 pypi setuptools 84.0.0 pyh332efcf_0 conda-forge six 1.17.0 pypi_0 pypi tensorflow 2.21.0 pypi_0 pypi termcolor 3.3.0 pypi_0 pypi tifffile 2026.3.3 pypi_0 pypi tk 8.6.13 noxft_h1df4ec4_4 conda-forge tqdm 4.70.1 pypi_0 pypi typing-extensions 4.16.0 pypi_0 pypi tzdata 2026c h151e31d_0 conda-forge urllib3 2.8.0 pypi_0 pypi wheel 0.48.0 pyhd8ed1ab_0 conda-forge wrapt 2.4.1 pypi_0 pypi zstd 1.5.7 hb78ec9c_7 conda-forge (deepEMhancer) [root@rockylinux9 deepEMhancer]# (deepEMhancer) [root@rockylinux9 deepEMhancer]# conda deactivate [root@rockylinux9 deepEMhancer]# }} これで一応インストールは完了です ***EnvironmentModules [#r45f11d0] 「/apps/modulefiles/deepEMhancer」 #code(nonumber){{ #%Module1.0 proc ModulesHelp { } { puts stderr "deepEMhancer 0.18" puts stderr "TensorFlow 2.21 / CUDA 12 / Blackwell GPU environment" } module-whatis "deepEMhancer 0.18 with TensorFlow 2.21 CUDA 12 support" set root /apps/pyenv/versions/miniforge3-26.7.2-0/envs/deepEMhancer set nvidia $root/lib/python3.11/site-packages/nvidia # deepEMhancer / Python prepend-path PATH $root/bin # TensorFlow CUDA libraries installed by pip prepend-path LD_LIBRARY_PATH $nvidia/nvjitlink/lib prepend-path LD_LIBRARY_PATH $nvidia/nccl/lib prepend-path LD_LIBRARY_PATH $nvidia/curand/lib prepend-path LD_LIBRARY_PATH $nvidia/cuda_runtime/lib prepend-path LD_LIBRARY_PATH $nvidia/cuda_nvrtc/lib prepend-path LD_LIBRARY_PATH $nvidia/cuda_cupti/lib prepend-path LD_LIBRARY_PATH $nvidia/cusparse/lib prepend-path LD_LIBRARY_PATH $nvidia/cufft/lib prepend-path LD_LIBRARY_PATH $nvidia/cublas/lib prepend-path LD_LIBRARY_PATH $nvidia/cusolver/lib prepend-path LD_LIBRARY_PATH $nvidia/cudnn/lib }} ***動作確認 [#e0d8fbf1] GPUを利用して計算速度を向上できます 実際にGPUが使えるかのテストをしてみます #code(nonumber){{ [saber@rockylinux9 ~]$ module load deepEMhancer [saber@rockylinux9 ~]$ which python /apps/pyenv/versions/miniforge3-26.7.2-0/envs/deepEMhancer/bin/python [saber@rockylinux9 ~]$ which deepemhancer /apps/pyenv/versions/miniforge3-26.7.2-0/envs/deepEMhancer/bin/deepemhancer [saber@rockylinux9 ~]$ deepemhancer --version 0.18 [saber@rockylinux9 ~]$ python -c "import tensorflow as tf; print(tf.__version__); print(tf.config.list_physical_devices('GPU'))" WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1790433738.879650 40767 port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point (略 I0000 00:00:1790433738.905989 40767 cpu_feature_guard.cc:227] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1790433739.567853 40767 port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point (略 W0000 00:00:1790433739.884478 40767 gpu_device.cc:2459] TensorFlow was not built with CUDA kernel binaries compatible with compute capability 12.0a. CUDA kernels will be jit-compiled from PTX, which could take 30 minutes or longer. [PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')] [saber@rockylinux9 ~]$ }} 「oneDNN custom operations are on」はoneDNN（oneAPI Deep Neural Network Library）とintelさんのCPU/GPUにおいてニューラルネットワーク計算を高速化するライブラリを使いますよって事 でも「You may see slightly different numerical results due to floating-point round-off errors from different computation orders.」と浮動小数点演算では丸め誤差がでるからねと. 「To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.」それがいやなら TF_ENABLE_ONEDNN_OPTS=0 を定義してねとの事. ここではそのままにしてます. あと最新版のtensorflow 2.21.0 を使ってますが、cc12.0 つまりは sm_120 な blackwellにはネイティブで対応してないくて、PTX経由のJITコンパイルで用意される感じですね. tensorflow の sm_120向けはコンパイルすればできそうですが、辞めておきます. ***つかう [#v940e17c] モデルファイルが必要ですが、/appsとかに配置しておきます #code(nonumber){{ [root@rockylinux9 ~]# module use /apps/modulefiles/ [root@rockylinux9 ~]# module load deepEMhancer [root@rockylinux9 ~]# deepemhancer --download /apps/deepEMhancerModels [root@rockylinux9 ~]# ls -l /apps/deepEMhancerModels/ total 0 drwxr-xr-x 2 root root 160 Sep 26 23:57 production_checkpoints [root@rockylinux9 ~]# ls -l /apps/deepEMhancerModels/production_checkpoints/ total 904376 -rw-r--r-- 1 root root 311505312 Sep 26 23:57 deepEMhancer_highRes.hd5 -rw-r--r-- 1 root root 204777344 Sep 26 23:57 deepEMhancer_masked.hd5 -rw-r--r-- 1 root root 204886472 Sep 26 23:57 deepEMhancer_tightTarget.hd5 -rw-r--r-- 1 root root 204897936 Sep 26 23:57 deepEMhancer_wideTarget.hd5 -rw-r--r-- 1 root root 74 Sep 26 23:57 versions.txt [root@rockylinux9 ~]# }} っでテストrunです. #code(nonumber){{ [saber@rockylinux9 ~]$ module use /apps/modulefiles/ [saber@rockylinux9 ~]$ module load deepEMhancer [saber@rockylinux9 ~]$ mkdir -p test/deepEMhancer && cd $_ [saber@rockylinux9 deepEMhancer]$ deepemhancer -h usage: deepemhancer -i INPUTMAP -o OUTPUTMAP [-p {wideTarget,tightTarget,highRes}] [-i2 HALFMAP2] [-s SAMPLINGRATE] [--noiseStats NOISE_MEAN NOISE_STD] [-m BINARYMASK] [--deepLearningModelPath PATH_TO_MODELS_DIR] [--cleaningStrengh CLEANINGSTRENGH] [-g GPUIDS] [-b BATCH_SIZE] [--version] [-h] [--download [DOWNLOAD_DEST]] （略 [saber@rockylinux9 deepEMhancer]$ }} これが正しいか不明ですが、relionのデータを使って流してみた. #code(nonumber){{ [saber@rockylinux9 deepEMhancer]$ [saber@rockylinux9 deepEMhancer]$ deepemhancer -i ../Tutorial5.0/Refine3D/job029/run_it000_half1_class001.mrc \ -i2 ../Tutorial5.0/Refine3D/job029/run_it000_half2_class001.mrc \ -p tightTarget -o model.mrc --deepLearningModelPath /apps/deepEMhancerModels/production_checkpoints/ -g 0 updating environment to select gpu: [0] WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1790442345.232244 40966 port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to (略 WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1790442345.906649 40966 port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to (略 Loading and averaging half maps... Half maps loaded. loading model /apps/deepEMhancerModels/production_checkpoints/deepEMhancer_tightTarget.hd5 ... DONE! Loading and normalizing input volume... Automatic radial noise detected beyond 36 % of volume side DONE!. Shape at 1.00 A/voxel after padding-> (320, 320, 320) Neural net inference : : [saber@rockylinux9 deepEMhancer]$ ls -l [saber@rockylinux9 deepEMhancer]$ ls -l total 65540 -rw-r--r-- 1 saber saber 67109888 May 12 19:24 model.mrc [saber@rockylinux9 deepEMhancer]$ -rw-r--r-- 1 saber saber 67109888 Sep 27 02:18 model.mrc [saber@rockylinux9 deepEMhancer]$ }} ***cryoSPARC向けのスクリプト [#z94ec15a] 「/apps/cs_deepEMhancer.sh」 #code(nonumber){{ #!/usr/bin/env bash if command -v conda > /dev/null 2>&1; then conda deactivate > /dev/null 2>&1 || true # ignore any errors conda deactivate > /dev/null 2>&1 || true # ignore any errors fi unset _CE_CONDA unset CONDA_DEFAULT_ENV unset CONDA_EXE unset CONDA_PREFIX unset CONDA_PROMPT_MODIFIER unset CONDA_PYTHON_EXE unset CONDA_SHLVL unset PYTHONPATH unset LD_PRELOAD unset LD_LIBRARY_PATH source /apps/pyenv/versions/miniforge3-26.7.2-0/etc/profile.d/conda.sh conda activate deepEMhancer exec deepemhancer $@ }}
