EvolutionaryScaleModeling
本家様 https://github.com/facebookresearch/esm

「python <= 3.9 and pytorch」とあるので、condaでpython 3.9の環境を作り、そこに移り、pytorchを入れる
pytorchはhttps://pytorch.org/get-started/locally/のご神託に従っています.

[root@rockylinux ~]# export PYENV_ROOT=/apps/pyenv
[root@rockylinux ~]# export PATH=$PYENV_ROOT/bin:$PATH
[root@rockylinux ~]# export PATH=$PYENV_ROOT/versions/anaconda3-2022.05/bin/:$PATH
 
[root@rockylinux ~]# conda create -n ESM python=3.9
[root@rockylinux ~]# source activate ESM
 
(ESM) [root@rockylinux ~]# conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
(ESM) [root@rockylinux ~]# conda install matplotlib
 
(ESM) [root@rockylinux ~]# pip install fair-esm
(ESM) [root@rockylinux ~]# pip install git+https://github.com/facebookresearch/esm.git
 
(ESM) [root@rockylinux ~]# which nvcc
/apps/pyenv/versions/anaconda3-2022.05/envs/ESM/bin/nvcc
 
(ESM) [root@rockylinux ~]# nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Jun__8_16:49:14_PDT_2022
Cuda compilation tools, release 11.7, V11.7.99
Build cuda_11.7.r11.7/compiler.31442593_0
 
(ESM) [root@rockylinux ~]#
(ESM) [root@rockylinux ~]# pip install "fair-esm[esmfold]"
(ESM) [root@rockylinux ~]# pip install 'dllogger @ git+https://github.com/NVIDIA/dllogger.git'
(ESM) [root@rockylinux ~]# pip install 'openfold @ git+https://github.com/aqlaboratory/openfold.git@4b41059694619831a7db195b7e0988fc4ff3a307'
 
 
(ESM) [root@rockylinux ~]# source deactivate
[root@rockylinux ~]#

EnvironmentModules

[root@rockylinux ~]# vi /apps/modulefiles/ESM
#%Module
set          root   /apps/pyenv/versions/anaconda3-2022.05/envs/ESM
prepend-path PATH   $root/bin
 
[root@rockylinux ~]#

使い方としては module load されて、自分のホームディレクトリに落とした git ないで使うといった感じがいいのかも
最近は、自分でconda環境を用意してそこで運用という前提が多いかな

[illya@rockylinux ~]$ git clone https://github.com/facebookresearch/esm ESM
[illya@rockylinux ESM]$ ls -l
total 68
-rw-rw-r--. 1 illya illya   268 Nov 23 11:49 CODE_OF_CONDUCT.rst
-rw-rw-r--. 1 illya illya  1253 Nov 23 11:49 CONTRIBUTING.md
-rw-rw-r--. 1 illya illya   888 Nov 23 11:49 environment.yml
drwxrwxr-x. 5 illya illya   251 Nov 23 11:49 esm
drwxrwxr-x. 5 illya illya   239 Nov 23 11:49 examples
-rw-rw-r--. 1 illya illya   814 Nov 23 11:49 hubconf.py
-rw-rw-r--. 1 illya illya  1086 Nov 23 11:49 LICENSE
-rw-rw-r--. 1 illya illya   132 Nov 23 11:49 pyproject.toml
-rw-rw-r--. 1 illya illya 39790 Nov 23 11:49 README.md
drwxrwxr-x. 3 illya illya    65 Nov 23 11:49 scripts
-rw-rw-r--. 1 illya illya  1189 Nov 23 11:49 setup.py
drwxrwxr-x. 2 illya illya   173 Nov 23 11:49 tests
[illya@rockylinux ESM]$

っでテスト

[illya@rockylinux ~]$ module load ESM
[illya@rockylinux ~]$ which python
/apps/pyenv/versions/anaconda3-2022.05/envs/ESM/bin/python
[illya@rockylinux ~]$ python -V
Python 3.9.15
[illya@rockylinux ~]$
[illya@rockylinux ~]$ python
Python 3.9.15 (main, Nov  4 2022, 16:13:54)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import esm
>>> model, alphabet = esm.pretrained.esm2_t33_650M_UR50D()
Downloading: "https://dl.fbaipublicfiles.com/fair-esm/models/esm2_t33_650M_UR50D.pt" to /home/illya/.cache/torch/hub/checkpoints/esm2_t33_650M_UR50D.pt
Downloading: "https://dl.fbaipublicfiles.com/fair-esm/regression/esm2_t33_650M_UR50D-contact-regression.pt" to /home/illya/.cache/torch/hub/checkpoints/esm2_t33_650M_UR50D-contact-regression.pt
>>>
>>> batch_converter = alphabet.get_batch_converter()
>>> model.eval()
ESM2(
  (embed_tokens): Embedding(33, 1280, padding_idx=1)
  (layers): ModuleList(
    (0): TransformerLayer(
 :
 :
    (dense): Linear(in_features=1280, out_features=1280, bias=True)
    (layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
  )
)
>>>
>>> data = [
...     ("protein1", "MKTVRQERLKSIVRILERSKEPVSGAQLAEELSVSRQVIVQDIAYLRSLGYNIVATPRGYVLAGG"),
...     ("protein2", "KALTARQQEVFDLIRDHISQTGMPPTRAEIAQRLGFRSPNAAEEHLKALARKGVIEIVSGASRGIRLLQEE"),
...     ("protein2 with mask","KALTARQQEVFDLIRD<mask>ISQTGMPPTRAEIAQRLGFRSPNAAEEHLKALARKGVIEIVSGASRGIRLLQEE"),
...     ("protein3",  "K A <mask> I S Q"),
... ]
>>>
>>> batch_labels, batch_strs, batch_tokens = batch_converter(data)
>>> batch_lens = (batch_tokens != alphabet.padding_idx).sum(1)
>>>
>>> with torch.no_grad():
...     results = model(batch_tokens, repr_layers=[33], return_contacts=True)
...
>>> token_representations = results["representations"][33]
>>>
>>> sequence_representations = []
>>> for i, tokens_len in enumerate(batch_lens):
...     sequence_representations.append(token_representations[i, 1 : tokens_len - 1].mean(0))
...
>>>
>>> import matplotlib.pyplot as plt
>>> for (_, seq), tokens_len, attention_contacts in zip(data, batch_lens, results["contacts"]):
...     plt.matshow(attention_contacts[: tokens_len, : tokens_len])
...     plt.title(seq)
...     plt.show()

ここで画面が表示されます. 画面を閉じると次の画像が表示されます
2022y11m23d_121239900.png 2022y11m23d_121304571.png 2022y11m23d_121312124.png 2022y11m23d_121320276.png

っでさらにテスト
ESMFフォールド構造予測

[illya@rockylinux ~]$ python
Python 3.9.15 (main, Nov  4 2022, 16:13:54)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import esm
>>> model = esm.pretrained.esmfold_v1()
Downloading: "https://dl.fbaipublicfiles.com/fair-esm/models/esmfold_3B_v1.pt" to /home/illya/.cache/torch/hub/checkpoints/esmfold_3B_v1.pt
Downloading: "https://dl.fbaipublicfiles.com/fair-esm/models/esm2_t36_3B_UR50D.pt" to /home/illya/.cache/torch/hub/checkpoints/esm2_t36_3B_UR50D.pt
Downloading: "https://dl.fbaipublicfiles.com/fair-esm/regression/esm2_t36_3B_UR50D-contact-regression.pt" to /home/illya/.cache/torch/hub/checkpoints/esm2_t36_3B_UR50D-contact-regression.pt
>>>
>>> model = model.eval().cuda()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/apps/pyenv/versions/anaconda3-2022.05/envs/ESM/lib/python3.9/site-packages/torch/nn/modules/module.py", line 747, in cuda
    return self._apply(lambda t: t.cuda(device))
  File "/apps/pyenv/versions/anaconda3-2022.05/envs/ESM/lib/python3.9/site-packages/torch/nn/modules/module.py", line 639, in _apply
    module._apply(fn)
  File "/apps/pyenv/versions/anaconda3-2022.05/envs/ESM/lib/python3.9/site-packages/torch/nn/modules/module.py", line 639, in _apply
    module._apply(fn)
  File "/apps/pyenv/versions/anaconda3-2022.05/envs/ESM/lib/python3.9/site-packages/torch/nn/modules/module.py", line 639, in _apply
    module._apply(fn)
  [Previous line repeated 3 more times]
  File "/apps/pyenv/versions/anaconda3-2022.05/envs/ESM/lib/python3.9/site-packages/torch/nn/modules/module.py", line 662, in _apply
    param_applied = fn(param)
  File "/apps/pyenv/versions/anaconda3-2022.05/envs/ESM/lib/python3.9/site-packages/torch/nn/modules/module.py", line 747, in <lambda>
    return self._apply(lambda t: t.cuda(device))
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 5.80 GiB total capacity; 5.39 GiB already allocated; 15.81 MiB free; 5.62 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
>>>
>>>
>>> exit()
[illya@rockylinux ~]$

A2000(6GB)では無理なのかな?


トップ   編集 添付 複製 名前変更     ヘルプ   最終更新のRSS
Last-modified: 2022-11-23 (水) 17:50:27 (4d)