iSCSI

「古いファイルサーバが複数ある. これを一つにまとめてNFSサーバにしたい」
と要求が来た.
各ファイルサーバの容量はまちまちなので beeGFS やら ceph、GlusterFSらは厳しいかな.
Gfarmがあるけどあれはファイルシステムが別世界なので面倒.

っで基本に立ち直ってそのファイルサーバをiscsi-targetにして、iscsi-initiatorなるクライアントがLVMなファイルサーバにしてみた.

使うOSはCentOS7

iscsi-target†

「yum grouplist」を見ると "Network File System Client"　に「targetcli」がある. まずはこの「tagetcli」でiscsi-targetを作る

[root@iscsi-target ~]# yum install targetcli
 
[root@iscsi-target ~]# yum info targetcli
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: ftp.nara.wide.ad.jp
 * extras: ftp.nara.wide.ad.jp
 * updates: ftp.nara.wide.ad.jp
Installed Packages
Name        : targetcli
Arch        : noarch
Version     : 2.1.53
Release     : 1.el7_9
Size        : 254 k
Repo        : installed
From repo   : updates
Summary     : An administration shell for storage targets
URL         : https://github.com/open-iscsi/targetcli-fb
License     : ASL 2.0
Description : An administration shell for configuring iSCSI, FCoE, and other
            : SCSI targets, using the TCM/LIO kernel target subsystem. FCoE
            : users will also need to install and use fcoe-utils.
 
[root@iscsi-target ~]#

ここではiscsiとして供出するデバイスは /dev/sdb まるごととします。パーティションは切ってないです.
ただ、/dev/sdbと指定するとデバイスが増えた際にIDが変更されてしまう.

なら可変しにくいデバイス名を付けたいもの. ファイルシステムにしてしまえば uuid やら label が使えるが
そのままなんので無理. さりとてLVMにして供出るのもいいのだが、供出した先でLVMでさいど纏うのは微妙...
なので /dev/disk/by-path で指定してみる

[root@iscsi-target ~]# cat /proc/partitions
major minor  #blocks  name
 
   8        0   16777216 sda
   8        1       2048 sda1
   8        2     524288 sda2
   8        3    1048576 sda3
   8        4          1 sda4
   8        5   15200256 sda5
   8       16   33554432 sdb
  11        0    4601856 sr0
[root@iscsi-target ~]#
[root@iscsi-target ~]# ls -l /dev/disk/by-path/
total 0
lrwxrwxrwx. 1 root root  9 Feb 14 21:36 pci-0000:02:03.0-ata-1.0 -> ../../sr0
lrwxrwxrwx. 1 root root  9 Feb 14 21:36 pci-0000:03:00.0-scsi-0:0:0:0 -> ../../sda
lrwxrwxrwx. 1 root root 10 Feb 14 21:36 pci-0000:03:00.0-scsi-0:0:0:0-part1 -> ../../sda1
lrwxrwxrwx. 1 root root 10 Feb 14 21:36 pci-0000:03:00.0-scsi-0:0:0:0-part2 -> ../../sda2
lrwxrwxrwx. 1 root root 10 Feb 14 21:36 pci-0000:03:00.0-scsi-0:0:0:0-part3 -> ../../sda3
lrwxrwxrwx. 1 root root 10 Feb 14 21:36 pci-0000:03:00.0-scsi-0:0:0:0-part4 -> ../../sda4
lrwxrwxrwx. 1 root root 10 Feb 14 21:36 pci-0000:03:00.0-scsi-0:0:0:0-part5 -> ../../sda5
lrwxrwxrwx. 1 root root  9 Feb 14 23:03 pci-0000:03:00.0-scsi-0:0:1:0 -> ../../sdb
[root@iscsi-target ~]#

[root@iscsi-target ~]# targetcli
targetcli shell version 2.1.53
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.
 
/> ls
o- / ............................................................................. [...]
  o- backstores .................................................................. [...]
  | o- block ...................................................... [Storage Objects: 0]
  | o- fileio ..................................................... [Storage Objects: 0]
  | o- pscsi ...................................................... [Storage Objects: 0]
  | o- ramdisk .................................................... [Storage Objects: 0]
  o- iscsi ................................................................ [Targets: 0]
  o- loopback ............................................................. [Targets: 0]
/>
/> cd /backstores/block
/backstores/block> ls
o- block .......................................................... [Storage Objects: 0]
 
/backstores/block> create name=disk1 dev=/dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:1:0
 
Created block storage object disk1 using /dev/sdb.
/backstores/block> ls
o- block .......................................................... [Storage Objects: 1]
  o- disk1  [/dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:1:0 (32.0GiB) write-thru deactivated]
    o- alua ........................................................... [ALUA Groups: 1]
      o- default_tg_pt_gp ............................... [ALUA state: Active/optimized]
 
/backstores/block> cd /
/> ls
o- / ............................................................................. [...]
  o- backstores .................................................................. [...]
  | o- block ...................................................... [Storage Objects: 1]
  | | o- disk1  [/dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:1:0 (32.0GiB) write-thru deactivated]
  | |   o- alua ....................................................... [ALUA Groups: 1]
  | |     o- default_tg_pt_gp ........................... [ALUA state: Active/optimized]
  | o- fileio ..................................................... [Storage Objects: 0]
  | o- pscsi ...................................................... [Storage Objects: 0]
  | o- ramdisk .................................................... [Storage Objects: 0]
  o- iscsi ................................................................ [Targets: 0]
  o- loopback ............................................................. [Targets: 0]
/>

次に IQN の設定
名前は自動で作らせてもいいけど「iqn.2022-02.local.iscsi-target:disk」とします

/> cd /iscsi
/iscsi> create iqn.2022-02.local.iscsi-target:disk
Created target iqn.2022-02.local.iscsi-target:disk.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.
/iscsi>

ただこれですと

/iscsi> ls
o- iscsi .................................................................. [Targets: 1]
  o- iqn.2022-02.local.iscsi-target:disk ..................................... [TPGs: 1]
    o- tpg1 ..................................................... [no-gen-acls, no-auth]
      o- acls ................................................................ [ACLs: 0]
      o- luns ................................................................ [LUNs: 0]
      o- portals .......................................................... [Portals: 1]
        o- 0.0.0.0:3260 ........................................................... [OK]
/iscsi>

次に /dev/sdb と繋げる

/iscsi> cd iqn.2022-02.local.iscsi-target:disk/tpg1/luns
/iscsi/iqn.20...isk/tpg1/luns> 
/iscsi/iqn.20...isk/tpg1/luns> create /backstores/block/disk1
 
/iscsi/iqn.20...isk/tpg1/luns> cd /iscsi
/iscsi> ls
o- iscsi .................................................................. [Targets: 1]
  o- iqn.2022-02.local.iscsi-target:disk ..................................... [TPGs: 1]
    o- tpg1 ..................................................... [no-gen-acls, no-auth]
      o- acls ................................................................ [ACLs: 0]
      o- luns ................................................................ [LUNs: 1]
      | o- lun0  [block/disk1 (/dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:1:0) (default_tg_pt_gp)]
      o- portals .......................................................... [Portals: 1]
        o- 0.0.0.0:3260 ........................................................... [OK]
/iscsi>
 
[root@iscsi-target ~]# targetcli saveconfig

次は接続しようとする iscsi-initiator の「iqn」をACLに登録する. これは後ほど.

FWの設定

[root@iscsi-target ~]# firewall-cmd --add-service=iscsi-target --permanent
[root@iscsi-target ~]# firewall-cmd --reload
 
[root@iscsi-target ~]# systemctl enable target.service
[root@iscsi-target ~]# systemctl start target.service

↑

iscsi-initiator†

iscsi-initiator側の設定

[root@iscsi-initiator ~]# yum install iscsi-initiator-utils

その後に自分が名乗るiqnを付ける. 既定の値が一応あるけど、変更します

[root@iscsi-initiator ~]# cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1994-05.com.redhat:c13c411491f
 ↓
InitiatorName=iqn.2022-02.local.sybyl:iscsi-initiator
[root@iscsi-initiator ~]#

っでここでtarget側にこの「iqn.2022-02.local.sybyl:iscsi-initiator」を伝える

/iscsi> ls
o- iscsi .................................................................. [Targets: 1]
  o- iqn.2022-02.local.iscsi-target:disk ..................................... [TPGs: 1]
    o- tpg1 ..................................................... [no-gen-acls, no-auth]
      o- acls ................................................................ [ACLs: 0]
      o- luns ................................................................ [LUNs: 1]
      | o- lun0  [block/disk1 (/dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:1:0) (default_tg_pt_gp)]
      o- portals .......................................................... [Portals: 1]
        o- 0.0.0.0:3260 ........................................................... [OK]
/iscsi> cd iqn.2022-02.local.iscsi-target:disk/tpg1/acls
 
/iscsi/iqn.20...isk/tpg1/acls> create iqn.2022-02.local.sybyl:iscsi-initiator

そのうえでinitiator側の操作に移る
まず現状のデバイスは

[root@iscsi-initiator ~]# lsblk -S
NAME HCTL       TYPE VENDOR   MODEL             REV TRAN
sda  0:0:0:0    disk VMware   Virtual disk     2.0
sdb  0:0:1:0    disk VMware   Virtual disk     2.0
sr0  3:0:0:0    rom  NECVMWar VMware SATA CD00 1.00 sata
[root@iscsi-initiator ~]#

である。
次にtargetを探して

[root@iscsi-initiator ~]# iscsiadm -m discovery -t st -p 192.168.0.142
192.168.0.142:3260,1 iqn.2022-02.local.iscsi-target:disk
 
[root@iscsi-initiator ~]# iscsiadm -m node --login
 
[root@iscsi-initiator ~]# lsblk -S
NAME HCTL       TYPE VENDOR   MODEL             REV TRAN
sda  0:0:0:0    disk VMware   Virtual disk     2.0
sdb  0:0:1:0    disk VMware   Virtual disk     2.0
sdc  34:0:0:0   disk LIO-ORG  disk1            4.0  iscsi
sr0  3:0:0:0    rom  NECVMWar VMware SATA CD00 1.00 sata
 
[root@iscsi-initiator ~]#
 
(削除は)
iscsiadm -m node -T iqn.2022-02.local.iscsi-target:disk --portal 192.168.0.142:3260,1 --logout

っで一応見えている

↑

めも†

(parted) mklabel gpt
(parted) mkpart primary 0% 100%
(parted) set 1 lvm on
(parted) p
 
 
[root@iscsi-initiator ~]# pvcreate /dev/sdb1
[root@iscsi-initiator ~]# pvcreate /dev/sdc1
[root@iscsi-initiator ~]# pvscan
  PV /dev/sdb1                      lvm2 [<32.00 GiB]
  PV /dev/sdc1                      lvm2 [31.96 GiB]
  Total: 2 [63.96 GiB] / in use: 0 [0   ] / in no VG: 2 [63.96 GiB]
[root@iscsi-initiator ~]# vgcreate VG00 /dev/sdb1 /dev/sdc1
  Volume group "VG00" successfully created
[root@iscsi-initiator ~]#
[root@iscsi-initiator ~]# lvcreate -n L0 -l 100%FREE VG00
[root@iscsi-initiator ~]# pvscan
  PV /dev/sdb1   VG VG00            lvm2 [<32.00 GiB / 0    free]
  PV /dev/sdc1   VG VG00            lvm2 [31.93 GiB / 0    free]
  Total: 2 [<63.93 GiB] / in use: 2 [<63.93 GiB] / in no VG: 0 [0   ]
[root@iscsi-initiator ~]#
 
[root@iscsi-initiator ~]# lvscan
  ACTIVE            '/dev/VG00/L0' [<63.93 GiB] inherit
[root@iscsi-initiator ~]#
[root@iscsi-initiator ~]#
[root@iscsi-initiator ~]# mkfs.xfs -L data /dev/VG00/L0
[root@iscsi-initiator ~]# mkdir /data
[root@iscsi-initiator ~]# mount LABEL=data /data
[root@iscsi-initiator ~]# df -lTh /data
Filesystem          Type  Size  Used Avail Use% Mounted on
/dev/mapper/VG00-L0 xfs    64G   33M   64G   1% /data
[root@iscsi-initiator ~]#

fstabオプション

LABEL=data /data xfs defaults,_netdev        0 0
(あるいは)
LABEL=data /data xfs defaults,_netdev,nofail 0 0

「_netdev」これがないとinitiatorが有効(ネットワークが有効)になる前に処理してしまいエラーになる.なので必須かな. nfs mountにもつけたほうがいいのかも
「nofail」デバイスが存在しないなら無視する. initiatorでの運用の場合は initiatorが正しく働かない場合でも無理やりboot-upしますので、いいかは微妙. boot-up後にinitiator不調を感知、通知機能が必要かな.
initiatorでの運用なら「nofail」はやめたほうが良いみたい. 「nofail」を入れないとそのデバイスで行うサービス(nfsd)とかも止めてくれるので.

御品書き
■samba(更新) DNS
win参加 lin参加
 ユーザ登録(1,2)
ホームディレクトリ
 共有フォルダ(1)
VSS DC追加 GPO SSO
透過的ファイル圧縮
 Server-Side Copy
ルータ超え時刻同期
 ログオンスクリプト
 NAS参加(1) NFS DFS
Mac参加(NFS)
ownCloud
リモートディスクトップ
 ディスクトップ
 smb3マルチチャンネル
 backup
メモ OpenAM
samba/LDAP XP
RSAT nisでsmb
admin-tools