![[PukiWiki]](image/picc.png "[PukiWiki]")
BROADCOM(Avago)の SAS 9300-9iを入手した
rockylinux9.5マシンに載せて起動したら
[root@c2 ~]# lspci -vv
(略
01:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 (rev 02)
Subsystem: Micro-Star International Co., Ltd. [MSI] Device 101e
:
LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM not supported
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
:
LnkSta: Speed 8GT/s (ok), Width x4 (downgraded)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
:
Kernel driver in use: mpt3sas
Kernel modules: mpt3sas
:
[root@c2 ~]#とデバイス自体はPCIe Gen3(8GT/s)の8 lanesのキャパ(LnkCap)を持ち、実際はPCIe Gen3(8GT/s)の4 lanesで繋がっている(LnkSta)。
HBAに搭載されているfirmwareのバージョンを確認するために sas3flash を入手します。
BROADCOMのサポートページhttps://www.broadcom.com/support/download-search
にて、
Product Groupを「Legacy Products」
Product Familyを「Legacy Host Bus Adapters」
Productに「SAS 9300-8i Host Bus Adapter」
Asset typeに「Firmware」を選択して
「Search」ボタンを押下
ヒットした項目からLinux版の最新を選び、それをダウンロードします。
ダウンロードしたファイル(Installer_P16_for_Linux.zip, 910kB)を展開すると、サブフォルダ「 Installer_P16_for_Linux/sas3flash_linux_x64_rel 」にsas3flashコマンドが用意されている。
このファイルをマシンへコピーします。staticなプログラムで単に実行権限を付与すれば使えます。
詳細な使い方は同封のpdfを参照
[root@c2 ~]# chmod +x sas3flash
[root@c2 ~]# ./sas3flash -list
Avago Technologies SAS3 Flash Utility
Version 17.00.00.00 (2018.04.02)
Copyright 2008-2018 Avago Technologies. All rights reserved.
Adapter Selected is a Avago SAS: SAS3008(C0)
Controller Number : 0
Controller : SAS3008(C0)
PCI Address : 00:01:00:00
SAS Address : xxxxxxx-x-xxxx-xxxx
NVDATA Version (Default) : 0b.02.01.03
NVDATA Version (Persistent) : 0b.02.01.03
Firmware Product ID : 0x2221 (IT)
Firmware Version : 12.00.00.00
NVDATA Vendor : LSI
NVDATA Product ID : SAS9300-8i
BIOS Version : 08.29.00.00
UEFI BSD Version : N/A
FCODE Version : N/A
Board Name : SAS9300-8i
Board Assembly : xx-xxxxx-xxx
Board Tracer Number : xxxxxxxxxx
Finished Processing Commands Successfully.
Exiting SAS3Flash.
[root@c2 ~]#firmwareのバージョンは「12.00.00.00」
「Installer_P16_for_UEFI」には「sas3flash.efi」が含まれる
sas3flashコマンドで確認できるそうな. ここではHDDを接続しないで実行してみます
[root@c2 ~]# ./sas3flash -o -testlsall
Avago Technologies SAS3 Flash Utility
Version 17.00.00.00 (2018.04.02)
Copyright 2008-2018 Avago Technologies. All rights reserved.
Advanced Mode Set
Adapter Selected is a Avago SAS: SAS3008(C0)
Executing Operation: Test Link State ALL
Phy 0: Link Down
Phy 1: Link Down
Phy 2: Link Down
Phy 3: Link Down
Phy 4: Link Down
Phy 5: Link Down
Phy 6: Link Down
Phy 7: Link Down
Test Link State All PASSED!
Finished Processing Commands Successfully.
Exiting SAS3Flash.
[root@c2 ~]#検索結果の中に「9300_8e_Package_P16_IT_FW_BIOS_for_MSDOS_Windows」がある。
この中にあるfirmwareを適用します
このzipファイル(9300_8e_Package_P16_IT_FW_BIOS_for_MSDOS_Windows.zip, 2.4MB)をダウンロードして展開して
「 9300_8e_Package_P16_IT_FW_BIOS_for_MSDOS_Windows / Firmware / SAS9300_8e_IT 」にある「SAS9300_8e_IT.bin」(1.1MB) をマシンに運ぶ
っで適用
[root@c2 ~]# ./sas3flash -o -f SAS9300_8e_IT.bin
Avago Technologies SAS3 Flash Utility
Version 17.00.00.00 (2018.04.02)
Copyright 2008-2018 Avago Technologies. All rights reserved.
Advanced Mode Set
Adapter Selected is a Avago SAS: SAS3008(C0)
Executing Operation: Flash Firmware Image
Firmware Image has a Valid Checksum.
Firmware Version 16.00.10.00
Firmware Image compatible with Controller.
Valid NVDATA Image found.
NVDATA Major Version 0e.01
Checking for a compatible NVData image...
NVDATA Device ID and Chip Revision match verified.
NVDATA Versions Compatible.
Valid Initialization Image verified.
Valid BootLoader Image verified.
Beginning Firmware Download...
Firmware Download Successful.
Verifying Download...
Firmware Flash Successful.
Resetting Adapter...
Adapter Successfully Reset.
NVDATA Version 0e.01.00.08
Finished Processing Commands Successfully.
Exiting SAS3Flash.
[root@c2 ~]#
(確認)
[root@c2 ~]# ./sas3flash -list
Avago Technologies SAS3 Flash Utility
Version 17.00.00.00 (2018.04.02)
Copyright 2008-2018 Avago Technologies. All rights reserved.
Adapter Selected is a Avago SAS: SAS3008(C0)
Controller Number : 0
Controller : SAS3008(C0)
PCI Address : 00:01:00:00
SAS Address : 500605b-0-0915-6125
NVDATA Version (Default) : 0e.01.00.08
NVDATA Version (Persistent) : 0e.01.00.08
Firmware Product ID : 0x2221 (IT)
Firmware Version : 16.00.10.00 <-- updateされました
NVDATA Vendor : LSI
NVDATA Product ID : SAS9300-8e
BIOS Version : 08.29.00.00
UEFI BSD Version : N/A
FCODE Version : N/A
Board Name : SAS9300-8i
Board Assembly : H3-25573-00G
Board Tracer Number : SV42828862
Finished Processing Commands Successfully.
Exiting SAS3Flash.
[root@c2 ~]#「9300_8e_Package_P16_IT_FW_BIOS_for_MSDOS_Windows.zip, 2.4MB)」にはBIOSも含まれている
一応これもupdateしてみる。場所は「9300_8e_Package_P16_IT_FW_BIOS_for_MSDOS_Windows / sasbios_rel 」にある「mptsas3.rom」
[root@c2 ~]# ./sas3flash -o -b mptsas3.rom
Avago Technologies SAS3 Flash Utility
Version 17.00.00.00 (2018.04.02)
Copyright 2008-2018 Avago Technologies. All rights reserved.
Advanced Mode Set
Adapter Selected is a Avago SAS: SAS3008(C0)
Executing Operation: Flash BIOS Image
Validating BIOS Image...
BIOS Header Signature is Valid
BIOS Image has a Valid Checksum.
BIOS PCI Structure Signature Valid.
BIOS Image Compatible with the SAS Controller.
Attempting to Flash BIOS Image...
Verifying Download...
Flash BIOS Image Successful.
Finished Processing Commands Successfully.
Exiting SAS3Flash.
[root@c2 ~]#
(確認)
[root@c2 ~]# ./sas3flash -list
Avago Technologies SAS3 Flash Utility
Version 17.00.00.00 (2018.04.02)
Copyright 2008-2018 Avago Technologies. All rights reserved.
Adapter Selected is a Avago SAS: SAS3008(C0)
Controller Number : 0
Controller : SAS3008(C0)
PCI Address : 00:01:00:00
SAS Address : 500605b-0-0915-6125
NVDATA Version (Default) : 0e.01.00.08
NVDATA Version (Persistent) : 0e.01.00.08
Firmware Product ID : 0x2221 (IT)
Firmware Version : 16.00.10.00
NVDATA Vendor : LSI
NVDATA Product ID : SAS9300-8e
BIOS Version : 08.37.00.00 <--- [08.29.00.00]からupdate
UEFI BSD Version : N/A
FCODE Version : N/A
Board Name : SAS9300-8i
Board Assembly : H3-25573-00G
Board Tracer Number : SV42828862
Finished Processing Commands Successfully.
Exiting SAS3Flash.
[root@c2 ~]#上記までの場合は、boot途中で「Ctrl-C」とかで専用ツールに移って現状の確認とかこの「SAS 9300-9i」でできるRAID0/1の設定が可能となる。
っで最近はUEFI向けのBIOSを含むとboot途中からBIOSに入って、その中でHBAの設定ができるみたい。
そのために「SAS 9300-9i」にUEFI向けのBIOSを入れてみる
同じく「9300_8e_Package_P16_IT_FW_BIOS_for_MSDOS_Windows.zip」から
「 9300_8e_Package_P16_IT_FW_BIOS_for_MSDOS_Windows / uefi_bsd_rel 」の「mpt3x64.rom」をマシンに運んで
[root@c2 ~]# ./sas3flash -o -b mpt3x64.rom
Avago Technologies SAS3 Flash Utility
Version 17.00.00.00 (2018.04.02)
Copyright 2008-2018 Avago Technologies. All rights reserved.
Advanced Mode Set
Adapter Selected is a Avago SAS: SAS3008(C0)
Executing Operation: Flash BIOS Image
Validating BIOS Image...
BIOS Header Signature is Valid
BIOS Image has a Valid Checksum.
BIOS PCI Structure Signature Valid.
BIOS Image Compatible with the SAS Controller.
Attempting to Flash BIOS Image...
Verifying Download...
Flash BIOS Image Successful.
Finished Processing Commands Successfully.
Exiting SAS3Flash.
[root@c2 ~]#
[root@c2 ~]# ./sas3flash -list
Avago Technologies SAS3 Flash Utility
Version 17.00.00.00 (2018.04.02)
Copyright 2008-2018 Avago Technologies. All rights reserved.
Adapter Selected is a Avago SAS: SAS3008(C0)
Controller Number : 0
Controller : SAS3008(C0)
PCI Address : 00:01:00:00
SAS Address : 500605b-0-0915-6125
NVDATA Version (Default) : 0e.01.00.08
NVDATA Version (Persistent) : 0e.01.00.08
Firmware Product ID : 0x2221 (IT)
Firmware Version : 16.00.10.00
NVDATA Vendor : LSI
NVDATA Product ID : SAS9300-8e
BIOS Version : 08.37.00.00
UEFI BSD Version : 18.00.00.00 <-- 追加された
FCODE Version : N/A
Board Name : SAS9300-8i
Board Assembly : H3-25573-00G
Board Tracer Number : SV42828862
Finished Processing Commands Successfully.
Exiting SAS3Flash.
[root@c2 ~]#これでBIOS画面から直接HBAを調整できる
hw-RAIDで使ったエンクロージャをHBAに繋げたらsoftware-raidとして認識された模様
root@e:~# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [raid10]
md126 : active raid6 sdh[7] sdc[6] sda[5] sde[4] sdf[3] sdd[2] sdb[1] sdg[0]
5857345536 blocks super external:/md127/0 level 6, 256k chunk, algorithm 10 [8/8] [UUUUUUUU]
[==========>..........] resync = 51.8% (506472340/976224256) finish=88.5min speed=88425K/sec
md127 : inactive sdh[7](S) sdc[6](S) sda[5](S) sde[4](S) sdf[3](S) sdd[2](S) sdb[1](S) sdg[0](S)
4306624 blocks super external:ddf
unused devices: <none>
root@e:~#折角だからrebuild完了後にbonne++1.03eでベンチを取った
| Sequential Output | Sequential Input | Random Seeks | Sequential Create | Random Create | |||||||
| Per Char | Block | Rewrite | Per Char | Block | Create | Read | Delete | Create | Read | Delete | |
| 155247(100) | 582999(14) | 243333(10) | 159262(93) | 495989(14) | 518.9(2) | 91080(99) | 510357(99) | 1692(1) | 88867(99) | 405650(99) | 1444(2) |
| 154876(100) | 593092(15) | 232083(9) | 157886(92) | 476595(12) | 439.5(1) | 90949(99) | 512250(99) | 1791(2) | 1793(2) | 407973(100) | 1030(2) |
| 154888(100) | 606027(15) | 236983(10) | 162408(95) | 520776(14) | 524.3(2) | 91232(99) | 511405(99) | 3113(3) | 1442(1) | 403530(99) | 1293(2) |
dd
root@e:/export# dd if=/dev/zero of=./zero1 bs=1G count=10 conv=fsync
10+0 レコード入力
10+0 レコード出力
10737418240 bytes (11 GB, 10 GiB) copied, 19.2921 s, 557 MB/s
root@e:/export# dd if=/dev/zero of=./zero2 bs=1G count=10 conv=fsync
10+0 レコード入力
10+0 レコード出力
10737418240 bytes (11 GB, 10 GiB) copied, 19.4495 s, 552 MB/s
root@e:/export# dd if=/dev/zero of=./zero3 bs=4k count=2621440 conv=fsync
2621440+0 レコード入力
2621440+0 レコード出力
10737418240 bytes (11 GB, 10 GiB) copied, 18.3251 s, 586 MB/s
root@e:/export#fio(fio-cdm)
| 1st | 2nd | 3rd | ||||
| Read(MB/s) | Write(MB/s) | Read(MB/s) | Write(MB/s) | Read(MB/s) | Write(MB/s) | |
| Seq | 116.377 | 58.679 | 116.802 | 34.788 | 117.404 | 24.044 |
| 512K | 39.238 | 37.078 | 38.420 | 35.230 | 38.531 | 31.961 |
| 4K | 40927.000 | 43436.000 | 40102.000 | 43600.000 | 42688.000 | 40645.000 |
| 4KQD32 | 5.216 | 83921.000 | 5.201 | 78461.000 | 5.211 | 82770.000 |
