blob: f5e32c92460013f71579d641c89ed70b79c1ee19 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
|
<!-- TaskZfsDiskReplacement, Version: 1, Modified: 2018/12/02, Author: trac -->
#Task: ZFS Disk Replacement
The process of replacing mirrored zfs disks is fairly simple. The changes are done by zpool attach and detach.
```sh
zpool detach <pool> <disk-id>
zpool attach <pool> <disk-id-to-mirror> <disk-id-mirrored-to>
```
The heavy lifting is done by zfs itself.
## process
### PREP
* use the [pdf article link](http://trac.suspectdevices.com/trac/wiki/ZFSDiskReplacement?format=pdfarticle) to print this before going down
* If possible pre wipe and check the disks on a separate linux machine _(note: /dev/sdf is an placeholder for the disk mounted on that system)_
```sh
root@homebox:~# wipefs -af --backup /dev/sdf
/dev/sdf: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/dev/sdf: 8 bytes were erased at offset 0x222ee64e00 (gpt): 45 46 49 20 50 41 52 54
/dev/sdf: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdc: calling ioctl to re-read partition table: Success
root@homebox:~# fdisk /dev/sdf
....
Command (m for help): g
Created a new GPT disklabel (GUID: EBC5A0C9-E871-544F-A8EA-E31FCA655F9C).
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
root@homebox:~# badblocks /dev/sdf
....
```
* insure that you can ssh into the box
### On site
The following assumes you have escalated to root privileges (sudo bash), in this case we are replacing /dev/sdc and /dev/sdd in the pool named 'level'
* check for the correct disk.
The following should cause the disk to light up\
_(<CTRL> C when you have identified the disk. Careful with the if/of here)_.
```sh
root@bs2020:~# dd if=/dev/sdc of=/dev/null
* find the disk in the pool.
root@bs2020:~# zpool status
pool: devel
state: ONLINE
scan: resilvered 9.95G in 0h4m with 0 errors on Sat Nov 10 22:00:41 2018
config:
NAME STATE READ WRITE CKSUM
devel ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
scsi-35000c50054fee503 ONLINE 0 0 0
scsi-35000c5005501b45b ONLINE 0 0 0
errors: No known data errors
...
root@bs2020:~# ls -ls /dev/disk/by-id/|grep scsi|grep -v "\-part"
0 lrwxrwxrwx 1 root root 9 Nov 10 21:22 scsi-350000395a8336d34 -> ../../sde
0 lrwxrwxrwx 1 root root 9 Nov 10 21:22 scsi-35000c50054fee503 -> ../../sdd
0 lrwxrwxrwx 1 root root 9 Nov 10 21:56 scsi-35000c5005501b45b -> ../../sdc
0 lrwxrwxrwx 1 root root 9 Nov 10 21:22 scsi-35000cca00b33a264 -> ../../sdf
0 lrwxrwxrwx 1 root root 9 Nov 10 21:22 scsi-3600508e00000000069cf3977618f1408 -> ../../sdg
root@bs2020:~#
```
_We notice above that the disk we are looking for is scsi-35000c5005501b45b_
* detach the disk from the pool.
```sh
root@bs2020:~# zpool detach devel scsi-35000c5005501b45b
root@bs2020:~# zpool status
pool: devel
state: ONLINE
scan: resilvered 9.95G in 0h4m with 0 errors on Sat Nov 10 22:00:41 2018
config:
NAME STATE READ WRITE CKSUM
devel ONLINE 0 0 0
scsi-35000c50054fee503 ONLINE 0 0 0
errors: No known data errors
pool: infra
...
root@bs2020:~#
```
* even if expanding the disk size insure that auto expand is off.
```sh
root@bs2020:~# zpool set autoexpand=off devel
```
* Swap out the old disk with the new one.
* find the new disk's id.
```sh
root@bs2020:~# partprobe
root@bs2020:~# ls -ls /dev/disk/by-id/|grep sdc
0 lrwxrwxrwx 1 root root 9 Nov 10 21:56 scsi-xxxxxxxxxxxxxxx -> ../../sdc
...
0 lrwxrwxrwx 1 root root 9 Nov 10 21:56 xxx-xxxxxxxxxxxxxxx -> ../../sdc
```
* _If the drive id does not change reboot the server_
* attach the new disk to the zfs pool _(scsi-xxxxxxxxxxxxxxxx is the new id from the above step)_
```sh
root@bs2020:~# zpool attach devel scsi-35000c50054fee503 scsi-xxxxxxxxxxxxxx
```
* wait for pool to resliver
```sh
root@bs2020:~# zpool status
pool: devel
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Sat Nov 10 21:56:04 2018
8.54G scanned out of 9.95G at 35.5M/s, 0h0m to go
8.54G resilvered, 85.85% done
config:
NAME STATE READ WRITE CKSUM
devel ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
scsi-35000c50054fee503 ONLINE 0 0 0
scsi-35000c5005501b45b ONLINE 0 0 0 (resilvering)
errors: No known data errors
pool: infra
...
root@bs2020:~# zpool status
.... repeat until finished reslivering ....
root@bs2020:~# zpool status
pool: devel
state: ONLINE
scan: scrub repaired 0B in 0h4m with 0 errors on Sat Nov 10 21:58:04 2018
config:
NAME STATE READ WRITE CKSUM
devel ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
scsi-35000cca00b33a264 ONLINE 0 0 0
scsi-350000395a8336d34 ONLINE 0 0 0
errors: No known data errors
...
```
* if expanding disk check for new size and if not expand it
# zfs list and check for larger disk pool
* repeat process for disk in bay below (we already know its old id from above).
```sh
root@bs2020:~# dd if=/dev/sdd of=/dev/null
root@bs2020:~# zpool detach devel scsi-35000c50054fee503
... swap disks ...
root@bs2020:~# partprobe
root@bs2020:~# ls -ls /dev/disk/by-id/|grep sdd
0 lrwxrwxrwx 1 root root 9 Nov 10 21:56 scsi-yyyyyyyyyyyyyyyy-> ../../sdd
... reboot if necessary ...
root@bs2020:~# wipefs -a /dev/sdd
...
root@bs2020:~# fdisk /dev/sdd
...
root@bs2020:~# zpool attach devel scsi-xxxxxxxxxxxxxx scsi-yyyyyyyyyyyyyyy
... wait for resliver...
```
* use the process below to grow disks to new size
```sh
# zpool set autoexpand=on devel
# zpool online -e devel scsi-xxxxxxxxxxxxxxxxxxxx
# zpool online -e devel scsi-yyyyyyyyyyyyyyyyyyy
# zpool set autoexpand=off devel
```
### references
* https://tomasz.korwel.net/2014/01/03/growing-zfs-pool/
* https://jsosic.wordpress.com/2013/01/01/expanding-zfs-zpool-raid/
* https://serverfault.com/questions/5336/how-do-i-make-linux-recognize-a-new-sata-dev-sda-drive-i-hot-swapped-in-without
|
