| blob | 9821c9fd71113376962ff76681363ef2f8efec4d |
1 .\" Hey, Emacs, edit this file in -*- nroff-fill -*- mode
2 .\"-
3 .\" Copyright (c) 1997, 1998
4 .\" Nan Yang Computer Services Limited. All rights reserved.
5 .\"
6 .\" This software is distributed under the so-called ``Berkeley
7 .\" License'':
8 .\"
9 .\" Redistribution and use in source and binary forms, with or without
10 .\" modification, are permitted provided that the following conditions
11 .\" are met:
12 .\" 1. Redistributions of source code must retain the above copyright
13 .\" notice, this list of conditions and the following disclaimer.
14 .\" 2. Redistributions in binary form must reproduce the above copyright
15 .\" notice, this list of conditions and the following disclaimer in the
16 .\" documentation and/or other materials provided with the distribution.
17 .\" 3. All advertising materials mentioning features or use of this software
18 .\" must display the following acknowledgement:
19 .\" This product includes software developed by Nan Yang Computer
20 .\" Services Limited.
21 .\" 4. Neither the name of the Company nor the names of its contributors
22 .\" may be used to endorse or promote products derived from this software
23 .\" without specific prior written permission.
24 .\"
25 .\" This software is provided ``as is'', and any express or implied
26 .\" warranties, including, but not limited to, the implied warranties of
27 .\" merchantability and fitness for a particular purpose are disclaimed.
28 .\" In no event shall the company or contributors be liable for any
29 .\" direct, indirect, incidental, special, exemplary, or consequential
30 .\" damages (including, but not limited to, procurement of substitute
31 .\" goods or services; loss of use, data, or profits; or business
32 .\" interruption) however caused and on any theory of liability, whether
33 .\" in contract, strict liability, or tort (including negligence or
34 .\" otherwise) arising in any way out of the use of this software, even if
35 .\" advised of the possibility of such damage.
36 .\"
37 .\" $FreeBSD: src/share/man/man4/vinum.4,v 1.22.2.9 2002/04/22 08:19:35 kuriyama Exp $
38 .\"
39 .Dd December 12, 2014
40 .Dt VINUM 4
41 .Os
42 .Sh NAME
43 .Nm vinum
44 .Nd Logical Volume Manager
45 .Sh SYNOPSIS
46 .Cd "pseudo-device vinum"
47 .Sh DESCRIPTION
48 .Nm
49 is a logical volume manager inspired by, but not derived from, the Veritas
50 Volume Manager.
51 It provides the following features:
52 .Bl -bullet
53 .It
54 It provides device-independent logical disks, called
55 .Em volumes .
56 Volumes are
57 not restricted to the size of any disk on the system.
58 .It
59 The volumes consist of one or more
60 .Em plexes ,
61 each of which contain the
62 entire address space of a volume.
63 This represents an implementation of RAID-1
64 (mirroring).
65 Multiple plexes can also be used for
66 .\" XXX What about sparse plexes? Do we want them?
67 .Bl -bullet
68 .It
69 Increased read throughput.
70 .Nm
71 will read data from the least active disk, so if a volume has plexes on multiple
72 disks, more data can be read in parallel.
73 .Nm
74 reads data from only one plex, but it writes data to all plexes.
75 .It
76 Increased reliability.
77 By storing plexes on different disks, data will remain
78 available even if one of the plexes becomes unavailable.
79 In comparison with a
80 RAID-5 plex (see below), using multiple plexes requires more storage space, but
81 gives better performance, particularly in the case of a drive failure.
82 .It
83 Additional plexes can be used for on-line data reorganization.
84 By attaching an
85 additional plex and subsequently detaching one of the older plexes, data can be
86 moved on-line without compromising access.
87 .It
88 An additional plex can be used to obtain a consistent dump of a file system.
89 By
90 attaching an additional plex and detaching at a specific time, the detached plex
91 becomes an accurate snapshot of the file system at the time of detachment.
92 .\" Make sure to flush!
93 .El
94 .It
95 Each plex consists of one or more logical disk slices, called
96 .Em subdisks .
97 Subdisks are defined as a contiguous block of physical disk storage.
98 A plex may
99 consist of any reasonable number of subdisks (in other words, the real limit is
100 not the number, but other factors, such as memory and performance, associated
101 with maintaining a large number of subdisks).
102 .It
103 A number of mappings between subdisks and plexes are available:
104 .Bl -bullet
105 .It
106 .Em "Concatenated plexes"
107 consist of one or more subdisks, each of which
108 is mapped to a contiguous part of the plex address space.
109 .It
110 .Em "Striped plexes"
111 consist of two or more subdisks of equal size.
112 The file
113 address space is mapped in
114 .Em stripes ,
115 integral fractions of the subdisk
116 size.
117 Consecutive plex address space is mapped to stripes in each subdisk in
118 turn.
119 .if t \{\
120 .ig
121 .\" FIXME
122 .br
123 .ne 1.5i
124 .PS
125 move right 2i
126 down
127 SD0: box
128 SD1: box
129 SD2: box
131 "plex 0" at SD0.n+(0,.2)
132 "subdisk 0" rjust at SD0.w-(.2,0)
133 "subdisk 1" rjust at SD1.w-(.2,0)
134 "subdisk 2" rjust at SD2.w-(.2,0)
135 .PE
136 ..
137 .\}
138 The subdisks of a striped plex must all be the same size.
139 .It
140 .Em "RAID-5 plexes"
141 require at least three equal-sized subdisks.
142 They
143 resemble striped plexes, except that in each stripe, one subdisk stores parity
144 information.
145 This subdisk changes in each stripe: in the first stripe, it is the
146 first subdisk, in the second it is the second subdisk, etc.
147 In the event of a
148 single disk failure,
149 .Nm
150 will recover the data based on the information stored on the remaining subdisks.
151 This mapping is particularly suited to read-intensive access.
152 The subdisks of a
153 RAID-5 plex must all be the same size.
154 .\" Make sure to flush!
155 .El
156 .It
157 .Em Drives
158 are the lowest level of the storage hierarchy.
159 They represent disk special
160 devices.
161 .It
162 .Nm
163 offers automatic startup.
164 Unlike
165 .Ux
166 file systems,
167 .Nm
168 volumes contain all the configuration information needed to ensure that they are
169 started correctly when the subsystem is enabled.
170 This is also a significant
171 advantage over the Veritas\(tm File System.
172 This feature regards the presence
173 of the volumes.
174 It does not mean that the volumes will be mounted
175 automatically, since the standard startup procedures with
176 .Pa /etc/fstab
177 perform this function.
178 .El
179 .Sh KERNEL CONFIGURATION
180 .Nm
181 is currently supplied as a KLD module, and does not require
182 configuration.
183 As with other klds, it is absolutely necessary to match the kld
184 to the version of the operating system.
185 Failure to do so will cause
186 .Nm
187 to issue an error message and terminate.
188 .Pp
189 It is possible to configure
190 .Nm
191 in the kernel, but this is not recommended.
192 To do so, add this line to the
193 kernel configuration file:
194 .Pp
195 .D1 Cd "pseudo-device vinum"
196 .Ss Debug Options
197 The current version of
198 .Nm ,
199 both the kernel module and the user program
200 .Xr vinum 8 ,
201 include significant debugging support.
202 It is not recommended to remove
203 this support at the moment, but if you do you must remove it from both the
204 kernel and the user components.
205 To do this, edit the files
206 .Pa /usr/src/sbin/vinum/Makefile
207 and
208 .Pa /sys/dev/raid/vinum/Makefile
209 and edit the
210 .Va CFLAGS
211 variable to remove the
212 .Li -DVINUMDEBUG
213 option.
214 If you have
215 configured
216 .Nm
217 into the kernel, either specify the line
218 .Pp
219 .D1 Cd "options VINUMDEBUG"
220 .Pp
221 in the kernel configuration file or remove the
222 .Li -DVINUMDEBUG
223 option from
224 .Pa /usr/src/sbin/vinum/Makefile
225 as described above.
226 .Pp
227 If the
228 .Va VINUMDEBUG
229 variables do not match,
230 .Xr vinum 8
231 will fail with a message
232 explaining the problem and what to do to correct it.
233 .Pp
234 .Nm
235 was previously available in two versions: a freely available version which did
236 not contain RAID-5 functionality, and a full version including RAID-5
237 functionality, which was available only from Cybernet Systems Inc.
238 The present
239 version of
240 .Nm
241 includes the RAID-5 functionality.
242 .Sh RUNNING VINUM
243 .Nm
244 is part of the base
245 .Dx
246 system.
247 It does not require installation.
248 To start it, start the
249 .Xr vinum 8
250 program, which will load the kld if it is not already present.
251 Before using
252 .Nm ,
253 it must be configured.
254 See
255 .Xr vinum 8
256 for information on how to create a
257 .Nm
258 configuration.
259 .Pp
260 Normally, you start a configured version of
261 .Nm
262 at boot time.
263 Set the variable
264 .Va start_vinum
265 in
266 .Pa /etc/rc.conf
267 to
268 .Dq Li YES
269 to start
270 .Nm
271 at boot time.
272 (See
273 .Xr rc.conf 5
274 for more details.)
275 .Pp
276 If
277 .Nm
278 is loaded as a kld (the recommended way), the
279 .Nm Cm stop
280 command will unload it
281 (see
282 .Xr vinum 8 ) .
283 You can also do this with the
284 .Xr kldunload 8
285 command.
286 .Pp
287 The kld can only be unloaded when idle, in other words when no volumes are
288 mounted and no other instances of the
289 .Xr vinum 8
290 program are active.
291 Unloading the kld does not harm the data in the volumes.
292 .Ss Configuring and Starting Objects
293 Use the
294 .Xr vinum 8
295 utility to configure and start
296 .Nm
297 objects.
298 .Sh IOCTL CALLS
299 .Xr ioctl 2
300 calls are intended for the use of the
301 .Xr vinum 8
302 configuration program only.
303 They are described in the header file
304 .Pa /sys/dev/raid/vinum/vinumio.h .
305 .Ss Disk Labels
306 Conventional disk special devices have a
307 .Em "disk label"
308 in the second sector of the device.
309 See
310 .Xr disklabel 5
311 for more details.
312 This disk label describes the layout of the partitions within
313 the device.
314 .Nm
315 does not subdivide volumes, so volumes do not contain a physical disk label.
316 For convenience,
317 .Nm
318 implements the ioctl calls
319 .Dv DIOCGDINFO
320 (get disk label),
321 .Dv DIOCGPART
322 (get partition information),
323 .Dv DIOCWDINFO
324 (write partition information) and
325 .Dv DIOCSDINFO
326 (set partition information).
327 .Dv DIOCGDINFO
328 and
329 .Dv DIOCGPART
330 refer to an internal
331 representation of the disk label which is not present on the volume.
332 As a
333 result, the
334 .Fl r
335 option of
336 .Xr disklabel 8 ,
337 which reads the
338 .Dq "raw disk" ,
339 will fail.
340 .Pp
341 In general,
342 .Xr disklabel 8
343 serves no useful purpose on a
344 .Nm
345 volume.
346 .Pp
347 .Nm
348 ignores the
349 .Dv DIOCWDINFO
350 and
351 .Dv DIOCSDINFO ioctls, since there is nothing to change.
352 As a result, any attempt to modify the disk label will be silently ignored.
353 .Sh MAKING FILE SYSTEMS
354 Since
355 .Nm
356 volumes do not contain partitions, the names do not need to conform to the
357 standard rules for naming disk partitions.
358 For a physical disk partition, the
359 last letter of the device name specifies the partition identifier (a to p).
360 .Nm
361 volumes need not conform to this convention, but if they do not,
362 .Xr newfs 8
363 will complain that it cannot determine the partition.
364 To solve this problem,
365 use the
366 .Fl v
367 flag to
368 .Xr newfs 8 .
369 For example, if you have a volume
370 .Pa concat ,
371 use the following command to create a
372 .Xr UFS 5
373 file system on it:
374 .Pp
375 .Dl "newfs -v /dev/vinum/concat"
376 .Sh OBJECT NAMING
377 .Nm
378 assigns default names to plexes and subdisks, although they may be overridden.
379 We do not recommend overriding the default names.
380 Experience with the
381 Veritas\(tm
382 volume manager, which allows arbitrary naming of objects, has shown that this
383 flexibility does not bring a significant advantage, and it can cause confusion.
384 .Pp
385 Names may contain any non-blank character, but it is recommended to restrict
386 them to letters, digits and the underscore characters.
387 The names of volumes,
388 plexes and subdisks may be up to 64 characters long, and the names of drives may
389 up to 32 characters long.
390 When choosing volume and plex names, bear in mind
391 that automatically generated plex and subdisk names are longer than the name
392 from which they are derived.
393 .Bl -bullet
394 .It
395 When
396 .Nm
397 creates or deletes objects, it creates a directory
398 .Pa /dev/vinum ,
399 in which it makes device entries for each volume.
400 It also creates the
401 subdirectories,
402 .Pa /dev/vinum/plex
403 and
404 .Pa /dev/vinum/sd ,
405 in which it stores device entries for the plexes and subdisks. In addition, it
406 creates two more directories,
407 .Pa /dev/vinum/vol
408 and
409 .Pa /dev/vinum/drive ,
410 in which it stores hierarchical information for volumes and drives.
411 .It
412 In addition,
413 .Nm
414 creates three super-devices,
415 .Pa /dev/vinum/control ,
416 .Pa /dev/vinum/Control
417 and
418 .Pa /dev/vinum/controld .
419 .Pa /dev/vinum/control
420 is used by
421 .Xr vinum 8
422 when it has been compiled without the
423 .Dv VINUMDEBUG
424 option,
425 .Pa /dev/vinum/Control
426 is used by
427 .Xr vinum 8
428 when it has been compiled with the
429 .Dv VINUMDEBUG
430 option, and
431 .Pa /dev/vinum/controld
432 is used by the
433 .Nm
434 daemon.
435 The two control devices for
436 .Xr vinum 8
437 are used to synchronize the debug status of kernel and user modules.
438 .It
439 Unlike
440 .Ux
441 drives,
442 .Nm
443 volumes are not subdivided into partitions, and thus do not contain a disk
444 label.
445 Unfortunately, this confuses a number of utilities, notably
446 .Xr newfs 8 ,
447 which normally tries to interpret the last letter of a
448 .Nm
449 volume name as a partition identifier.
450 If you use a volume name which does not
451 end in the letters
452 .Ql a
453 to
454 .Ql c ,
455 you must use the
456 .Fl v
457 flag to
458 .Xr newfs 8
459 in order to tell it to ignore this convention.
460 .\"
461 .It
462 Plexes do not need to be assigned explicit names.
463 By default, a plex name is
464 the name of the volume followed by the letters
465 .Pa .p
466 and the number of the
467 plex.
468 For example, the plexes of volume
469 .Pa vol3
470 are called
471 .Pa vol3.p0 , vol3.p1
472 and so on.
473 These names can be overridden, but it is not recommended.
474 .It
475 Like plexes, subdisks are assigned names automatically, and explicit naming is
476 discouraged.
477 A subdisk name is the name of the plex followed by the letters
478 .Pa .s
479 and a number identifying the subdisk.
480 For example, the subdisks of
481 plex
482 .Pa vol3.p0
483 are called
484 .Pa vol3.p0.s0 , vol3.p0.s1
485 and so on.
486 .It
487 By contrast,
488 .Em drives
489 must be named.
490 This makes it possible to move a drive to a different location
491 and still recognize it automatically.
492 Drive names may be up to 32 characters
493 long.
494 .El
495 .Ss Example
496 Assume the
497 .Nm
498 objects described in the section
499 .Sx "CONFIGURATION FILE"
500 in
501 .Xr vinum 8 .
502 The directory
503 .Pa /dev/vinum
504 looks like:
505 .Bd -literal -offset indent
506 # ls -lR /dev/vinum
507 total 5
508 crwxr-xr-- 1 root wheel 91, 2 Mar 30 16:08 concat
509 crwx------ 1 root wheel 91, 0x40000000 Mar 30 16:08 control
510 crwx------ 1 root wheel 91, 0x40000001 Mar 30 16:08 controld
511 drwxrwxrwx 2 root wheel 512 Mar 30 16:08 drive
512 drwxrwxrwx 2 root wheel 512 Mar 30 16:08 plex
513 drwxrwxrwx 2 root wheel 512 Mar 30 16:08 rvol
514 drwxrwxrwx 2 root wheel 512 Mar 30 16:08 sd
515 crwxr-xr-- 1 root wheel 91, 3 Mar 30 16:08 strcon
516 crwxr-xr-- 1 root wheel 91, 1 Mar 30 16:08 stripe
517 crwxr-xr-- 1 root wheel 91, 0 Mar 30 16:08 tinyvol
518 drwxrwxrwx 7 root wheel 512 Mar 30 16:08 vol
519 crwxr-xr-- 1 root wheel 91, 4 Mar 30 16:08 vol5
521 /dev/vinum/drive:
522 total 0
523 crw-r----- 1 root operator 4, 15 Oct 21 16:51 drive2
524 crw-r----- 1 root operator 4, 31 Oct 21 16:51 drive4
526 /dev/vinum/plex:
527 total 0
528 crwxr-xr-- 1 root wheel 91, 0x10000002 Mar 30 16:08 concat.p0
529 crwxr-xr-- 1 root wheel 91, 0x10010002 Mar 30 16:08 concat.p1
530 crwxr-xr-- 1 root wheel 91, 0x10000003 Mar 30 16:08 strcon.p0
531 crwxr-xr-- 1 root wheel 91, 0x10010003 Mar 30 16:08 strcon.p1
532 crwxr-xr-- 1 root wheel 91, 0x10000001 Mar 30 16:08 stripe.p0
533 crwxr-xr-- 1 root wheel 91, 0x10000000 Mar 30 16:08 tinyvol.p0
534 crwxr-xr-- 1 root wheel 91, 0x10000004 Mar 30 16:08 vol5.p0
535 crwxr-xr-- 1 root wheel 91, 0x10010004 Mar 30 16:08 vol5.p1
537 /dev/vinum/sd:
538 total 0
539 crwxr-xr-- 1 root wheel 91, 0x20000002 Mar 30 16:08 concat.p0.s0
540 crwxr-xr-- 1 root wheel 91, 0x20100002 Mar 30 16:08 concat.p0.s1
541 crwxr-xr-- 1 root wheel 91, 0x20010002 Mar 30 16:08 concat.p1.s0
542 crwxr-xr-- 1 root wheel 91, 0x20000003 Mar 30 16:08 strcon.p0.s0
543 crwxr-xr-- 1 root wheel 91, 0x20100003 Mar 30 16:08 strcon.p0.s1
544 crwxr-xr-- 1 root wheel 91, 0x20010003 Mar 30 16:08 strcon.p1.s0
545 crwxr-xr-- 1 root wheel 91, 0x20110003 Mar 30 16:08 strcon.p1.s1
546 crwxr-xr-- 1 root wheel 91, 0x20000001 Mar 30 16:08 stripe.p0.s0
547 crwxr-xr-- 1 root wheel 91, 0x20100001 Mar 30 16:08 stripe.p0.s1
548 crwxr-xr-- 1 root wheel 91, 0x20000000 Mar 30 16:08 tinyvol.p0.s0
549 crwxr-xr-- 1 root wheel 91, 0x20100000 Mar 30 16:08 tinyvol.p0.s1
550 crwxr-xr-- 1 root wheel 91, 0x20000004 Mar 30 16:08 vol5.p0.s0
551 crwxr-xr-- 1 root wheel 91, 0x20100004 Mar 30 16:08 vol5.p0.s1
552 crwxr-xr-- 1 root wheel 91, 0x20010004 Mar 30 16:08 vol5.p1.s0
553 crwxr-xr-- 1 root wheel 91, 0x20110004 Mar 30 16:08 vol5.p1.s1
555 /dev/vinum/vol:
556 total 5
557 crwxr-xr-- 1 root wheel 91, 2 Mar 30 16:08 concat
558 drwxr-xr-x 4 root wheel 512 Mar 30 16:08 concat.plex
559 crwxr-xr-- 1 root wheel 91, 3 Mar 30 16:08 strcon
560 drwxr-xr-x 4 root wheel 512 Mar 30 16:08 strcon.plex
561 crwxr-xr-- 1 root wheel 91, 1 Mar 30 16:08 stripe
562 drwxr-xr-x 3 root wheel 512 Mar 30 16:08 stripe.plex
563 crwxr-xr-- 1 root wheel 91, 0 Mar 30 16:08 tinyvol
564 drwxr-xr-x 3 root wheel 512 Mar 30 16:08 tinyvol.plex
565 crwxr-xr-- 1 root wheel 91, 4 Mar 30 16:08 vol5
566 drwxr-xr-x 4 root wheel 512 Mar 30 16:08 vol5.plex
568 /dev/vinum/vol/concat.plex:
569 total 2
570 crwxr-xr-- 1 root wheel 91, 0x10000002 Mar 30 16:08 concat.p0
571 drwxr-xr-x 2 root wheel 512 Mar 30 16:08 concat.p0.sd
572 crwxr-xr-- 1 root wheel 91, 0x10010002 Mar 30 16:08 concat.p1
573 drwxr-xr-x 2 root wheel 512 Mar 30 16:08 concat.p1.sd
575 /dev/vinum/vol/concat.plex/concat.p0.sd:
576 total 0
577 crwxr-xr-- 1 root wheel 91, 0x20000002 Mar 30 16:08 concat.p0.s0
578 crwxr-xr-- 1 root wheel 91, 0x20100002 Mar 30 16:08 concat.p0.s1
580 /dev/vinum/vol/concat.plex/concat.p1.sd:
581 total 0
582 crwxr-xr-- 1 root wheel 91, 0x20010002 Mar 30 16:08 concat.p1.s0
584 /dev/vinum/vol/strcon.plex:
585 total 2
586 crwxr-xr-- 1 root wheel 91, 0x10000003 Mar 30 16:08 strcon.p0
587 drwxr-xr-x 2 root wheel 512 Mar 30 16:08 strcon.p0.sd
588 crwxr-xr-- 1 root wheel 91, 0x10010003 Mar 30 16:08 strcon.p1
589 drwxr-xr-x 2 root wheel 512 Mar 30 16:08 strcon.p1.sd
591 /dev/vinum/vol/strcon.plex/strcon.p0.sd:
592 total 0
593 crwxr-xr-- 1 root wheel 91, 0x20000003 Mar 30 16:08 strcon.p0.s0
594 crwxr-xr-- 1 root wheel 91, 0x20100003 Mar 30 16:08 strcon.p0.s1
596 /dev/vinum/vol/strcon.plex/strcon.p1.sd:
597 total 0
598 crwxr-xr-- 1 root wheel 91, 0x20010003 Mar 30 16:08 strcon.p1.s0
599 crwxr-xr-- 1 root wheel 91, 0x20110003 Mar 30 16:08 strcon.p1.s1
601 /dev/vinum/vol/stripe.plex:
602 total 1
603 crwxr-xr-- 1 root wheel 91, 0x10000001 Mar 30 16:08 stripe.p0
604 drwxr-xr-x 2 root wheel 512 Mar 30 16:08 stripe.p0.sd
606 /dev/vinum/vol/stripe.plex/stripe.p0.sd:
607 total 0
608 crwxr-xr-- 1 root wheel 91, 0x20000001 Mar 30 16:08 stripe.p0.s0
609 crwxr-xr-- 1 root wheel 91, 0x20100001 Mar 30 16:08 stripe.p0.s1
611 /dev/vinum/vol/tinyvol.plex:
612 total 1
613 crwxr-xr-- 1 root wheel 91, 0x10000000 Mar 30 16:08 tinyvol.p0
614 drwxr-xr-x 2 root wheel 512 Mar 30 16:08 tinyvol.p0.sd
616 /dev/vinum/vol/tinyvol.plex/tinyvol.p0.sd:
617 total 0
618 crwxr-xr-- 1 root wheel 91, 0x20000000 Mar 30 16:08 tinyvol.p0.s0
619 crwxr-xr-- 1 root wheel 91, 0x20100000 Mar 30 16:08 tinyvol.p0.s1
621 /dev/vinum/vol/vol5.plex:
622 total 2
623 crwxr-xr-- 1 root wheel 91, 0x10000004 Mar 30 16:08 vol5.p0
624 drwxr-xr-x 2 root wheel 512 Mar 30 16:08 vol5.p0.sd
625 crwxr-xr-- 1 root wheel 91, 0x10010004 Mar 30 16:08 vol5.p1
626 drwxr-xr-x 2 root wheel 512 Mar 30 16:08 vol5.p1.sd
628 /dev/vinum/vol/vol5.plex/vol5.p0.sd:
629 total 0
630 crwxr-xr-- 1 root wheel 91, 0x20000004 Mar 30 16:08 vol5.p0.s0
631 crwxr-xr-- 1 root wheel 91, 0x20100004 Mar 30 16:08 vol5.p0.s1
633 /dev/vinum/vol/vol5.plex/vol5.p1.sd:
634 total 0
635 crwxr-xr-- 1 root wheel 91, 0x20010004 Mar 30 16:08 vol5.p1.s0
636 crwxr-xr-- 1 root wheel 91, 0x20110004 Mar 30 16:08 vol5.p1.s1
637 .Ed
638 .Pp
639 In the case of unattached plexes and subdisks, the naming is reversed.
640 Subdisks
641 are named after the disk on which they are located, and plexes are named after
642 the subdisk.
643 .\" XXX
644 .Bf -symbolic
645 This mapping is still to be determined.
646 .Ef
647 .Ss Object States
648 Each
649 .Nm
650 object has a
651 .Em state
652 associated with it.
653 .Nm
654 uses this state to determine the handling of the object.
655 .Ss Volume States
656 Volumes may have the following states:
657 .Bl -hang -width 14n
658 .It Em down
659 The volume is completely inaccessible.
660 .It Em up
661 The volume is up and at least partially functional.
662 Not all plexes may be
663 available.
664 .El
665 .Ss "Plex States"
666 Plexes may have the following states:
667 .Bl -hang -width 14n
668 .It Em referenced
669 A plex entry which has been referenced as part of a volume, but which is
670 currently not known.
671 .It Em faulty
672 A plex which has gone completely down because of I/O errors.
673 .It Em down
674 A plex which has been taken down by the administrator.
675 .It Em initializing
676 A plex which is being initialized.
677 .El
678 .Pp
679 The remaining states represent plexes which are at least partially up.
680 .Bl -hang -width 14n
681 .It Em corrupt
682 A plex entry which is at least partially up.
683 Not all subdisks are available,
684 and an inconsistency has occurred.
685 If no other plex is uncorrupted, the volume
686 is no longer consistent.
687 .It Em degraded
688 A RAID-5 plex entry which is accessible, but one subdisk is down, requiring
689 recovery for many I/O requests.
690 .It Em flaky
691 A plex which is really up, but which has a reborn subdisk which we do not
692 completely trust, and which we do not want to read if we can avoid it.
693 .It Em up
694 A plex entry which is completely up.
695 All subdisks are up.
696 .El
697 .Ss "Subdisk States"
698 Subdisks can have the following states:
699 .Bl -hang -width 14n
700 .It Em empty
701 A subdisk entry which has been created completely.
702 All fields are correct, and
703 the disk has been updated, but the on the disk is not valid.
704 .It Em referenced
705 A subdisk entry which has been referenced as part of a plex, but which is
706 currently not known.
707 .It Em initializing
708 A subdisk entry which has been created completely and which is currently being
709 initialized.
710 .El
711 .Pp
712 The following states represent invalid data.
713 .Bl -hang -width 14n
714 .It Em obsolete
715 A subdisk entry which has been created completely.
716 All fields are correct, the
717 config on disk has been updated, and the data was valid, but since then the
718 drive has been taken down, and as a result updates have been missed.
719 .It Em stale
720 A subdisk entry which has been created completely.
721 All fields are correct, the
722 disk has been updated, and the data was valid, but since then the drive has been
723 crashed and updates have been lost.
724 .El
725 .Pp
726 The following states represent valid, inaccessible data.
727 .Bl -hang -width 14n
728 .It Em crashed
729 A subdisk entry which has been created completely.
730 All fields are correct, the
731 disk has been updated, and the data was valid, but since then the drive has gone
732 down.
733 No attempt has been made to write to the subdisk since the crash, so the
734 data is valid.
735 .It Em down
736 A subdisk entry which was up, which contained valid data, and which was taken
737 down by the administrator.
738 The data is valid.
739 .It Em reviving
740 The subdisk is currently in the process of being revived.
741 We can write but not
742 read.
743 .El
744 .Pp
745 The following states represent accessible subdisks with valid data.
746 .Bl -hang -width 14n
747 .It Em reborn
748 A subdisk entry which has been created completely.
749 All fields are correct, the
750 disk has been updated, and the data was valid, but since then the drive has gone
751 down and up again.
752 No updates were lost, but it is possible that the subdisk
753 has been damaged.
754 We won't read from this subdisk if we have a choice.
755 If this
756 is the only subdisk which covers this address space in the plex, we set its
757 state to up under these circumstances, so this status implies that there is
758 another subdisk to fulfil the request.
759 .It Em up
760 A subdisk entry which has been created completely.
761 All fields are correct, the
762 disk has been updated, and the data is valid.
763 .El
764 .Ss "Drive States"
765 Drives can have the following states:
766 .Bl -hang -width 14n
767 .It Em referenced
768 At least one subdisk refers to the drive, but it is not currently accessible to
769 the system.
770 No device name is known.
771 .It Em down
772 The drive is not accessible.
773 .It Em up
774 The drive is up and running.
775 .El
776 .Sh DEBUGGING PROBLEMS WITH VINUM
777 Solving problems with
778 .Nm
779 can be a difficult affair.
780 This section suggests some approaches.
781 .Ss Configuration problems
782 It is relatively easy (too easy) to run into problems with the
783 .Nm
784 configuration.
785 If you do, the first thing you should do is stop configuration
786 updates:
787 .Pp
788 .Dl "vinum setdaemon 4"
789 .Pp
790 This will stop updates and any further corruption of the on-disk configuration.
791 .Pp
792 Next, look at the on-disk configuration with the
793 .Nm Cm dumpconfig
794 command, for example:
795 .if t .ps -3
796 .if t .vs -3
797 .Bd -literal
798 # \fBvinum dumpconfig\fP
799 Drive 4: Device /dev/da3s0h
800 Created on crash.lemis.com at Sat May 20 16:32:44 2000
801 Config last updated Sat May 20 16:32:56 2000
802 Size: 601052160 bytes (573 MB)
803 volume obj state up
804 volume src state up
805 volume raid state down
806 volume r state down
807 volume foo state up
808 plex name obj.p0 state corrupt org concat vol obj
809 plex name obj.p1 state corrupt org striped 128b vol obj
810 plex name src.p0 state corrupt org striped 128b vol src
811 plex name src.p1 state up org concat vol src
812 plex name raid.p0 state faulty org disorg vol raid
813 plex name r.p0 state faulty org disorg vol r
814 plex name foo.p0 state up org concat vol foo
815 plex name foo.p1 state faulty org concat vol foo
816 sd name obj.p0.s0 drive drive2 plex obj.p0 state reborn len 409600b driveoffset 265b plexoffset 0b
817 sd name obj.p0.s1 drive drive4 plex obj.p0 state up len 409600b driveoffset 265b plexoffset 409600b
818 sd name obj.p1.s0 drive drive1 plex obj.p1 state up len 204800b driveoffset 265b plexoffset 0b
819 sd name obj.p1.s1 drive drive2 plex obj.p1 state reborn len 204800b driveoffset 409865b plexoffset 128b
820 sd name obj.p1.s2 drive drive3 plex obj.p1 state up len 204800b driveoffset 265b plexoffset 256b
821 sd name obj.p1.s3 drive drive4 plex obj.p1 state up len 204800b driveoffset 409865b plexoffset 384b
822 .Ed
823 .if t .vs +3
824 .if t .ps +3
825 .Pp
826 The configuration on all disks should be the same.
827 If this is not the case,
828 please save the output to a file and report the problem.
829 There is probably
830 little that can be done to recover the on-disk configuration, but if you keep a
831 copy of the files used to create the objects, you should be able to re-create
832 them.
833 The
834 .Cm create
835 command does not change the subdisk data, so this will not cause data
836 corruption.
837 You may need to use the
838 .Cm resetconfig
839 command if you have this kind of trouble.
840 .Ss Kernel Panics
841 In order to analyse a panic which you suspect comes from
842 .Nm
843 you will need to build a debug kernel.
844 See the online handbook at
845 .Pa http://www.dragonflybsd.org/docs/user/list/DebugKernelCrashDumps/
846 for more details of how to do this.
847 .Pp
848 Perform the following steps to analyse a
849 .Nm
850 problem:
851 .Bl -enum
852 .It
853 Copy the following files to the directory in which you will be
854 performing the analysis, typically
855 .Pa /var/crash :
856 .Pp
857 .Bl -bullet -compact
858 .It
859 .Pa /sys/dev/raid/vinum/.gdbinit.crash ,
860 .It
861 .Pa /sys/dev/raid/vinum/.gdbinit.kernel ,
862 .It
863 .Pa /sys/dev/raid/vinum/.gdbinit.serial ,
864 .It
865 .Pa /sys/dev/raid/vinum/.gdbinit.vinum
866 and
867 .It
868 .Pa /sys/dev/raid/vinum/.gdbinit.vinum.paths
869 .El
870 .It
871 Make sure that you build the
872 .Nm
873 module with debugging information.
874 The standard
875 .Pa Makefile
876 builds a module with debugging symbols by default.
877 If the version of
878 .Nm
879 in
880 .Pa /boot/kernel
881 does not contain symbols, you will not get an error message, but the stack trace
882 will not show the symbols.
883 Check the module before starting
884 .Xr kgdb 1 :
885 .Bd -literal
886 $ file /boot/kernel/vinum.ko
887 /boot/kernel/vinum.ko: ELF 32-bit LSB shared object, Intel 80386,
888 version 1 (SYSV), dynamically linked, not stripped
889 .Ed
890 .Pp
891 If the output shows that
892 .Pa /boot/kernel/vinum.ko
893 is stripped, you will have to find a version which is not.
894 Usually this will be
895 either in
896 .Pa /usr/obj/usr/src/sys/SYSTEM_NAME/usr/src/sys/dev/raid/vinum/vinum.ko
897 (if you have built
898 .Nm
899 with a
900 .Dq Li "make world" )
901 or
902 .Pa /sys/dev/raid/vinum/vinum.ko
903 (if you have built
904 .Nm
905 in this directory).
906 Modify the file
907 .Pa .gdbinit.vinum.paths
908 accordingly.
909 .It
910 Either take a dump or use remote serial
911 .Xr gdb 1
912 to analyse the problem.
913 To analyse a dump, say
914 .Pa /var/crash/vmcore.5 ,
915 link
916 .Pa /var/crash/.gdbinit.crash
917 to
918 .Pa /var/crash/.gdbinit
919 and enter:
920 .Bd -literal -offset indent
921 cd /var/crash
922 kgdb kernel.debug vmcore.5
923 .Ed
924 .Pp
925 This example assumes that you have installed the correct debug kernel at
926 .Pa /var/crash/kernel.debug .
927 If not, substitute the correct name of the debug kernel.
928 .Pp
929 To perform remote serial debugging,
930 link
931 .Pa /var/crash/.gdbinit.serial
932 to
933 .Pa /var/crash/.gdbinit
934 and enter
935 .Bd -literal -offset indent
936 cd /var/crash
937 kgdb kernel.debug
938 .Ed
939 .Pp
940 In this case, the
941 .Pa .gdbinit
942 file performs the functions necessary to establish connection.
943 The remote
944 machine must already be in debug mode: enter the kernel debugger and select
945 .Ic gdb
946 (see
947 .Xr ddb 4
948 for more details.)
949 The serial
950 .Pa .gdbinit
951 file expects the serial connection to run at 38400 bits per second; if you run
952 at a different speed, edit the file accordingly (look for the
953 .Va remotebaud
954 specification).
955 .Pp
956 The following example shows a remote debugging session using the
957 .Ic debug
958 command of
959 .Xr vinum 8 :
960 .Bd -literal
961 .if t .ps -3
962 .if t .vs -3
963 GDB 4.16 (i386-unknown-dragonfly), Copyright 1996 Free Software Foundation, Inc.
964 Debugger (msg=0xf1093174 "vinum debug") at ../../i386/i386/db_interface.c:318
965 318 in_Debugger = 0;
966 #1 0xf108d9bc in vinumioctl (dev=0x40001900, cmd=0xc008464b, data=0xf6dedee0 "",
967 flag=0x3, p=0xf68b7940) at
968 /usr/src/sys/dev/raid/vinum/vinumioctl.c:102
969 102 Debugger ("vinum debug");
970 (kgdb) bt
971 #0 Debugger (msg=0xf0f661ac "vinum debug") at ../../i386/i386/db_interface.c:318
972 #1 0xf0f60a7c in vinumioctl (dev=0x40001900, cmd=0xc008464b, data=0xf6923ed0 "",
973 flag=0x3, p=0xf688e6c0) at
974 /usr/src/sys/dev/raid/vinum/vinumioctl.c:109
975 #2 0xf01833b7 in spec_ioctl (ap=0xf6923e0c) at ../../miscfs/specfs/spec_vnops.c:424
976 #3 0xf0182cc9 in spec_vnoperate (ap=0xf6923e0c) at ../../miscfs/specfs/spec_vnops.c:129
977 #4 0xf01eb3c1 in ufs_vnoperatespec (ap=0xf6923e0c) at ../../ufs/ufs/ufs_vnops.c:2312
978 #5 0xf017dbb1 in vn_ioctl (fp=0xf1007ec0, com=0xc008464b, data=0xf6923ed0 "",
979 p=0xf688e6c0) at vnode_if.h:395
980 #6 0xf015dce0 in ioctl (p=0xf688e6c0, uap=0xf6923f84) at ../../kern/sys_generic.c:473
981 #7 0xf0214c0b in syscall (frame={tf_es = 0x27, tf_ds = 0x27, tf_edi = 0xefbfcff8,
982 tf_esi = 0x1, tf_ebp = 0xefbfcf90, tf_isp = 0xf6923fd4, tf_ebx = 0x2,
983 tf_edx = 0x804b614, tf_ecx = 0x8085d10, tf_eax = 0x36, tf_trapno = 0x7,
984 tf_err = 0x2, tf_eip = 0x8060a34, tf_cs = 0x1f, tf_eflags = 0x286,
985 tf_esp = 0xefbfcf78, tf_ss = 0x27}) at ../../i386/i386/trap.c:1100
986 #8 0xf020a1fc in Xint0x80_syscall ()
987 #9 0x804832d in ?? ()
988 #10 0x80482ad in ?? ()
989 #11 0x80480e9 in ?? ()
990 .if t .vs
991 .if t .ps
992 .Ed
993 .Pp
994 When entering from the debugger, it is important that the source of frame 1
995 (listed by the
996 .Pa .gdbinit
997 file at the top of the example) contains the text
998 .Dq Li "Debugger (\*[q]vinum debug\*[q]);" .
999 .Pp
1000 This is an indication that the address specifications are correct.
1001 If you get
1002 some other output, your symbols and the kernel module are out of sync, and the
1003 trace will be meaningless.
1004 .El
1005 .Pp
1006 For an initial investigation, the most important information is the output of
1007 the
1008 .Ic bt
1009 (backtrace) command above.
1010 .Ss Reporting Problems with Vinum
1011 If you find any bugs in
1012 .Nm ,
1013 please report them to
1014 .An Greg Lehey Aq Mt grog@lemis.com .
1015 Supply the following
1016 information:
1017 .Bl -bullet
1018 .It
1019 The output of the
1020 .Nm Cm list
1021 command
1022 (see
1023 .Xr vinum 8 ) .
1024 .It
1025 Any messages printed in
1026 .Pa /var/log/messages .
1027 All such messages will be identified by the text
1028 .Dq Li vinum
1029 at the beginning.
1030 .It
1031 If you have a panic, a stack trace as described above.
1032 .El
1033 .Sh SEE ALSO
1034 .Xr disklabel 5 ,
1035 .Xr disklabel 8 ,
1036 .Xr newfs 8 ,
1037 .Xr vinum 8
1038 .Sh HISTORY
1039 .Nm
1040 first appeared in
1041 .Fx 3.0 .
1042 The RAID-5 component of
1043 .Nm
1044 was developed by Cybernet Inc.\&
1045 .Pq Pa http://www.cybernet.com/ ,
1046 for its NetMAX product.
1047 .Sh AUTHORS
1048 .An Greg Lehey Aq Mt grog@lemis.com .
1049 .Sh BUGS
1050 .Nm
1051 is a new product.
1052 Bugs can be expected.
1053 The configuration mechanism is not yet
1054 fully functional.
1055 If you have difficulties, please look at the section
1056 .Sx "DEBUGGING PROBLEMS WITH VINUM"
1057 before reporting problems.
1058 .Pp
1059 Kernels with the
1060 .Nm
1061 pseudo-device appear to work, but are not supported.
1062 If you have trouble with
1063 this configuration, please first replace the kernel with a
1064 .No non- Ns Nm
1065 kernel and test with the kld module.
1066 .Pp
1067 Detection of differences between the version of the kernel and the kld is not
1068 yet implemented.
1069 .Pp
1070 The RAID-5 functionality is new in
1071 .Fx 3.3 .
1072 Some problems have been
1073 reported with
1074 .Nm
1075 in combination with soft updates, but these are not reproducible on all
1076 systems.
1077 If you are planning to use
1078 .Nm
1079 in a production environment, please test carefully.