1 .\" Copyright (c) 2003 Greg Lehey
2 .\" All rights reserved.
4 .\" Redistribution and use in source and binary forms, with or without
5 .\" modification, are permitted provided that the following conditions
7 .\" 1. Redistributions of source code must retain the above copyright
8 .\" notice, this list of conditions and the following disclaimer.
9 .\" 2. Redistributions in binary form must reproduce the above copyright
10 .\" notice, this list of conditions and the following disclaimer in the
11 .\" documentation and/or other materials provided with the distribution.
13 .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
14 .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
15 .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
16 .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
17 .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
18 .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
19 .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
20 .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
21 .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
22 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
25 .\" $FreeBSD: src/share/man/man4/gdb.4,v 1.9 2004/01/28 00:02:10 grog Exp $
26 .\" $DragonFly: src/share/man/man4/gdb.4,v 1.4 2006/10/14 23:06:42 swildner Exp $
33 .Nd external kernel debugger
35 .Cd "makeoptions DEBUG=-g"
37 .Cd "options GDB_REMOTE_CHAT"
41 kernel debugger is a variation of
43 which understands some aspects of the
46 It can be used in a number of ways:
49 It can be used to examine the memory of the processor on which it runs.
51 It can be used to analyse a processor dump after a panic.
53 It can be used to debug another system interactively via a serial or firewire
55 In this mode, the processor can be stopped and single stepped.
57 With a firewire link, it can be used to examine the memory of a remote system
58 without the participation of that system.
59 In this mode, the processor cannot be stopped and single stepped, but it can be
60 of use when the remote system has crashed and is no longer responding.
63 When used for remote debugging,
65 requires the presence of the
68 Commands exist to switch between
72 .Sh PREPARING FOR DEBUGGING
73 When debugging kernels, it is practically essential to have built a kernel with
75 .Pq Cd "makeoptions DEBUG=-g" .
76 It is easiest to perform operations from the kernel build directory, by default
77 .Pa /usr/obj/usr/src/sys/GENERIC .
79 First, ensure you have a copy of the debug macros in the directory:
83 This command performs some transformations on the macros installed in
84 .Pa /usr/src/tools/debugscripts
85 to adapt them to the local environment.
86 .Ss "Inspecting the environment of the local machine"
87 To look at and change the contents of the memory of the system you are running
90 .Dl "gdb -k -wcore kernel.debug /dev/mem"
92 In this mode, you need the
99 is a kernel data file.
100 You can look at live data, and if you include the
102 option, you can change it at your peril.
103 The system does not stop (obviously), so a number of things will not work.
104 You can set breakpoints, but you cannot
106 execution, so they will not work.
107 .Ss "Debugging a crash dump"
108 By default, crash dumps are stored in the directory
110 Investigate them from the kernel build directory with:
112 .Dl "gdb -k kernel.debug /var/crash/vmcore.29"
114 In this mode, the system is obviously stopped, so you can only look at it.
115 .Ss "Debugging a live system with a remote link"
116 In the following discussion, the term
118 refers to the system running the debugger, and
120 refers to the live system being debugged.
122 To debug a live system with a remote link, the kernel must be compiled with the
126 .Cd "options BREAK_TO_DEBUGGER"
127 enables the debugging machine stop the debugged machine once a connection has
128 been established by pressing
130 .Ss "Debugging a live system with a remote serial link"
131 When using a serial port for the remote link on the i386 platform, the serial
132 port must be identified by setting the flag bit
134 for the specified interface.
135 Generally, this port will also be used as a serial console (flag bit
137 so the flags field for the interface in the kernel configuration file
142 To share a console and debug connection on a serial line, use the
143 .Cd "options GDB_REMOTE_CHAT"
145 .Ss "Debugging a live system with a remote firewire link"
146 As with serial debugging, to debug a live system with a firewire link, the
147 kernel must be compiled with the option
150 .Cd "options GDB_REMOTE_CHAT"
151 is not necessary, since the firewire implementation uses separate ports for the
152 console and debug connection.
154 A number of steps must be performed to set up a firewire link:
157 Ensure that both systems have
159 support, and that the kernel of the remote system includes the
164 If they are not compiled into the kernel, load the KLDs:
166 .Dl "kldload firewire"
168 On the remote system only:
169 .Bd -literal -offset indent
174 You should see something like this in the
176 output of the remote system:
177 .Bd -literal -offset indent
179 fwohci0: node_id=0x8800ffc0, gen=2, non CYCLEMASTER mode
180 firewire0: 2 nodes, maxhop <= 1, cable IRM = 1
181 firewire0: bus manager 1
182 firewire0: New S400 device ID:00c04f3226e88061
183 dcons_crom0: <dcons configuration ROM> on firewire0
184 dcons_crom0: bus_addr 0x22a000
187 It is a good idea to load these modules at boot time with the following entry in
188 .Pa /boot/loader.conf :
190 .Dl dcons_crom_enable="YES"
192 This ensures that all three modules are loaded.
193 There is no harm in loading
197 on the local system, but if you only want to load the
199 module, include the following in
200 .Pa /boot/loader.conf :
202 .Dl firewire_enable="YES"
206 to find the firewire node corresponding to the remote machine.
207 On the local machine you might see:
208 .Bd -literal -offset indent
210 2 devices (info_len=2)
212 1 0x00c04f3226e88061 0
213 0 0x000199000003622b 1
216 The first node is always the local system, so in this case, node 0 is the remote
218 If there are more than two systems, check from the other end to find which node
219 corresponds to the remote system.
220 On the remote machine, it looks like this:
221 .Bd -literal -offset indent
223 2 devices (info_len=2)
225 0 0x000199000003622b 0
226 1 0x00c04f3226e88061 1
229 Next, establish a firewire connection with
232 .Dl "dconschat -br -G 5556 -t 0x000199000003622b"
234 .Li 0x000199000003622b
235 is the EUI64 address of the remote node, as determined from the output of
238 When started in this manner,
240 establishes a local tunnel connection from port
242 to the remote debugger.
243 You can also establish a console port connection with the
245 option to the same invocation
249 manpage for further details.
254 does not return control to the user.
255 It displays error messages and console output for the remote system, so it is a
256 good idea to start it in its own window.
258 Finally, establish connection:
259 .Bd -literal -offset indent
261 GNU gdb 5.2.1 (FreeBSD)
262 .Em "(political statements omitted)"
263 Ready to go. Enter 'tr' to connect to the remote target
264 with /dev/cuaa0, 'tr /dev/cuaa1' to connect to a different port
265 or 'trf portno' to connect to the remote target with the firewire
266 interface. portno defaults to 5556.
268 Type 'getsyms' after connection to load kld symbols.
270 If you're debugging a local system, you can use 'kldsyms' instead
271 to load the kld symbols. That's a less obnoxious interface.
278 macro assumes a connection on port 5556.
279 If you want to use a different port (by changing the invocation of
284 For example, if you want to use port 4711, run
288 .Dl "dconschat -br -G 4711 -t 0x000199000003622b"
290 Then establish connection with:
291 .Bd -literal -offset indent
292 (gdb) tr localhost:4711
296 .Ss "Non-cooperative debugging a live system with a remote firewire link"
297 In addition to the conventional debugging via firewire described in the previous
298 section, it is possible to debug a remote system without its cooperation, once
299 an initial connection has been established.
300 This corresponds to debugging a local machine using
302 It can be very useful if a system crashes and the debugger no longer responds.
303 To use this method, set the
306 .Va hw.firewire.fwmem.eui64_hi
308 .Va hw.firewire.fwmem.eui64_lo
309 to the upper and lower halves of the EUI64 ID of the remote system,
311 From the previous example, the remote machine shows:
312 .Bd -literal -offset indent
314 2 devices (info_len=2)
316 0 0x000199000003622b 0
317 1 0x00c04f3226e88061 1
321 .Bd -literal -offset indent
322 # sysctl -w hw.firewire.fwmem.eui64_hi=0x00019900
323 hw.firewire.fwmem.eui64_hi: 0 -> 104704
324 # sysctl -w hw.firewire.fwmem.eui64_lo=0x0003622b
325 hw.firewire.fwmem.eui64_lo: 0 -> 221739
328 Note that the variables must be explicitly stated in hexadecimal.
329 After this, you can examine the remote machine's state with the following input:
330 .Bd -literal -offset indent
331 # gdb -k kernel.debug /dev/fwmem0.0
332 GNU gdb 5.2.1 (FreeBSD)
333 .Em "(messages omitted)"
334 Reading symbols from /modules/dcons.ko...done.
335 Loaded symbols for /modules/dcons.ko
336 Reading symbols from /modules/dcons_crom.ko...done.
337 Loaded symbols for /modules/dcons_crom.ko
338 #0 sched_switch (td=0xc0922fe0) at /usr/src/sys/kern/sched_4bsd.c:621
342 In this case, it is not necessary to load the symbols explicitly.
343 The remote system continues to run.
345 The user interface to
352 This section discusses only the extensions for kernel debugging that get
353 installed in the kernel build directory.
354 .Ss "Debugging environment"
355 The following macros manipulate the debugging environment:
356 .Bl -tag -width indent
360 This command is only meaningful when performing remote debugging.
364 information for the target machine and invite user to paste it back in.
365 This is required because
367 does not allow data to be passed to shell scripts.
368 It is necessary for remote debugging and crash dumps; for local memory debugging
373 Read in the symbol tables for the debugging machine.
374 This does not work for
375 remote debugging and crash dumps; use
378 .It Ic tr Ar interface
379 Debug a remote system via the specified serial or firewire interface.
381 Debug a remote system via serial interface
384 Debug a remote system via serial interface
387 Debug a remote system via firewire interface at default port 5556.
394 are convenience commands which invoke
396 .Ss "The current process environment"
397 The following macros are convenience functions intended to make things easier
401 .Bl -tag -width indent
403 Select stack frame 0 and show assembler-level details.
405 Select stack frame 1 and show assembler-level details.
407 Select stack frame 2 and show assembler-level details.
409 Select stack frame 3 and show assembler-level details.
411 Select stack frame 4 and show assembler-level details.
413 Select stack frame 5 and show assembler-level details.
415 Show 12 words in hex, starting at current
419 List the next 10 instructions from the current
423 Show the register contents and the first four parameters of the current stack
426 Show the first parameter of current stack frame in various formats.
428 Show the second parameter of current stack frame in various formats.
430 Show the third parameter of current stack frame in various formats.
432 Show the fourth parameter of current stack frame in various formats.
434 Show the fifth parameter of current stack frame in various formats.
436 Show the last 12 words on stack in hexadecimal.
438 Show the register contents and the first ten parameters.
440 Single step 1 instruction (over calls) and show next instruction.
442 Single step 1 instruction (through calls) and show next instruction.
444 .Ss "Examining other processes"
445 The following macros access other processes.
449 does not understand the concept of multiple processes, so they effectively
453 .Bl -tag -width indent
455 Show a backtrace for the process
458 Show backtraces for all processes in the system.
460 Show a backtrace for the process previously selected with
463 Show a backtrace from the
466 .It Ic defproc Ar pid
467 Specify the PID of the process for some other commands in this section.
471 of the stack of the process previously selected with
474 Show some PCB contents of the process
477 .Ss "Examining data structures"
480 commands to look at most data structures.
481 The macros in this section are
482 convenience functions which typically display the data in a more readable
483 format, or which omit less interesting parts of the structure.
484 .Bl -tag -width indent
486 Show information about the buffer header pointed to by the variable
488 in the current frame.
494 in the current frame.
496 Show detailed information about the buffer header
498 pointed at by the local variable
501 Show summary information about the buffer header
503 pointed at by the parameter
506 Print a number of fields from the buffer header pointed at in by the pointer
508 in the current environment.
510 Show some information of the
512 pointed to by the local variable
515 .Ss "Miscellaneous macros"
516 .Bl -tag -width indent
518 Check unallocated memory for modifications.
519 This assumes that the kernel has been compiled with
520 .Cd "options DIAGNOSTIC"
521 This causes the contents of free memory to be set to
524 Print the system message buffer.
525 This corresponds to the
528 This macro used to be called
530 It can take a very long time over a serial line,
531 and it is even slower via firewire
532 or local memory due to inefficiencies in
534 When debugging a crash dump or over firewire, it is not necessary to start
536 to access the message buffer: instead, use an appropriate variation of
537 .Bd -literal -offset indent
538 dmesg -M /var/crash/vmcore.0 -N kernel.debug
539 dmesg -M /dev/fwmem0.0 -N kernel.debug
544 utility without options.
546 Print the command name of the current process.
549 This corresponds in concept, but not in appearance, to the
552 When debugging a crash dump or over firewire, it is not necessary to start
556 output: instead, use an appropriate variation of
557 .Bd -literal -offset indent
558 ps -M /var/crash/vmcore.0 -N kernel.debug
559 ps -M /dev/fwmem0.0 -N kernel.debug
562 Kludge for writing macros.
563 When writing macros, it is convenient to paste them
567 Unfortunately, if the macro is already defined,
573 It will not give up until you answer
575 This command is that answer.
576 It does nothing else except to print a warning
577 message to remind you to remove it again.
590 This man page was written by
591 .An "Greg Lehey" Aq grog@FreeBSD.org .
596 was never designed to debug kernels, and it is not a very good match.
601 implementation is very inefficient, and many operations are slow.
603 Serial debugging is even slower, and race conditions can make it difficult to
604 run the link at more than 9600 bps.
605 Firewire connections do not have this problem.
609 In general, the person who wrote them did so while looking for a specific
610 problem, so they may not be general enough, and they may behave badly when used
611 in ways for which they were not intended, even if those ways make sense.
613 Many of these commands only work on the ia32 architecture.