1 .\" Copyright (c) 2003 Greg Lehey
2 .\" All rights reserved.
4 .\" Redistribution and use in source and binary forms, with or without
5 .\" modification, are permitted provided that the following conditions
7 .\" 1. Redistributions of source code must retain the above copyright
8 .\" notice, this list of conditions and the following disclaimer.
9 .\" 2. Redistributions in binary form must reproduce the above copyright
10 .\" notice, this list of conditions and the following disclaimer in the
11 .\" documentation and/or other materials provided with the distribution.
13 .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
14 .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
15 .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
16 .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
17 .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
18 .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
19 .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
20 .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
21 .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
22 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
25 .\" $FreeBSD: src/share/man/man4/gdb.4,v 1.9 2004/01/28 00:02:10 grog Exp $
32 .Nd external kernel debugger
34 .Cd "makeoptions DEBUG=-g"
36 .Cd "options GDB_REMOTE_CHAT"
40 kernel debugger is a variation of
42 which understands some aspects of the
45 It can be used in a number of ways:
48 It can be used to examine the memory of the processor on which it runs.
50 It can be used to analyse a processor dump after a panic.
52 It can be used to debug another system interactively via a serial or firewire
54 In this mode, the processor can be stopped and single stepped.
56 With a firewire link, it can be used to examine the memory of a remote system
57 without the participation of that system.
58 In this mode, the processor cannot be stopped and single stepped, but it can be
59 of use when the remote system has crashed and is no longer responding.
62 When used for remote debugging,
64 requires the presence of the
67 Commands exist to switch between
71 .Sh PREPARING FOR DEBUGGING
72 When debugging kernels, it is practically essential to have built a kernel with
74 .Pq Cd "makeoptions DEBUG=-g" .
75 It is easiest to perform operations from the kernel build directory, by default
76 .Pa /usr/obj/usr/src/sys/X86_64_GENERIC .
78 First, ensure you have a copy of the debug macros in the directory:
82 This command performs some transformations on the macros installed in
83 .Pa /usr/src/tools/debugscripts
84 to adapt them to the local environment.
85 .Ss "Inspecting the environment of the local machine"
86 To look at and change the contents of the memory of the system you are running
89 .Dl "gdb -k -wcore kernel.debug /dev/mem"
91 In this mode, you need the
98 is a kernel data file.
99 You can look at live data, and if you include the
101 option, you can change it at your peril.
102 The system does not stop (obviously), so a number of things will not work.
103 You can set breakpoints, but you cannot
105 execution, so they will not work.
106 .Ss "Debugging a crash dump"
107 By default, crash dumps are stored in the directory
109 Investigate them from the kernel build directory with:
111 .Dl "gdb -k kernel.debug /var/crash/vmcore.29"
113 In this mode, the system is obviously stopped, so you can only look at it.
114 .Ss "Debugging a live system with a remote link"
115 In the following discussion, the term
117 refers to the system running the debugger, and
119 refers to the live system being debugged.
121 To debug a live system with a remote link, the kernel must be compiled with the
125 .Cd "options BREAK_TO_DEBUGGER"
126 enables the debugging machine stop the debugged machine once a connection has
127 been established by pressing
129 .Ss "Debugging a live system with a remote serial link"
130 When using a serial port for the remote link on the i386 platform, the serial
131 port must be identified by setting the flag bit
133 for the specified interface.
134 Generally, this port will also be used as a serial console (flag bit
136 so the flags field for the interface in the kernel configuration file
141 To share a console and debug connection on a serial line, use the
142 .Cd "options GDB_REMOTE_CHAT"
144 .Ss "Debugging a live system with a remote firewire link"
145 As with serial debugging, to debug a live system with a firewire link, the
146 kernel must be compiled with the option
149 .Cd "options GDB_REMOTE_CHAT"
150 is not necessary, since the firewire implementation uses separate ports for the
151 console and debug connection.
153 A number of steps must be performed to set up a firewire link:
156 Ensure that both systems have
158 support, and that the kernel of the remote system includes the
163 If they are not compiled into the kernel, load the KLDs:
165 .Dl "kldload firewire"
167 On the remote system only:
168 .Bd -literal -offset indent
173 You should see something like this in the
175 output of the remote system:
176 .Bd -literal -offset indent
178 fwohci0: node_id=0x8800ffc0, gen=2, non CYCLEMASTER mode
179 firewire0: 2 nodes, maxhop <= 1, cable IRM = 1
180 firewire0: bus manager 1
181 firewire0: New S400 device ID:00c04f3226e88061
182 dcons_crom0: <dcons configuration ROM> on firewire0
183 dcons_crom0: bus_addr 0x22a000
186 It is a good idea to load these modules at boot time with the following entry in
187 .Pa /boot/loader.conf :
189 .Dl dcons_crom_enable="YES"
191 This ensures that all three modules are loaded.
192 There is no harm in loading
196 on the local system, but if you only want to load the
198 module, include the following in
199 .Pa /boot/loader.conf :
201 .Dl firewire_enable="YES"
205 to find the firewire node corresponding to the remote machine.
206 On the local machine you might see:
207 .Bd -literal -offset indent
209 2 devices (info_len=2)
211 1 0x00c04f3226e88061 0
212 0 0x000199000003622b 1
215 The first node is always the local system, so in this case, node 0 is the remote
217 If there are more than two systems, check from the other end to find which node
218 corresponds to the remote system.
219 On the remote machine, it looks like this:
220 .Bd -literal -offset indent
222 2 devices (info_len=2)
224 0 0x000199000003622b 0
225 1 0x00c04f3226e88061 1
228 Next, establish a firewire connection with
231 .Dl "dconschat -br -G 5556 -t 0x000199000003622b"
233 .Li 0x000199000003622b
234 is the EUI64 address of the remote node, as determined from the output of
237 When started in this manner,
239 establishes a local tunnel connection from port
241 to the remote debugger.
242 You can also establish a console port connection with the
244 option to the same invocation
248 manpage for further details.
253 does not return control to the user.
254 It displays error messages and console output for the remote system, so it is a
255 good idea to start it in its own window.
257 Finally, establish connection:
258 .Bd -literal -offset indent
260 GNU gdb 5.2.1 (FreeBSD)
261 .Em "(political statements omitted)"
262 Ready to go. Enter 'tr' to connect to the remote target
263 with /dev/cuaa0, 'tr /dev/cuaa1' to connect to a different port
264 or 'trf portno' to connect to the remote target with the firewire
265 interface. portno defaults to 5556.
267 Type 'getsyms' after connection to load kld symbols.
269 If you're debugging a local system, you can use 'kldsyms' instead
270 to load the kld symbols. That's a less obnoxious interface.
277 macro assumes a connection on port 5556.
278 If you want to use a different port (by changing the invocation of
283 For example, if you want to use port 4711, run
287 .Dl "dconschat -br -G 4711 -t 0x000199000003622b"
289 Then establish connection with:
290 .Bd -literal -offset indent
291 (gdb) tr localhost:4711
295 .Ss "Non-cooperative debugging a live system with a remote firewire link"
296 In addition to the conventional debugging via firewire described in the previous
297 section, it is possible to debug a remote system without its cooperation, once
298 an initial connection has been established.
299 This corresponds to debugging a local machine using
301 It can be very useful if a system crashes and the debugger no longer responds.
302 To use this method, set the
305 .Va hw.firewire.fwmem.eui64_hi
307 .Va hw.firewire.fwmem.eui64_lo
308 to the upper and lower halves of the EUI64 ID of the remote system,
310 From the previous example, the remote machine shows:
311 .Bd -literal -offset indent
313 2 devices (info_len=2)
315 0 0x000199000003622b 0
316 1 0x00c04f3226e88061 1
320 .Bd -literal -offset indent
321 # sysctl -w hw.firewire.fwmem.eui64_hi=0x00019900
322 hw.firewire.fwmem.eui64_hi: 0 -> 104704
323 # sysctl -w hw.firewire.fwmem.eui64_lo=0x0003622b
324 hw.firewire.fwmem.eui64_lo: 0 -> 221739
327 Note that the variables must be explicitly stated in hexadecimal.
328 After this, you can examine the remote machine's state with the following input:
329 .Bd -literal -offset indent
330 # gdb -k kernel.debug /dev/fwmem0.0
331 GNU gdb 5.2.1 (FreeBSD)
332 .Em "(messages omitted)"
333 Reading symbols from /modules/dcons.ko...done.
334 Loaded symbols for /modules/dcons.ko
335 Reading symbols from /modules/dcons_crom.ko...done.
336 Loaded symbols for /modules/dcons_crom.ko
337 #0 sched_switch (td=0xc0922fe0) at /usr/src/sys/kern/sched_4bsd.c:621
341 In this case, it is not necessary to load the symbols explicitly.
342 The remote system continues to run.
344 The user interface to
351 This section discusses only the extensions for kernel debugging that get
352 installed in the kernel build directory.
353 .Ss "Debugging environment"
354 The following macros manipulate the debugging environment:
355 .Bl -tag -width indent
359 This command is only meaningful when performing remote debugging.
363 information for the target machine and invite user to paste it back in.
364 This is required because
366 does not allow data to be passed to shell scripts.
367 It is necessary for remote debugging and crash dumps; for local memory debugging
372 Read in the symbol tables for the debugging machine.
373 This does not work for
374 remote debugging and crash dumps; use
377 .It Ic tr Ar interface
378 Debug a remote system via the specified serial or firewire interface.
380 Debug a remote system via serial interface
383 Debug a remote system via serial interface
386 Debug a remote system via firewire interface at default port 5556.
393 are convenience commands which invoke
395 .Ss "The current process environment"
396 The following macros are convenience functions intended to make things easier
400 .Bl -tag -width indent
402 Select stack frame 0 and show assembler-level details.
404 Select stack frame 1 and show assembler-level details.
406 Select stack frame 2 and show assembler-level details.
408 Select stack frame 3 and show assembler-level details.
410 Select stack frame 4 and show assembler-level details.
412 Select stack frame 5 and show assembler-level details.
414 Show 12 words in hex, starting at current
418 List the next 10 instructions from the current
422 Show the register contents and the first four parameters of the current stack
425 Show the first parameter of current stack frame in various formats.
427 Show the second parameter of current stack frame in various formats.
429 Show the third parameter of current stack frame in various formats.
431 Show the fourth parameter of current stack frame in various formats.
433 Show the fifth parameter of current stack frame in various formats.
435 Show the last 12 words on stack in hexadecimal.
437 Show the register contents and the first ten parameters.
439 Single step 1 instruction (over calls) and show next instruction.
441 Single step 1 instruction (through calls) and show next instruction.
443 .Ss "Examining other processes"
444 The following macros access other processes.
448 does not understand the concept of multiple processes, so they effectively
452 .Bl -tag -width indent
454 Show a backtrace for the process
457 Show backtraces for all processes in the system.
459 Show a backtrace for the process previously selected with
462 Show a backtrace from the
465 .It Ic defproc Ar pid
466 Specify the PID of the process for some other commands in this section.
470 of the stack of the process previously selected with
473 Show some PCB contents of the process
476 .Ss "Examining data structures"
479 commands to look at most data structures.
480 The macros in this section are
481 convenience functions which typically display the data in a more readable
482 format, or which omit less interesting parts of the structure.
483 .Bl -tag -width indent
485 Show information about the buffer header pointed to by the variable
487 in the current frame.
493 in the current frame.
495 Show detailed information about the buffer header
497 pointed at by the local variable
500 Show summary information about the buffer header
502 pointed at by the parameter
505 Print a number of fields from the buffer header pointed at in by the pointer
507 in the current environment.
509 Show some information of the
511 pointed to by the local variable
514 .Ss "Miscellaneous macros"
515 .Bl -tag -width indent
517 Check unallocated memory for modifications.
518 This assumes that the kernel has been compiled with
519 .Cd "options DIAGNOSTIC"
520 This causes the contents of free memory to be set to
523 Print the system message buffer.
524 This corresponds to the
527 This macro used to be called
529 It can take a very long time over a serial line,
530 and it is even slower via firewire
531 or local memory due to inefficiencies in
533 When debugging a crash dump or over firewire, it is not necessary to start
535 to access the message buffer: instead, use an appropriate variation of
536 .Bd -literal -offset indent
537 dmesg -M /var/crash/vmcore.0 -N kernel.debug
538 dmesg -M /dev/fwmem0.0 -N kernel.debug
543 utility without options.
545 Print the command name of the current process.
548 This corresponds in concept, but not in appearance, to the
551 When debugging a crash dump or over firewire, it is not necessary to start
555 output: instead, use an appropriate variation of
556 .Bd -literal -offset indent
557 ps -M /var/crash/vmcore.0 -N kernel.debug
558 ps -M /dev/fwmem0.0 -N kernel.debug
561 Kludge for writing macros.
562 When writing macros, it is convenient to paste them
566 Unfortunately, if the macro is already defined,
572 It will not give up until you answer
574 This command is that answer.
575 It does nothing else except to print a warning
576 message to remind you to remove it again.
589 This man page was written by
590 .An Greg Lehey Aq Mt grog@FreeBSD.org .
595 was never designed to debug kernels, and it is not a very good match.
600 implementation is very inefficient, and many operations are slow.
602 Serial debugging is even slower, and race conditions can make it difficult to
603 run the link at more than 9600 bps.
604 Firewire connections do not have this problem.
608 In general, the person who wrote them did so while looking for a specific
609 problem, so they may not be general enough, and they may behave badly when used
610 in ways for which they were not intended, even if those ways make sense.
612 Many of these commands only work on the ia32 architecture.