README

   1 Linux kernel one-time stacktrace module - What is it?
   2 /////////////////////////////////////////////////////
   3
   4 The Linux one-time stacktrace (otst) module can be used to generate stacktraces
   5 during runtime for debugging purpose. A symbol can be written to the file
   6 /proc/driver/otst, e.g. ...
   7
   8 # echo -n "netif_rx" > /proc/driver/otst
   9
  10 ... thus on the netxt call of the function netif_rx, a stacktrace is being
  11 generated _once_, thus you are able to see the calling context of your function.
  12 To show currently traced functions, use:
  13
  14 # cat /proc/driver/otst
  15
  16 More than one function can be traced at a time.
  17
  18 Example output:
  19
  20 [   83.267411] otst: one-time stacktrace driver loaded!
  21 [   95.591140] otst: symbol netif_rx registered!
  22 [  121.770022] otst: triggered stacktrace for symbol netif_rx at 0xffffffff814c0050:
  23 [  121.770027] Pid: 0, comm: kworker/0:1 Not tainted 3.0.0-rc1+ #3
  24 [  121.770030] Call Trace:
  25 [  121.770033]  <#DB>  [<ffffffffa002838b>] otst_handler+0x2b/0x30 [otst]
  26 [  121.770047]  [<ffffffff815ba4a7>] aggr_pre_handler+0x57/0xb0
  27 [  121.770053]  [<ffffffff814c0050>] ? net_rx_action+0x2e0/0x2e0
  28 [  121.770057]  [<ffffffff815b960c>] kprobe_exceptions_notify+0x3fc/0x460
  29 [  121.770062]  [<ffffffff815ba166>] notifier_call_chain+0x56/0x80
  30 [  121.770067]  [<ffffffff815ba1ca>] atomic_notifier_call_chain+0x1a/0x20
  31 [  121.770071]  [<ffffffff815ba1fe>] notify_die+0x2e/0x30
  32 [  121.770075]  [<ffffffff815b7263>] do_int3+0x63/0xd0
  33 [  121.770079]  [<ffffffff815b6a88>] int3+0x28/0x40
  34 [  121.770083]  [<ffffffff814c0051>] ? netif_rx+0x1/0x190
  35 [  121.770086]  <<EOE>>  <IRQ>  [<ffffffff814c0352>] ? netif_rx_ni+0x12/0x30
  36 [  121.770094]  [<ffffffff814f69b9>] ip_dev_loopback_xmit+0x79/0xa0
  37 [  121.770098]  [<ffffffff814f7930>] ip_mc_output+0x250/0x260
  38 [  121.770102]  [<ffffffff814b1353>] ? __alloc_skb+0x83/0x170
  39 [  121.770106]  [<ffffffff814f6a09>] ip_local_out+0x29/0x30
  40 [  121.770111]  [<ffffffff81528aeb>] igmp_send_report+0x1db/0x210
  41 [  121.770117]  [<ffffffff81087d78>] ? sched_clock_cpu+0xb8/0x110
  42 [  121.770121]  [<ffffffff81529380>] igmp_timer_expire+0x100/0x130
  43 [  121.770125]  [<ffffffff8104e2e2>] ? scheduler_tick+0x132/0x2b0
  44 [  121.770130]  [<ffffffff8106e8aa>] run_timer_softirq+0x16a/0x390
  45 [  121.770134]  [<ffffffff81529280>] ? ip_mc_destroy_dev+0x80/0x80
  46 [  121.770139]  [<ffffffff8102830d>] ? lapic_next_event+0x1d/0x30
  47 [  121.770144]  [<ffffffff8106577f>] __do_softirq+0xbf/0x200
  48 [  121.770148]  [<ffffffff81085967>] ? hrtimer_interrupt+0x127/0x210
  49 [  121.770153]  [<ffffffff815bf51c>] call_softirq+0x1c/0x30
  50 [  121.770157]  [<ffffffff8100d2e5>] do_softirq+0x65/0xa0
  51 [  121.770161]  [<ffffffff81065595>] irq_exit+0xb5/0xc0
  52 [  121.770165]  [<ffffffff815bfe5e>] smp_apic_timer_interrupt+0x6e/0x99
  53 [  121.770170]  [<ffffffff815becd3>] apic_timer_interrupt+0x13/0x20
  54 [  121.770172]  <EOI>  [<ffffffff81013e1d>] ? mwait_idle+0xad/0x1c0
  55 [  121.770180]  [<ffffffff815ba1ca>] ? atomic_notifier_call_chain+0x1a/0x20
  56 [  121.770185]  [<ffffffff8100b0b7>] cpu_idle+0xb7/0x110
  57 [  121.770190]  [<ffffffff815ae4a1>] start_secondary+0x1c0/0x1c7
  58 [  174.500038] otst: symbol netif_rx unregistered!
  59
  60 For questions, bugs, patches write a mail to:
  61
  62   Daniel Borkmann <borkmann@iogearbox.net>
  63