| Age | Commit message (Collapse) | Author |
|---|
|
When we cat <debugfs>/tracing/stack_trace, we may cause circular lock:
sys_read()
t_start()
arch_spin_lock(&max_stack_lock);
t_show()
seq_printf(), vsnprintf() .... /* they are all trace-able,
when they are traced, max_stack_lock may be required again. */
The following script can trigger this circular dead lock very easy:
#!/bin/bash
echo 1 > /proc/sys/kernel/stack_tracer_enabled
mount -t debugfs xxx /mnt > /dev/null 2>&1
(
# make check_stack() zealous to require max_stack_lock
for ((; ;))
{
echo 1 > /mnt/tracing/stack_max_size
}
) &
for ((; ;))
{
cat /mnt/tracing/stack_trace > /dev/null
}
To fix this bug, we increase the percpu trace_active before
require the lock.
Reported-by: Li Zefan <lizf@cn.fujitsu.com>
Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
LKML-Reference: <4B67D4F9.9080905@cn.fujitsu.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Name space cleanup. No functional change.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Ingo Molnar <mingo@elte.hu>
Cc: linux-arch@vger.kernel.org
|
|
Further name space cleanup. No functional change
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Ingo Molnar <mingo@elte.hu>
Cc: linux-arch@vger.kernel.org
|
|
The raw_spin* namespace was taken by lockdep for the architecture
specific implementations. raw_spin_* would be the ideal name space for
the spinlocks which are not converted to sleeping locks in preempt-rt.
Linus suggested to convert the raw_ to arch_ locks and cleanup the
name space instead of using an artifical name like core_spin,
atomic_spin or whatever
No functional change.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Ingo Molnar <mingo@elte.hu>
Cc: linux-arch@vger.kernel.org
|
|
It's unused.
It isn't needed -- read or write flag is already passed and sysctl
shouldn't care about the rest.
It _was_ used in two places at arch/frv for some reason.
Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
Cc: David Howells <dhowells@redhat.com>
Cc: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: James Morris <jmorris@namei.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
Extract duplicate code in t_start() and t_next().
Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <4A891A91.4030602@cn.fujitsu.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
|
Conflicts:
kernel/trace/trace_events_filter.c
We use the tracing/core version.
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
|
Every time we cat stack_trace, we leak memory allocated by seq_open().
Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
LKML-Reference: <4A67D8E8.3020500@cn.fujitsu.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
We can directly use %pF input format instead of sprint_symbol()
and %s input format.
Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
Reviewed-by: Li Zefan <lizf@cn.fujitsu.com>
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
|
|
This made my machine completely frozen:
# echo 1 > /proc/sys/kernel/stack_tracer_enabled
# echo 2 > /proc/sys/kernel/stack_tracer_enabled
The cause is register_ftrace_function() was called twice.
Also fix ftrace_enabled sysctl, though seems nothing bad happened
as I tested it.
Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <4A448D17.9010305@cn.fujitsu.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
|
The last entry in the stack_dump_trace is ULONG_MAX, which is not
a valid entry, but max_stack_trace.nr_entries has accounted for it.
So when printing the header, we should decrease it by one.
Before fix, print as following, for example:
Depth Size Location (53 entries) <--- should be 52
----- ---- --------
0) 3264 108 update_wall_time+0x4d5/0x9a0
...
51) 80 80 syscall_call+0x7/0xb
^^^
it's correct.
Signed-off-by: walimis <walimisdev@gmail.com>
LKML-Reference: <1244016090-7814-1-git-send-email-walimisdev@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Impact: cleanup
Most of the tracing files creation follow the same pattern:
ret = debugfs_create_file(...)
if (!ret)
pr_warning("Couldn't create ... entry\n")
Unify it!
Reported-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <1238109938-11840-1-git-send-email-fweisbec@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Ingo Molnar suggested, instead of:
Depth Size Location (27 entries)
----- ---- --------
0) 2880 48 lock_timer_base+0x2b/0x4f
1) 2832 80 __mod_timer+0x33/0xe0
2) 2752 16 __ide_set_handler+0x63/0x65
To have it be:
Depth Size Location (27 entries)
----- ---- --------
0) 2880 48 lock_timer_base+0x2b/0x4f
1) 2832 80 __mod_timer+0x33/0xe0
2) 2752 16 __ide_set_handler+0x63/0x65
Requested-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
|
|
If the stack tracing is disabled (by default) the stack_trace file
will only contain the header:
# cat /debug/tracing/stack_trace
Depth Size Location (0 entries)
----- ---- --------
This can be frustrating to a developer that does not realize that the
stack tracer is disabled. This patch adds the following text:
# cat /debug/tracing/stack_trace
Depth Size Location (0 entries)
----- ---- --------
#
# Stack tracer disabled
#
# To enable the stack tracer, either add 'stacktrace' to the
# kernel command line
# or 'echo 1 > /proc/sys/kernel/stack_tracer_enabled'
#
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
|
|
The stack tracer use to look like this:
# cat /debug/tracing/stack_trace
Depth Size Location (57 entries)
----- ---- --------
0) 5088 16 mempool_alloc_slab+0x16/0x18
1) 5072 144 mempool_alloc+0x4d/0xfe
2) 4928 16 scsi_sg_alloc+0x48/0x4a [scsi_mod]
Now it looks like this:
# cat /debug/tracing/stack_trace
Depth Size Location (57 entries)
----- ---- --------
0) 5088 16 mempool_alloc_slab+0x16/0x18
1) 5072 144 mempool_alloc+0x4d/0xfe
2) 4928 16 scsi_sg_alloc+0x48/0x4a [scsi_mod]
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
|
|
Impact: clean up
Andrew Morton suggested to use the stack_tracer_enabled variable
to decide whether or not to start stack tracing on bootup.
This lets us remove the start_stack_trace variable.
Reported-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
|
Impact: enhancement to stack tracer
The stack tracer currently is either on when configured in or
off when it is not. It can not be disabled when it is configured on.
(besides disabling the function tracer that it uses)
This patch adds a way to enable or disable the stack tracer at
run time. It defaults off on bootup, but a kernel parameter 'stacktrace'
has been added to enable it on bootup.
A new sysctl has been added "kernel.stack_tracer_enabled" to let
the user enable or disable the stack tracer at run time.
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
|
Impact: fix to output of stack trace
If a function is not found in the stack of the stack tracer, the
number printed is quite strange. This fixes the algorithm to handle
missing functions better.
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
|
Impact: fix for lockdep and ftrace
The raw_local_irq_save/restore confuses lockdep. This patch
converts them to the local_irq_save/restore variants.
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
|
into tracing/core
|
|
Impact: make output of stack_trace complete if buffer overruns
When read buffer overruns, the output of stack_trace isn't complete.
When printing records with seq_printf in t_show, if the read buffer
has overruned by the current record, then this record won't be
printed to user space through read buffer, it will just be dropped in
this printing.
When next printing, t_start should return the "*pos"th record, which
is the one dropped by previous printing, but it just returns
(m->private + *pos)th record.
Here we use a more sane method to implement seq_operations which can
be found in kernel code. Thus we needn't initialize m->private.
About testing, it's not easy to overrun read buffer, but we can use
seq_printf to print more padding bytes in t_show, then it's easy to
check whether or not records are lost.
This commit has been tested on both condition of overrun and non
overrun.
Signed-off-by: Liming Wang <liming.wang@windriver.com>
Acked-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
|
Impact: use new, consolidated APIs in ftrace plugins
This patch replaces the schedule safe preempt disable code with the
ftrace_preempt_disable() and ftrace_preempt_enable() safe functions.
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
|
The stack trace API does not record if the stack is not on the current
task's stack. That is, if the stack is the interrupt stack or NMI stack,
the output does not show. Also, the size of those stacks are not
consistent with the size of the thread stack, this makes the calculation
of the stack size usually bogus.
This all confuses the stack tracer. I unfortunately do not have time to
fix all these problems, but this patch does record the worst stack when
the stack pointer is on the tasks stack (instead of bogus numbers).
The patch simply returns if the stack pointer is not on the task's stack.
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
|
This patch adds indexes into the stack that the functions in the
stack dump were found at. As an added bonus, I also added a diff
to show which function is the most notorious consumer of the stack.
The output now looks like this:
# cat /debug/tracing/stack_trace
Depth Size Location (48 entries)
----- ---- --------
0) 2476 212 blk_recount_segments+0x39/0x59
1) 2264 12 bio_phys_segments+0x16/0x1d
2) 2252 20 blk_rq_bio_prep+0x23/0xaf
3) 2232 12 init_request_from_bio+0x74/0x77
4) 2220 56 __make_request+0x294/0x331
5) 2164 136 generic_make_request+0x34f/0x37d
6) 2028 56 submit_bio+0xe7/0xef
7) 1972 28 submit_bh+0xd1/0xf0
8) 1944 112 block_read_full_page+0x299/0x2a9
9) 1832 8 blkdev_readpage+0x14/0x16
10) 1824 28 read_cache_page_async+0x7e/0x109
11) 1796 16 read_cache_page+0x11/0x49
12) 1780 32 read_dev_sector+0x3c/0x72
13) 1748 48 read_lba+0x4d/0xaa
14) 1700 168 efi_partition+0x85/0x61b
15) 1532 72 rescan_partitions+0x10e/0x266
16) 1460 40 do_open+0x1c7/0x24e
17) 1420 292 __blkdev_get+0x79/0x84
18) 1128 12 blkdev_get+0x12/0x14
19) 1116 20 register_disk+0xd1/0x11e
20) 1096 28 add_disk+0x34/0x90
21) 1068 52 sd_probe+0x2b1/0x366
22) 1016 20 driver_probe_device+0xa5/0x120
23) 996 8 __device_attach+0xd/0xf
24) 988 32 bus_for_each_drv+0x3e/0x68
25) 956 24 device_attach+0x56/0x6c
26) 932 16 bus_attach_device+0x26/0x4d
27) 916 64 device_add+0x380/0x4b4
28) 852 28 scsi_sysfs_add_sdev+0xa1/0x1c9
29) 824 160 scsi_probe_and_add_lun+0x919/0xa2a
30) 664 36 __scsi_add_device+0x88/0xae
31) 628 44 ata_scsi_scan_host+0x9e/0x21c
32) 584 28 ata_host_register+0x1cb/0x1db
33) 556 24 ata_host_activate+0x98/0xb5
34) 532 192 ahci_init_one+0x9bd/0x9e9
35) 340 20 pci_device_probe+0x3e/0x5e
36) 320 20 driver_probe_device+0xa5/0x120
37) 300 20 __driver_attach+0x3f/0x5e
38) 280 36 bus_for_each_dev+0x40/0x62
39) 244 12 driver_attach+0x19/0x1b
40) 232 28 bus_add_driver+0x9c/0x1af
41) 204 28 driver_register+0x76/0xd2
42) 176 20 __pci_register_driver+0x44/0x71
43) 156 8 ahci_init+0x14/0x16
44) 148 100 _stext+0x42/0x122
45) 48 20 kernel_init+0x175/0x1dc
46) 28 28 kernel_thread_helper+0x7/0x10
The first column is simply an index starting from the inner most function
and counting down to the outer most.
The next column is the location that the function was found on the stack.
The next column is the size of the stack for that function.
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
|
This is another tracer using the ftrace infrastructure, that examines
at each function call the size of the stack. If the stack use is greater
than the previous max it is recorded.
You can always see (and set) the max stack size seen. By setting it
to zero will start the recording again. The backtrace is also available.
For example:
# cat /debug/tracing/stack_max_size
1856
# cat /debug/tracing/stack_trace
[<c027764d>] stack_trace_call+0x8f/0x101
[<c021b966>] ftrace_call+0x5/0x8
[<c02553cc>] clocksource_get_next+0x12/0x48
[<c02542a5>] update_wall_time+0x538/0x6d1
[<c0245913>] do_timer+0x23/0xb0
[<c0257657>] tick_do_update_jiffies64+0xd9/0xf1
[<c02576b9>] tick_sched_timer+0x4a/0xad
[<c0250fe6>] __run_hrtimer+0x3e/0x75
[<c02518ed>] hrtimer_interrupt+0xf1/0x154
[<c022c870>] smp_apic_timer_interrupt+0x71/0x84
[<c021b7e9>] apic_timer_interrupt+0x2d/0x34
[<c0238597>] finish_task_switch+0x29/0xa0
[<c05abd13>] schedule+0x765/0x7be
[<c05abfca>] schedule_timeout+0x1b/0x90
[<c05ab4d4>] wait_for_common+0xab/0x101
[<c05ab5ac>] wait_for_completion+0x12/0x14
[<c033cfc3>] blk_execute_rq+0x84/0x99
[<c0402470>] scsi_execute+0xc2/0x105
[<c040250a>] scsi_execute_req+0x57/0x7f
[<c043afe0>] sr_test_unit_ready+0x3e/0x97
[<c043bbd6>] sr_media_change+0x43/0x205
[<c046b59f>] media_changed+0x48/0x77
[<c046b5ff>] cdrom_media_changed+0x31/0x37
[<c043b091>] sr_block_media_changed+0x16/0x18
[<c02b9e69>] check_disk_change+0x1b/0x63
[<c046f4c3>] cdrom_open+0x7a1/0x806
[<c043b148>] sr_block_open+0x78/0x8d
[<c02ba4c0>] do_open+0x90/0x257
[<c02ba869>] blkdev_open+0x2d/0x56
[<c0296a1f>] __dentry_open+0x14d/0x23c
[<c0296b32>] nameidata_to_filp+0x24/0x38
[<c02a1c68>] do_filp_open+0x347/0x626
[<c02967ef>] do_sys_open+0x47/0xbc
[<c02968b0>] sys_open+0x23/0x2b
[<c021aadd>] sysenter_do_call+0x12/0x26
I've tested this on both x86_64 and i386.
Signed-off-by: Steven Rostedt <srostedt@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
|
