aboutsummaryrefslogtreecommitdiff
path: root/arch/s390/kernel/smp.c
AgeCommit message (Collapse)Author
2009-03-26[S390] cpumask: Use accessors code.Rusty Russell
Impact: use new API Use the accessors rather than frobbing bits directly. Most of this is in arch code I haven't even compiled, but is straightforward. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Signed-off-by: Mike Travis <travis@sgi.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2009-03-26[S390] cpumask: prepare for iterators to only go to nr_cpu_ids/nr_cpumask_bits.Rusty Russell
Impact: cleanup, futureproof In fact, all cpumask ops will only be valid (in general) for bit numbers < nr_cpu_ids. So use that instead of NR_CPUS in various places (I also updated the immediate sites to use the new cpumask_ operators). This is always safe: no cpu number can be >= nr_cpu_ids, and nr_cpu_ids is initialized to NR_CPUS at boot. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Signed-off-by: Mike Travis <travis@sgi.com> Acked-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2009-03-26[S390] smp: perform initial cpu reset before starting a cpuHeiko Carstens
Performing an initial cpu reset makes sure all registers and tlbs of the targeted cpu are initialized and flushed. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2009-03-26[S390] smp: fix memory leak on __cpu_upHeiko Carstens
If sigp_set_prefix fails on __cpu_up we leak the lowcore structures and async+panic stacks for the failed cpu. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2009-03-26[S390] zfcpdump: Prevent zcore from beeing built as a kernel module.Michael Holzheu
The zcore code switches to real addressing mode when creating a kernel dump. This is not possible, if it is built as a kernel module. With this patch zcore (zfcpdump) can't be built as a kernel module any more. Signed-off-by: Michael Holzheu <holzheu@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2009-03-26[S390] eliminate ipl_device from lowcoreMartin Schwidefsky
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2009-03-26[S390] eliminate cpuinfo_S390 structureMartin Schwidefsky
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2009-03-26[S390] lockdep: trace hardirq off in smp_send_stopChristian Borntraeger
With lockdep we got the following trace after a panic: Badness at /home/autobuild/BUILD/linux-2.6.28-20090204/kernel/lockdep.c:2878 [...] Call Trace: [<0000000000176334>] lock_acquire+0x54/0xbc [<000000000050b4fe>] __atomic_notifier_call_chain+0x6e/0xdc [<000000000050b59c>] atomic_notifier_call_chain+0x30/0x44 [<0000000000504274>] panic+0xd0/0x1e8 [...] INFO: lockdep is turned off. Last Breaking-Event-Address: [<0000000000170e62>] check_flags+0xae/0x15c possible reason: unannotated irqs-off. lockdep is right. We missed a trace_hardirq_off in our smp_send_stop function and smp_send_stop is called before the panic call chain. Reported-by: Mijo <Safradin mijo@linux.vnet.ibm.com> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2009-03-26[S390] Automatic IPL after dumpFrank Munzert
Provide new shutdown action "dump_reipl" for automatic ipl after dump. Signed-off-by: Frank Munzert <munzert@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
2009-01-09[S390] vdso: compile fixHeiko Carstens
!CONFIG_SMP: arch/s390/kernel/vdso.c: In function 'vdso_init': arch/s390/kernel/vdso.c:325: error: incompatible type for argument 2 of 'vdso_alloc_per_cpu' Also move the code out of the BUG_ON statement since it won't be executed on !CONFIG_BUG. And that would be a bug. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2009-01-03Merge branch 'cputime' of git://git390.osdl.marist.edu/pub/scm/linux-2.6Linus Torvalds
* 'cputime' of git://git390.osdl.marist.edu/pub/scm/linux-2.6: [PATCH] fast vdso implementation for CLOCK_THREAD_CPUTIME_ID [PATCH] improve idle cputime accounting [PATCH] improve precision of idle time detection. [PATCH] improve precision of process accounting. [PATCH] idle cputime accounting [PATCH] fix scaled & unscaled cputime accounting
2009-01-02Merge branch 'cpus4096-for-linus-2' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip * 'cpus4096-for-linus-2' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip: (66 commits) x86: export vector_used_by_percpu_irq x86: use logical apicid in x2apic_cluster's x2apic_cpu_mask_to_apicid_and() sched: nominate preferred wakeup cpu, fix x86: fix lguest used_vectors breakage, -v2 x86: fix warning in arch/x86/kernel/io_apic.c sched: fix warning in kernel/sched.c sched: move test_sd_parent() to an SMP section of sched.h sched: add SD_BALANCE_NEWIDLE at MC and CPU level for sched_mc>0 sched: activate active load balancing in new idle cpus sched: bias task wakeups to preferred semi-idle packages sched: nominate preferred wakeup cpu sched: favour lower logical cpu number for sched_mc balance sched: framework for sched_mc/smt_power_savings=N sched: convert BALANCE_FOR_xx_POWER to inline functions x86: use possible_cpus=NUM to extend the possible cpus allowed x86: fix cpu_mask_to_apicid_and to include cpu_online_mask x86: update io_apic.c to the new cpumask code x86: Introduce topology_core_cpumask()/topology_thread_cpumask() x86: xen: use smp_call_function_many() x86: use work_on_cpu in x86/kernel/cpu/mcheck/mce_amd_64.c ... Fixed up trivial conflict in kernel/time/tick-sched.c manually
2008-12-31[PATCH] fast vdso implementation for CLOCK_THREAD_CPUTIME_IDMartin Schwidefsky
The extract cpu time instruction (ectg) instruction allows the user process to get the current thread cputime without calling into the kernel. The code that uses the instruction needs to switch to the access registers mode to get access to the per-cpu info page that contains the two base values that are needed to calculate the current cputime from the CPU timer with the ectg instruction. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-12-31[PATCH] improve precision of idle time detection.Martin Schwidefsky
Increase the precision of the idle time calculation that is exported to user space via /sys/devices/system/cpu/cpu<x>/idle_time_us Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-12-25[S390] convert cpu related printks to pr_xxx macros.Martin Schwidefsky
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-12-25[S390] panic_stack leak in smp_alloc_lowcoreMartin Schwidefsky
Fix freeing of the panic_stack if the allocation of async_stack failed. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-12-25[S390] Remove config options.Martin Schwidefsky
On s390 we always want to run with precise cputime accounting. Remove the config options VIRT_TIMER and VIRT_CPU_ACCOUNTING. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-12-25[S390] convert s390 to generic IPI infrastructureHeiko Carstens
Since etr/stp don't need the old smp_call_function semantics anymore we can convert s390 to the generic IPI infrastructure. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-12-13cpumask: centralize cpu_online_map and cpu_possible_mapRusty Russell
Impact: cleanup Each SMP arch defines these themselves. Move them to a central location. Twists: 1) Some archs (m32, parisc, s390) set possible_map to all 1, so we add a CONFIG_INIT_ALL_POSSIBLE for this rather than break them. 2) mips and sparc32 '#define cpu_possible_map phys_cpu_present_map'. Those archs simply have phys_cpu_present_map replaced everywhere. 3) Alpha defined cpu_possible_map to cpu_present_map; this is tricky so I just manipulate them both in sync. 4) IA64, cris and m32r have gratuitous 'extern cpumask_t cpu_possible_map' declarations. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Reviewed-by: Grant Grundler <grundler@parisc-linux.org> Tested-by: Tony Luck <tony.luck@intel.com> Acked-by: Ingo Molnar <mingo@elte.hu> Cc: Mike Travis <travis@sgi.com> Cc: ink@jurassic.park.msu.ru Cc: rmk@arm.linux.org.uk Cc: starvik@axis.com Cc: tony.luck@intel.com Cc: takata@linux-m32r.org Cc: ralf@linux-mips.org Cc: grundler@parisc-linux.org Cc: paulus@samba.org Cc: schwidefsky@de.ibm.com Cc: lethal@linux-sh.org Cc: wli@holomorphy.com Cc: davem@davemloft.net Cc: jdike@addtoit.com Cc: mingo@redhat.com
2008-10-28[S390] Fix sysdev class file creation.Heiko Carstens
Use sysdev_class_create_file() to create create sysdev class attributes instead of sysfs_create_file(). Using sysfs_create_file() wasn't a very good idea since the show and store functions have a different amount of parameters for sysfs files and sysdev class files. In particular the pointer to the buffer is the last argument and therefore accesses to random memory regions happened. Still worked surprisingly well until we got a kernel panic. Cc: stable@kernel.org Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-09-08kernel/cpu.c: create a CPU_STARTING cpu_chain notifierManfred Spraul
Right now, there is no notifier that is called on a new cpu, before the new cpu begins processing interrupts/softirqs. Various kernel function would need that notification, e.g. kvm works around by calling smp_call_function_single(), rcu polls cpu_online_map. The patch adds a CPU_STARTING notification. It also adds a helper function that sends the message to all cpu_chain handlers. Tested on x86-64. All other archs are untested. Especially on sparc, I'm not sure if I got it right. Signed-off-by: Manfred Spraul <manfred@colorfullife.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-08-21[S390] Remove unneeded spinlock initialization.Heiko Carstens
Remove the now unneeded s390_idle.lock spinlock initialization after Josef Sipek did it the right way in arch/s390/kernel/process.c. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-07-21sysdev: Pass the attribute to the low level sysdev show/store functionAndi Kleen
This allow to dynamically generate attributes and share show/store functions between attributes. Right now most attributes are generated by special macros and lots of duplicated code. With the attribute passed it's instead possible to attach some data to the attribute and then use that in shared low level functions to do different things. I need this for the dynamically generated bank attributes in the x86 machine check code, but it'll allow some further cleanups. I converted all users in tree to the new show/store prototype. It's a single huge patch to avoid unbisectable sections. Runtime tested: x86-32, x86-64 Compiled only: ia64, powerpc Not compile tested/only grep converted: sh, arm, avr32 Signed-off-by: Andi Kleen <ak@linux.intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2008-06-26on_each_cpu(): kill unused 'retry' parameterJens Axboe
It's not even passed on to smp_call_function() anymore, since that was removed. So kill it. Acked-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2008-06-26smp_call_function: get rid of the unused nonatomic/retry argumentJens Axboe
It's never used and the comments refer to nonatomic and retry interchangably. So get rid of it. Acked-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2008-06-10[S390] Fix build failure in __cpu_up()Segher Boessenkool
The first argument to __ctl_store() should be the array to store stuff in, not just the first element of that array. With the current code in __cpu_up(), mainline GCC dies with an internal compiler error. I didn't diagnose that further, but just fixed the kernel bug. Signed-off-by: Segher Boessenkool <segher@kernel.crashing.org> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com> Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
2008-05-30[S390] Fix section mismatch warnings.Heiko Carstens
This fixes the last remaining section mismatch warnings in s390 architecture code. It reveals also a real bug introduced by... me with git commit 2069e978d5a6e7b45d58027e3de7f879b8c5e488 ("[S390] sparsemem vmemmap: initialize memmap.") Calling the generic vmemmap_alloc_block() function to get initialized memory is a nice idea, however that function is __meminit annotated and therefore the function might be gone if we try to call it later. This can happen if a DCSS segment gets added. So basically revert the patch and clear the memmap explicitly to fix the original bug. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-05-15[S390] smp: __smp_call_function_map vs cpu_online_map fix.Heiko Carstens
Both smp_call_function() and __smp_call_function_map() access cpu_online_map. Both functions run with preemption disabled which protects for cpus going offline. However new cpus can be added and therefore the cpu_online_map can change unexpectedly. So use the call_lock to protect against changes to the cpu_online_map in start_secondary() and all smp_call_* functions. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-04-30[S390] Automatically detect added cpus.Heiko Carstens
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-04-30[S390] smp: Fix locking order.Heiko Carstens
On some smp sysfs store attributes get_online_cpus() may block on cpu_hotplug.lock, but we hold already smp_cpu_state_mutex. Since the locking order on cpu hotplug via arch_update_cpu_topology is inverse this might lead to deadlocks. So make sure locking order is always the same. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-04-17[S390] Fix a lot of sparse warnings.Heiko Carstens
Most noteable part of this commit is the new local header file entry.h which contains all the function declarations of functions that get only called from asm code or are arch internal. That way we can avoid extern declarations in C files. This is more or less the same that was done for sparc64. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
2008-04-17[S390] Convert monitor calls to function calls.Heiko Carstens
Remove the program check generating monitor calls and use function calls instead. Theres is no real advantage in using monitor calls, but they do make debugging harder, because of all the program checks it generates. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
2008-04-17[S390] Vertical cpu management.Heiko Carstens
If vertical cpu polarization is active then the hypervisor will dispatch certain cpus for a longer time than other cpus for maximum performance. For example if a guest would have three virtual cpus, each of them with a share of 33 percent, then in case of vertical cpu polarization all of the processing time would be combined to a single cpu which would run all the time, while the other two cpus would get nearly no cpu time. There are three different types of vertical cpus: high, medium and low. Low cpus hardly get any real cpu time, while high cpus get a full real cpu. Medium cpus get something in between. In order to switch between the two possible modes (default is horizontal) a 0 for horizontal polarization or a 1 for vertical polarization must be written to the dispatching sysfs attribute: /sys/devices/system/cpu/dispatching The polarization of each single cpu can be figured out by the polarization sysfs attribute of each cpu: /sys/devices/system/cpu/cpuX/polarization horizontal, vertical:high, vertical:medium, vertical:low or unknown. When switching polarization the polarization attribute may contain the value unknown until the configuration change is done and the kernel has figured out the new polarization of each cpu. Note that running a system with different types of vertical cpus may result in significant performance regressions. If possible only one type of vertical cpus should be used. All other cpus should be offlined. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
2008-04-17[S390] cpu topology support for s390.Heiko Carstens
Add s390 backend so we can give the scheduler some hints about the cpu topology. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
2008-03-05[S390] Get rid of memcpy gcc warning workaround.Heiko Carstens
Compile smp.o with -Wno-nonnull so gcc stops warning about memcpy being used with a null parameter. Also remove the workaround code and use a char * cast instead of a void * cast to do computations. Cc: Bastian Blank <bastian@waldi.eu.org> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-02-19[S390] Initialize per cpu lowcores on cpu hotplug.Heiko Carstens
Just copy the first 512 read-only bytes of the current cpu lowcore if a new cpu gets onlined. The rest is zeroed out and must be explicitly initialized. Current code just copies the entire lowcore and initializes the needed fields. This should reveal bugs in future enhancements quite early. Also when the lowcore of the first cpu is replaced this is now done atomically (no interrupts, no machine checks). Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-02-05[S390] Fix couple of section mismatches.Heiko Carstens
Fix couple of section mismatches. And since we touch the code anyway change the IPL code to use C99 initializers. Cc: Michael Holzheu <holzheu@de.ibm.com> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-02-05[S390] Fix smp_call_function_mask semantics.Heiko Carstens
Make sure func isn't called on the local cpu just like on all other architectures that implement this function. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-01-26[S390] replace lock_cpu_hotplug with get_online_cpusMartin Schwidefsky
Git commit 86ef5c9a8edd78e6bf92879f32329d89b2d55b5a forgot a few lock_cpu_hotplug/unlock_cpu_hotplug pairs in arch/s390/kernel/smp.c Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-01-26[S390] add smp_call_function_maskCarsten Otte
This patch adds the s390 variant for smp_call_function_mask(). The implementation is pretty straight forward using the wrapper __smp_call_function_map() which already takes a cpumask_t argument. Signed-off-by: Carsten Otte <cotte@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-01-26[S390] Allocate and free cpu lowcores and stacks when needed/possible.Heiko Carstens
No need to preallocate the per cpu lowcores and stacks. Savings are 28-32k per offline cpu. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-01-26[S390] kernel: Shutdown Actions InterfaceMichael Holzheu
In case of a kernel panic it is currently possible to specify that a dump should be created, the system should be rebooted or stopped. Virtual sysfs files under the directory /sys/firmware/ are used for that configuration. In addition to that, there are kernel parameters 'vmhalt', 'vmpoff' and 'vmpanic', which can be used to specify z/VM commands, which are automatically executed in case of halt, power off or a kernel panic. This patch combines both functionalities and allows to specify the z/VM CP commands also via sysfs attributes. In addition to that, it enhances the existing handling of shutdown triggers (e.g. halt or panic) and associated shutdown actions (e.g. dump or reipl) and makes it more flexible. Signed-off-by: Michael Holzheu <holzheu@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-01-26[S390] Get rid of additional_cpus kernel parameter.Heiko Carstens
It caused only a lot of confusion. From now on cpu hotplug of up to NR_CPUS will work by default. If somebody wants to limit that then the possible_cpus parameter can be used. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2008-01-26[S390] Standby cpu activation/deactivation.Heiko Carstens
Add a new interface so that cpus can be put into standby state and configured state. Only offline cpus can be put into standby state or configured state. For that the new percpu sysfs attribute "configure" must be used. To put a cpu in standby state a "0" must be written to the attribute. In order to switch it into configured state a "1" must be written to the attribute. Only cpus in configured state can be brought online. In addition this patch introduces a static mapping of physical to logical cpus. As a result only the sysfs directories of present cpus will be created. To scan for new cpus the new sysfs attribute "rescan" must be used. Writing to /sys/devices/system/cpu/rescan will trigger a rescan of cpus and will create directories for new cpus. On IPL only configured cpus will be used. And on reboot/shutdown all cpus will remain in their current state (configured/standby). Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2007-11-20[S390] Dont overwrite lowcores on smp_send_stop().Heiko Carstens
Don't perform a sigp store-status-at-address on smp_send_stop(). It will overwrite the lowcores of other cpus and destroys valueable debug informations. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2007-11-05[S390] Rename "idle_time" attribute to "idle_time_us".Heiko Carstens
Seems that people prefer to have the unit encoded in the attribute name. Also makes parsing easier. Now we have: # cat /sys/devices/system/cpu/cpu0/idle_time_us 131473592 instead of # cat /sys/devices/system/cpu/cpu0/idle_time 131473592 us Cc: Arjan van de Ven <arjan@infradead.org> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2007-10-22[S390] tlb flush fix.Martin Schwidefsky
The current tlb flushing code for page table entries violates the s390 architecture in a small detail. The relevant section from the principles of operation (SA22-7832-02 page 3-47): "A valid table entry must not be changed while it is attached to any CPU and may be used for translation by that CPU except to (1) invalidate the entry by using INVALIDATE PAGE TABLE ENTRY or INVALIDATE DAT TABLE ENTRY, (2) alter bits 56-63 of a page-table entry, or (3) make a change by means of a COMPARE AND SWAP AND PURGE instruction that purges the TLB." That means if one thread of a multithreaded applciation uses a vma while another thread does an unmap on it, the page table entries of that vma needs to get removed with IPTE, IDTE or CSP. In some strange and rare situations a cpu could check-stop (die) because a entry has been pushed out of the TLB that is still needed to complete a (milli-coded) instruction. I've never seen it happen with the current code on any of the supported machines, so right now this is a theoretical problem. But I want to fix it nevertheless, to avoid headaches in the futures. To get this implemented correctly without changing common code the primitives ptep_get_and_clear, ptep_get_and_clear_full and ptep_set_wrprotect need to use the IPTE instruction to invalidate the pte before the new pte value gets stored. If IPTE is always used for the three primitives three important operations will have a performace hit: fork, mprotect and exit_mmap. Time for some workarounds: * 1: ptep_get_and_clear_full is used in unmap_vmas to remove page tables entries in a batched tlb gather operation. If the mmu_gather context passed to unmap_vmas has been started with full_mm_flush==1 or if only one cpu is online or if the only user of a mm_struct is the current process then the fullmm indication in the mmu_gather context is set to one. All TLBs for mm_struct are flushed by the tlb_gather_mmu call. No new TLBs can be created while the unmap is in progress. In this case ptep_get_and_clear_full clears the ptes with a simple store. * 2: ptep_get_and_clear is used in change_protection to clear the ptes from the page tables before they are reentered with the new access flags. At the end of the update flush_tlb_range clears the remaining TLBs. In general the ptep_get_and_clear has to issue IPTE for each pte and flush_tlb_range is a nop. But if there is only one user of the mm_struct then ptep_get_and_clear uses simple stores to do the update and flush_tlb_range will flush the TLBs. * 3: Similar to 2, ptep_set_wrprotect is used in copy_page_range for a fork to make all ptes of a cow mapping read-only. At the end of of copy_page_range dup_mmap will flush the TLBs with a call to flush_tlb_mm. Check for mm->mm_users and if there is only one user avoid using IPTE in ptep_set_wrprotect and let flush_tlb_mm clear the TLBs. Overall for single threaded programs the tlb flush code now performs better, for multi threaded programs it is slightly worse. In particular exit_mmap() now does a single IDTE for the mm and then just frees every page cache reference and every page table page directly without a delay over the mmu_gather structure. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2007-10-22[S390] Add per-cpu idle time / idle count sysfs attributes.Heiko Carstens
Add two new sysfs entries per cpu: idle_count and idle_time. idle_count contains the number of times a cpu went into idle state. idle_time contains the time a cpu spent in idle state in microseconds. This can be used e.g. by powertop to tell how often idle state is entered and left. # cat /sys/devices/system/cpu/cpu0/idle_count 504 # cat /sys/devices/system/cpu/cpu0/idle_time 469734037 us Cc: Arjan van de Ven <arjan@infradead.org> Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2007-07-29Remove fs.h from mm.hAlexey Dobriyan
Remove fs.h from mm.h. For this, 1) Uninline vma_wants_writenotify(). It's pretty huge anyway. 2) Add back fs.h or less bloated headers (err.h) to files that need it. As result, on x86_64 allyesconfig, fs.h dependencies cut down from 3929 files rebuilt down to 3444 (-12.3%). Cross-compile tested without regressions on my two usual configs and (sigh): alpha arm-mx1ads mips-bigsur powerpc-ebony alpha-allnoconfig arm-neponset mips-capcella powerpc-g5 alpha-defconfig arm-netwinder mips-cobalt powerpc-holly alpha-up arm-netx mips-db1000 powerpc-iseries arm arm-ns9xxx mips-db1100 powerpc-linkstation arm-assabet arm-omap_h2_1610 mips-db1200 powerpc-lite5200 arm-at91rm9200dk arm-onearm mips-db1500 powerpc-maple arm-at91rm9200ek arm-picotux200 mips-db1550 powerpc-mpc7448_hpc2 arm-at91sam9260ek arm-pleb mips-ddb5477 powerpc-mpc8272_ads arm-at91sam9261ek arm-pnx4008 mips-decstation powerpc-mpc8313_rdb arm-at91sam9263ek arm-pxa255-idp mips-e55 powerpc-mpc832x_mds arm-at91sam9rlek arm-realview mips-emma2rh powerpc-mpc832x_rdb arm-ateb9200 arm-realview-smp mips-excite powerpc-mpc834x_itx arm-badge4 arm-rpc mips-fulong powerpc-mpc834x_itxgp arm-carmeva arm-s3c2410 mips-ip22 powerpc-mpc834x_mds arm-cerfcube arm-shannon mips-ip27 powerpc-mpc836x_mds arm-clps7500 arm-shark mips-ip32 powerpc-mpc8540_ads arm-collie arm-simpad mips-jazz powerpc-mpc8544_ds arm-corgi arm-spitz mips-jmr3927 powerpc-mpc8560_ads arm-csb337 arm-trizeps4 mips-malta powerpc-mpc8568mds arm-csb637 arm-versatile mips-mipssim powerpc-mpc85xx_cds arm-ebsa110 i386 mips-mpc30x powerpc-mpc8641_hpcn arm-edb7211 i386-allnoconfig mips-msp71xx powerpc-mpc866_ads arm-em_x270 i386-defconfig mips-ocelot powerpc-mpc885_ads arm-ep93xx i386-up mips-pb1100 powerpc-pasemi arm-footbridge ia64 mips-pb1500 powerpc-pmac32 arm-fortunet ia64-allnoconfig mips-pb1550 powerpc-ppc64 arm-h3600 ia64-bigsur mips-pnx8550-jbs powerpc-prpmc2800 arm-h7201 ia64-defconfig mips-pnx8550-stb810 powerpc-ps3 arm-h7202 ia64-gensparse mips-qemu powerpc-pseries arm-hackkit ia64-sim mips-rbhma4200 powerpc-up arm-integrator ia64-sn2 mips-rbhma4500 s390 arm-iop13xx ia64-tiger mips-rm200 s390-allnoconfig arm-iop32x ia64-up mips-sb1250-swarm s390-defconfig arm-iop33x ia64-zx1 mips-sead s390-up arm-ixp2000 m68k mips-tb0219 sparc arm-ixp23xx m68k-amiga mips-tb0226 sparc-allnoconfig arm-ixp4xx m68k-apollo mips-tb0287 sparc-defconfig arm-jornada720 m68k-atari mips-workpad sparc-up arm-kafa m68k-bvme6000 mips-wrppmc sparc64 arm-kb9202 m68k-hp300 mips-yosemite sparc64-allnoconfig arm-ks8695 m68k-mac parisc sparc64-defconfig arm-lart m68k-mvme147 parisc-allnoconfig sparc64-up arm-lpd270 m68k-mvme16x parisc-defconfig um-x86_64 arm-lpd7a400 m68k-q40 parisc-up x86_64 arm-lpd7a404 m68k-sun3 powerpc x86_64-allnoconfig arm-lubbock m68k-sun3x powerpc-cell x86_64-defconfig arm-lusl7200 mips powerpc-celleb x86_64-up arm-mainstone mips-atlas powerpc-chrp32 Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-07-27[S390] Improve __smp_call_function_map.Heiko Carstens
There is no need to disable bottom halves when holding call_lock. Also this could imply that it is legal to call smp_call_function* from bh context, which it is not. Also test if func will be executed locally before disabling and aterwards enabling interrupts again. It's not necessary to disable and enable interrupts each time __smp_call_function_map gets called. Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>