aboutsummaryrefslogtreecommitdiff
path: root/arch/sh/mm/cache-sh4.c
AgeCommit message (Collapse)Author
2010-01-26sh: Mass ctrl_in/outX to __raw_read/writeX conversion.Paul Mundt
The old ctrl in/out routines are non-portable and unsuitable for cross-platform use. While drivers/sh has already been sanitized, there is still quite a lot of code that is not. This converts the arch/sh/ bits over, which permits us to flag the routines as deprecated whilst still building with -Werror for the architecture code, and to ensure that future users are not added. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2010-01-21sh: Kill off the special uncached section and fixmap.Paul Mundt
Now that cached_to_uncached works as advertized in 32-bit mode and we're never going to be able to map < 16MB anyways, there's no need for the special uncached section. Kill it off. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2010-01-02sh: Optimise flush_dcache_page() on SH4Matt Fleming
If the page is not mapped into any process's address space then aliases cannot exist in the cache. So reduce the amount of flushing we perform. Signed-off-by: Matt Fleming <matt@console-pimps.org>
2009-12-09sh: Can't compare physical and virtual addresses for aliasesMatt Fleming
It does not make sense to compare virtual and physical addresses for aliasing, only virtual addresses can be compared for aliases. Signed-off-by: Matt Fleming <matt@console-pimps.org> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-12-04sh: Drop associative writes for SH-4 cache flushes.Matt Fleming
When flushing/invalidating the icache/dcache via the memory-mapped IC/OC address arrays, the associative bit should only be used in conjunction with virtual addresses. However, we currently flush cache lines based on physical address, so stop using the associative bit. It is a better strategy to use non-associative writes (and physical tags) for flushing the caches anyway, because flushing by virtual address (as with the A-bit set) requires a valid TLB entry for that virtual address. If one does not exist in the TLB no exception is generated and the flush is silently ignored. This is also future-proofing for SH-4A parts which are gradually phasing out associative writes to the cache array due to the aforementioned case of certain flushes silently turning in to nops. Signed-off-by: Matt Fleming <matt@console-pimps.org> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-11-09Merge branch 'sh/stable-updates'Paul Mundt
2009-11-09sh: Account for cache aliases in flush_icache_range()Matt Fleming
The icache may also contain aliases so we must account for them just like we do when manipulating the dcache. We usually get away with aliases in the icache because the instructions that are read from memory are read-only, i.e. they never change. However, the place where this bites us is when the code has been modified. Signed-off-by: Matt Fleming <matt@console-pimps.org> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-10-30sh: Do not apply virt_to_phys() to a physical addressMatt Fleming
The variable 'phys' already contains the physical address to flush. It is not a virtual address and should not be passed to virt_to_phys(). Signed-off-by: Matt Fleming <matt@console-pimps.org> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-10-16Merge branch 'sh/stable-updates'Paul Mundt
Conflicts: arch/sh/mm/cache-sh4.c
2009-10-16sh: Fix up single page flushing to use PAGE_SIZE.Valentin Sitdikov
Presently The SH-4 cache flushing code uses flush_cache_4096() for most of the real flushing work, which breaks down to a fixed 4096 unroll and increment. Not only is this sub-optimal for larger page sizes, it's also uncovered a bug in sh4_flush_dcache_page() when large page sizes are used and we have no cache aliases -- resulting in only a part of the page's D-cache lines being written back. Signed-off-by: Valentin Sitdikov <valentin.sitdikov@siemens.com> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-10-10sh: Prepare for dynamic PMB supportMatt Fleming
To allow the MMU to be switched between 29bit and 32bit mode at runtime some constants need to swapped for functions that return a runtime value. Signed-off-by: Matt Fleming <matt@console-pimps.org> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-10-10sh: Obliterate the P1 area macrosMatt Fleming
Replace the use of PHYSADDR() with __pa(). PHYSADDR() is based on the idea that all addresses in P1SEG are untranslated, so we can access an address's physical page as an offset from P1SEG. This doesn't work for CONFIG_PMB/CONFIG_PMB_FIXED because pages in P1SEG and P2SEG are used for PMB mappings and so can be translated to any physical address. Likewise, replace a P1SEGADDR() use with virt_to_phys(). Signed-off-by: Matt Fleming <matt@console-pimps.org> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-10-10Merge branch 'sh/cachetlb'Paul Mundt
2009-10-09sh: Sprinkle __uses_jump_to_uncachedMatt Fleming
Fix some callers of jump_to_uncached() and back_to_cached() that were not annotated with __uses_jump_to_uncached. Signed-off-by: Matt Fleming <matt@console-pimps.org> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-09-09sh: Fix up redundant cache flushing for PAGE_SIZE > 4k.Paul Mundt
If PAGE_SIZE is presently over 4k we do a lot of extra flushing given that we purge the cache 4k at a time. Make it explicitly 4k per iteration, rather than iterating for PAGE_SIZE before looping over again. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-09-09sh: Rework sh4_flush_cache_page() for coherent kmap mapping.Paul Mundt
This builds on top of the MIPS r4k code that does roughly the same thing. This permits the use of kmap_coherent() for mapped pages with dirty dcache lines and falls back on kmap_atomic() otherwise. This also fixes up a problem with the alias check and defers to shm_align_mask directly. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-09-09sh: Kill off segment-based d-cache flushing on SH-4.Paul Mundt
This kills off the unrolled segment based flushers on SH-4 and switches over to a generic unrolled approach derived from the writethrough segment flusher. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-09-09sh: Kill off broken PHYSADDR() usage in sh4_flush_dcache_page().Paul Mundt
PHYSADDR() runs in to issues in 32-bit mode when we do not have the legacy P1/P2 areas mapped, as such, we need to use page_to_phys() directly, which also happens to do the right thing in legacy 29-bit mode. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-09-09sh: sh4_flush_cache_mm() optimizations.Paul Mundt
The i-cache flush in the case of VM_EXEC was added way back when as a sanity measure, and in practice we only care about evicting aliases from the d-cache. As a result, it's possible to drop the i-cache flush completely here. After careful profiling it's also come up that all of the work associated with hunting down aliases and doing ranged flushing ends up generating more overhead than simply blasting away the entire dcache, particularly if there are many mm's that need to be iterated over. As a result of that, just move back to flush_dcache_all() in these cases, which restores the old behaviour, and vastly simplifies the path. Additionally, on platforms without aliases at all, this can simply be nopped out. Presently we have the alias check in the SH-4 specific version, but this is true for all of the platforms, so move the check up to a generic location. This cuts down quite a bit on superfluous cacheop IPIs. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-09-09sh: Cleanup whitespace damage in sh4_flush_icache_range().Paul Mundt
There was quite a lot of tab->space damage done here from a former patch, clean it up once and for all. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-09-01Revert "sh: Kill off now redundant local irq disabling."Paul Mundt
This reverts commit 64a6d72213dd810dd55bd0a503c36150af41c3c3. Unfortunately we can't use on_each_cpu() for all of the cache ops, as some of them only require preempt disabling. This seems to be the same issue that impacts the mips r4k caches, where this code was based on. This fixes up a deadlock that showed up in some IRQ context cases. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-09-01Merge branch 'master' into sh/smpPaul Mundt
Conflicts: arch/sh/mm/cache-sh4.c
2009-09-01sh: Fix dcache flushing for N-way write-through caches.Matt Fleming
This adopts the special-cased 2-way write-through dcache flusher for N-ways and moves it in to the generic path. Assignment is done at runtime via the check for the CCR_CACHE_WT bit in the same path as the per-way writeback flushers. Signed-off-by: Matt Fleming <matt@console-pimps.org> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-27sh: Fix up sh4_flush_dcache_page() build on UP.Paul Mundt
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-24sh: Fix problems with cache flushing when cache is in write-through modeStuart Menefy
Change the method used to flush the cache in write-through mode to avoid corrupted data being written back to memory. Signed-off-by: Stuart Menefy <stuart.menefy@st.com> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-24sh: Improve comments int SH4 cache flushing codeStuart Menefy
This is a pure documentation, to try to explain why the cache flushing code for the SH4 is implemented the way it is. Signed-off-by: Stuart Menefy <stuart.menefy@st.com> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-21sh: Kill off now redundant local irq disabling.Paul Mundt
on_each_cpu() takes care of IRQ and preempt handling, the localized handling in each of the called functions can be killed off. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-21sh: Make cache flushers SMP-aware.Paul Mundt
This does a bit of rework for making the cache flushers SMP-aware. The function pointer-based flushers are renamed to local variants with the exported interface being commonly implemented and wrapping as necessary. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-20sh: Fix up cache-sh4 build on SMP.Paul Mundt
mapping is unused on the SMP build, trigger a build error. Move it under the ifdef. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-15sh: Migrate SH-4 cacheflush ops to function pointers.Paul Mundt
This paves the way for allowing individual CPUs to overload the individual flushing routines that they care about without having to depend on weak aliases. SH-4 is converted over initially, as it wires up pretty much everything. The majority of the other CPUs will simply use the default no-op implementation with their own region flushers wired up. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-15sh: Kill off unused flush_icache_user_range().Paul Mundt
We use flush_cache_page() outright in copy_to_user_page(), and nothing else needs it, so just kill it off. SH-5 still defines its own version, but that too will go away in the same fashion once it converts over. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-15sh: Don't export flush_dcache_all().Paul Mundt
flush_dcache_all() is used internally by the SH-4 cache code, it is not part of the exported cache API, so make it static and don't export it. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-15sh: Move alias computation to shared cache init.Paul Mundt
This migrates the alias computation and printing of probed cache parameters from the SH-4 code to the shared cpu_cache_init(). This permits other platforms with aliases to make use of the same probe logic without having to roll their own, and also produces consistent output regardless of platform. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-15sh: Centralize the CPU cache initialization routines.Paul Mundt
This provides a central point for CPU cache initialization routines. This replaces the antiquated p3_cache_init() method, which the vast majority of CPUs never cared about. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-15sh: NO_CONTEXT ASID optimizations for SH-4 cache flush.Paul Mundt
This optimizes for the cases when a CPU does not yet have a valid ASID context associated with it, as in this case there is no work for any of flush_cache_mm()/flush_cache_page()/flush_cache_range() to do. Based on the the MIPS implementation. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-08-04sh: Split out SH-4 __flush_xxx_region() ops.Paul Mundt
This splits out the SH-4 __flush_xxx_region() functions and defines them as weak symbols. This allows us to provide optimized versions without having to ifdef cache-sh4.c to death. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2009-07-22sh: Migrate from PG_mapped to PG_dcache_dirty.Paul Mundt
This inverts the delayed dcache flush a bit to be more in line with other platforms. At the same time this also gives us the ability to do some more optimizations and cleanup. Now that the update_mmu_cache() callsite only tests for the bit, the implementation can gradually be split out and made generic, rather than relying on special implementations for each of the peculiar CPU types. SH7705 in 32kB mode and SH-4 still need slightly different handling, but this is something that can remain isolated in the varying page copy/clear routines. On top of that, SH-X3 is dcache coherent, so there is no need to bother with any of these tests in the PTEAEX version of update_mmu_cache(), so we kill that off too. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2008-09-08sh: uninline flush_icache_all().Paul Mundt
This uses jump_to_uncached() which is now given the noinline attribute due to the special section mapping. Kill off the inline attribute to fix up compilation failure. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2008-07-28sh: Optimized flush_icache_range() implementation.Chris Smith
Add implementation of flush_icache_range() suitable for signal handler and kprobes. Remove flush_cache_sigtramp() and change signal.c to use flush_icache_range(). Signed-off-by: Chris Smith <chris.smith@st.com> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2008-01-28sh: Preparation for uncached jumps through PMB.Stuart Menefy
Presently most of the 29-bit physical parts do P1/P2 segmentation with a 1:1 cached/uncached mapping, jumping between the two to control the caching behaviour. This provides the basic infrastructure to maintain this behaviour on 32-bit physical parts that don't map P1/P2 at all, using a shiny new linker section and corresponding fixmap entry. Signed-off-by: Stuart Menefy <stuart.menefy@st.com> Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2007-09-24sh: Calculate cache aliases on L2 caches.Paul Mundt
Calculate the number of cache aliases on probed L2 caches, and while we're at it, print out the detected statistics at boot time for these also. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2007-09-24sh: Fix alias calculation for non-aliasing cases.Paul Mundt
There was an off-by-1 on the cache alias detection logic on SH-4, which caused n_aliases to always be 1 even when the page size precluded the existence of aliases. With this corrected, 64KB pages happily reports n_aliases == 0, and hits the appropriate fast paths in the flushing routines. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2007-09-21sh: Avoid smp_processor_id() in cache desc paths.Paul Mundt
current_cpu_data uses smp_processor_id() in order to find the corresponding cpu_data. As the cache descs are all currently identical, just have this look at probed results from the boot CPU. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2007-07-25sh: Reclaim beginning of P3 space for vmalloc area.Paul Mundt
The first 1MB of P3 space was reserved and used for page colouring, as we've reworked that to use fixmaps, we can reclaim the space and hand it back to VMALLOC_START. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2007-07-24sh: Add kmap_coherent()/kunmap_coherent() interface for SH-4.Paul Mundt
This wires up kmap_coherent() and kunmap_coherent() on SH-4, and moves away from the p3map_mutex and reserved P3 space, opting to use fixmaps for colouring instead. The copy_user_page()/clear_user_page() implementations are moved to this, which fixes the nasty blowups with spinlock debugging as a result of having some of these calls nested under the page table lock. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2007-03-05sh: Revert lazy dcache writeback changes.Paul Mundt
These ended up causing too many problems on older parts, revert for now.. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2007-02-13sh: Fixup cpu_data references for the non-boot CPUs.Paul Mundt
There are a lot of bogus cpu_data-> references that only end up working for the boot CPU, convert these to current_cpu_data to fixup SMP. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2007-02-13sh: Lazy dcache writeback optimizations.Paul Mundt
This converts the lazy dcache handling to the model described in Documentation/cachetlb.txt and drops the ptep_get_and_clear() hacks used for the aliasing dcaches on SH-4 and SH7705 in 32kB mode. As a bonus, this slightly cuts down on the cache flushing frequency. With that and the PTEA handling out of the way, the update_mmu_cache() implementations can be consolidated, and we no longer have to worry about which configuration the cache is in for the SH7705 case. And finally, explicitly disable the lazy writeback on SMP (SH-4A). Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2006-12-12sh: Convert remaining remap_area_pages() users to ioremap_page_range().Paul Mundt
A couple of these were missed. Signed-off-by: Paul Mundt <lethal@linux-sh.org>
2006-12-06sh: Fixup various PAGE_SIZE == 4096 assumptions.Paul Mundt
There were a number of places that made evil PAGE_SIZE == 4k assumptions that ended up breaking when trying to play with 8k and 64k page sizes, this fixes those up. The most significant change is the way we load THREAD_SIZE, previously this was done via: mov #(THREAD_SIZE >> 8), reg shll8 reg to avoid a memory access and allow the immediate load. With a 64k PAGE_SIZE, we're out of range for the immediate load size without resorting to special instructions available in later ISAs (movi20s and so on). The "workaround" for this is to bump up the shift to 10 and insert a shll2, which gives a bit more flexibility while still being much cheaper than a memory access. Signed-off-by: Paul Mundt <lethal@linux-sh.org>