aboutsummaryrefslogtreecommitdiff
path: root/include/asm-x86/pgtable-3level.h
diff options
context:
space:
mode:
authorJeremy Fitzhardinge <jeremy@goop.org>2008-01-30 13:34:11 +0100
committerIngo Molnar <mingo@elte.hu>2008-01-30 13:34:11 +0100
commitfa28ba21cec24d3fa1279bcae7e5d5ff6224635a (patch)
treedda47a7faa85428753691a50f49b0404f78ea0ec /include/asm-x86/pgtable-3level.h
parentf212ec4b7b4d84290f12c9c0416cdea283bf5f40 (diff)
x86: defer cr3 reload when doing pud_clear()
PAE mode requires that we reload cr3 in order to guarantee that changes to the pgd will be noticed by the processor. This means that in principle pud_clear needs to reload cr3 every time. However, because reloading cr3 implies a tlb flush, we want to avoid it where possible. pud_clear() is only used in a couple of places: - in free_pmd_range(), when pulling down a range of process address space, and - huge_pmd_unshare() In both cases, the calling code will do a a tlb flush anyway, so there's no need to do it within pud_clear(). In free_pmd_range(), the pud_clear is immediately followed by pmd_free_tlb(); we can hook that to make the mmu_gather do an unconditional full flush to make sure cr3 gets reloaded. In huge_pmd_unshare, it is followed by flush_tlb_range, which always results in a full cr3-reload tlb flush. Signed-off-by: Jeremy Fitzhardinge <jeremy@xensource.com> Cc: Andi Kleen <ak@suse.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: H. Peter Anvin <hpa@zytor.com> Cc: William Irwin <wli@holomorphy.com> Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Diffstat (limited to 'include/asm-x86/pgtable-3level.h')
-rw-r--r--include/asm-x86/pgtable-3level.h21
1 files changed, 15 insertions, 6 deletions
diff --git a/include/asm-x86/pgtable-3level.h b/include/asm-x86/pgtable-3level.h
index ed4c6f0e57e..a195c3e757b 100644
--- a/include/asm-x86/pgtable-3level.h
+++ b/include/asm-x86/pgtable-3level.h
@@ -96,14 +96,23 @@ static inline void pud_clear(pud_t *pudp)
set_pud(pudp, __pud(0));
/*
- * Pentium-II erratum A13: in PAE mode we explicitly have to flush
- * the TLB via cr3 if the top-level pgd is changed...
+ * In principle we need to do a cr3 reload here to make sure
+ * the processor recognizes the changed pgd. In practice, all
+ * the places where pud_clear() gets called are followed by
+ * full tlb flushes anyway, so we can defer the cost here.
*
- * XXX I don't think we need to worry about this here, since
- * when clearing the pud, the calling code needs to flush the
- * tlb anyway. But do it now for safety's sake. - jsgf
+ * Specifically:
+ *
+ * mm/memory.c:free_pmd_range() - immediately after the
+ * pud_clear() it does a pmd_free_tlb(). We change the
+ * mmu_gather structure to do a full tlb flush (which has the
+ * effect of reloading cr3) when the pagetable free is
+ * complete.
+ *
+ * arch/x86/mm/hugetlbpage.c:huge_pmd_unshare() - the call to
+ * this is followed by a flush_tlb_range, which on x86 does a
+ * full tlb flush.
*/
- write_cr3(read_cr3());
}
#define pud_page(pud) \