aboutsummaryrefslogtreecommitdiff
path: root/drivers/pci/intel-iommu.c
AgeCommit message (Collapse)Author
2008-04-24intel-iommu.c: dma ops fixIngo Molnar
Stephen Rothwell noticed that: Commit 2be621498d461b63ca6124f86e3b9582e1a8e722 ("x86: dma-ops on highmem fix") in Linus' tree introduced a new warning (noticed in the x86_64 allmodconfig build of linux-next): drivers/pci/intel-iommu.c:2240: warning: initialization from incompatible pointer type Which points at an instance of map_single that needs updating. Fix it to the new prototype. Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-04-20PCI: pci-iommu-iotlb-flushing-speedupmark gross
The following patch is an update to use an array instead of a list of IOVA's in the implementation of defered iotlb flushes. It takes inspiration from sba_iommu.c I like this implementation better as it encapsulates the batch process within intel-iommu.c, and no longer touches iova.h (which is shared) Performance data: Netperf 32byte UDP streaming 2.6.25-rc3-mm1: IOMMU-strict : 58Mps @ 62% cpu NO-IOMMU : 71Mbs @ 41% cpu List-based IOMMU-default-batched-IOTLB flush: 66Mbps @ 57% cpu with this patch: IOMMU-strict : 73Mps @ 75% cpu NO-IOMMU : 74Mbs @ 42% cpu Array-based IOMMU-default-batched-IOTLB flush: 72Mbps @ 62% cpu Signed-off-by: <mgross@linux.intel.com> Cc: Grant Grundler <grundler@parisc-linux.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2008-04-20PCI: iommu: iotlb flushingmark gross
This patch is for batching up the flushing of the IOTLB for the DMAR implementation found in the Intel VT-d hardware. It works by building a list of to be flushed IOTLB entries and a bitmap list of which DMAR engine they are from. After either a high water mark (250 accessible via debugfs) or 10ms the list of iova's will be reclaimed and the DMAR engines associated are IOTLB-flushed. This approach recovers 15 to 20% of the performance lost when using the IOMMU for my netperf udp stream benchmark with small packets. It can be disabled with a kernel boot parameter "intel_iommu=strict". Its use does weaken the IOMMU protections a bit. Signed-off-by: Mark Gross <mgross@linux.intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2008-03-24PCI: iova: lockdep false alarm fixMark Gross
lockdep goes off on the iova copy_reserved_iova() because it and a function it calls grabs locks in the from, and the to of the copy operation. The function grab locks of the same lock classes triggering the warning. The first lock grabbed is for the constant reserved areas that is never accessed after early boot. Technically you could do without grabbing the locks for the "from" structure its copying reserved areas from. But dropping the from locks to me looks wrong, even though it would be ok. The affected code only runs in early boot as its setting up the DMAR engines. This patch gives the reserved_ioval_list locks special lockdep classes. Signed-off-by: Mark Gross <mgross@linux.intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2008-02-23copyright owner and author clean up for intel iommu and related filesmark gross
The following is a clean up and correction of the copyright holding entities for the files associated with the intel iommu code. Signed-off-by: <mgross@linux.intel.com> Cc: Greg KH <greg@kroah.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-08intel-iommu: fault_reason index cleanupmark gross
Fix an off by one bug in the fault reason string reporting function, and clean up some of the code around this buglet. [akpm@linux-foundation.org: cleanup] Signed-off-by: mark gross <mgross@linux.intel.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Andi Kleen <ak@suse.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-08intel-iommu: PMEN supportmark gross
Add support for protected memory enable bits by clearing them if they are set at startup time. Some future boot loaders or firmware could have this bit set after it loads the kernel, and it needs to be cleared if DMA's are going to happen effectively. Signed-off-by: mark gross <mgross@intel.com> Acked-by: Muli Ben-Yehuda <muli@il.ibm.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Andi Kleen <ak@suse.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-06Genericizing iova.[ch]David Miller
I would like to potentially move the sparc64 IOMMU code over to using the nice new drivers/pci/iova.[ch] code for free area management.. In order to do that we have to detach the IOMMU page size assumptions which only really need to exist in the intel-iommu.[ch] code. This patch attempts to implement that. [akpm@linux-foundation.org: build fix] Signed-off-by: David S. Miller <davem@davemloft.net> Acked-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-02-01PCI: Spelling fixesJoe Perches
Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2007-10-30intel-iommu: Fix array overflowTakashi Iwai
Fix possible array overflow: drivers/pci/intel-iommu.c: In function ¡dmar_get_fault_reason¢: drivers/pci/intel-iommu.c:753: warning: array subscript is above array bounds drivers/pci/intel-iommu.c: In function ¡iommu_page_fault¢: drivers/pci/intel-iommu.c:753: warning: array subscript is above array bounds Signed-off-by: Takashi Iwai <tiwai@suse.de> Cc: Mark Gross <mgross@linux.intel.com> Acked-by: "Keshavamurthy, Anil S" <anil.s.keshavamurthy@intel.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-30x86 gart: rename iommu.h to gart.hJoerg Roedel
This patch renames the include file asm-x86/iommu.h to asm-x86/gart.h to make clear to which IOMMU implementation it belongs. The patch also adds "GART" to the Kconfig line. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Acked-by: Muli Ben-Yehuda <muli@il.ibm.com> Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2007-10-29intel-iommu fixesAl Viro
- off by one in dmar_get_fault_reason() (maximal index in array is ARRAY_SIZE()-1, not ARRAY_SIZE()) - NULL noise removal - __iomem annotation fix Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-23intel-iommu: fix sg_page()FUJITA Tomonori
Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp> Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
2007-10-22intel-iommu sg chaining supportFUJITA Tomonori
x86_64 defines ARCH_HAS_SG_CHAIN. So if IOMMU implementations don't support sg chaining, we will get data corruption. Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp> Acked-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com> Cc: Jens Axboe <jens.axboe@oracle.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-22intel-iommu: fix for IOMMU early crashKeshavamurthy, Anil S
pci_dev's->sysdata is highly overloaded and currently IOMMU is broken due to IOMMU code depending on this field. This patch introduces new field in pci_dev's dev.archdata struct to hold IOMMU specific per device IOMMU private data. Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com> Acked-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Greg KH <greg@kroah.com> Cc: Jeff Garzik <jeff@garzik.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-22intel-iommu: optimize sg map/unmap callsKeshavamurthy, Anil S
This patch adds PageSelectiveInvalidation support replacing existing DomainSelectiveInvalidation for intel_{map/unmap}_sg() calls and also enables to mapping one big contiguous DMA virtual address which is mapped to discontiguous physical address for SG map/unmap calls. "Doamin selective invalidations" wipes out the IOMMU address translation cache based on domain ID where as "Page selective invalidations" wipes out the IOMMU address translation cache for that address mask range which is more cache friendly when compared to Domain selective invalidations. Here is how it is done. 1) changes to iova.c alloc_iova() now takes a bool size_aligned argument, which when when set, returns the io virtual address that is naturally aligned to 2 ^ x, where x is the order of the size requested. Returning this io vitual address which is naturally aligned helps iommu to do the "page selective invalidations" which is IOMMU cache friendly over "domain selective invalidations". 2) Changes to driver/pci/intel-iommu.c Clean up intel_{map/unmap}_{single/sg} () calls so that s/g map/unamp calls is no more dependent on intel_{map/unmap}_single() intel_map_sg() now computes the total DMA virtual address required and allocates the size aligned total DMA virtual address and maps the discontiguous physical address to the allocated contiguous DMA virtual address. In the intel_unmap_sg() case since the DMA virtual address is contiguous and size_aligned, PageSelectiveInvalidation is used replacing earlier DomainSelectiveInvalidations. Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com> Cc: Greg KH <greg@kroah.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Suresh B <suresh.b.siddha@intel.com> Cc: Andi Kleen <ak@suse.de> Cc: Arjan van de Ven <arjan@infradead.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-22Intel IOMMU: Iommu floppy workaroundKeshavamurthy, Anil S
This config option (DMAR_FLPY_WA) sets up 1:1 mapping for the floppy device so that the floppy device which does not use DMA api's will continue to work. Once the floppy driver starts using DMA api's this config option can be turn off or this patch can be yanked out of kernel at that time. [akpm@linux-foundation.org: cleanups, rename things, build fix] [jengelh@computergmbh.de: Kconfig fixes] Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com> Cc: Andi Kleen <ak@suse.de> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Muli Ben-Yehuda <muli@il.ibm.com> Cc: "Siddha, Suresh B" <suresh.b.siddha@intel.com> Cc: Arjan van de Ven <arjan@infradead.org> Cc: Ashok Raj <ashok.raj@intel.com> Cc: "David S. Miller" <davem@davemloft.net> Cc: Christoph Lameter <clameter@sgi.com> Cc: Greg KH <greg@kroah.com> Signed-off-by: Jan Engelhardt <jengelh@gmx.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-22Intel IOMMU: Iommu Gfx workaroundKeshavamurthy, Anil S
When we fix all the opensource gfx drivers to use the DMA api's, at that time we can yank this config options out. [jengelh@computergmbh.de: Kconfig fixes] Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com> Cc: Andi Kleen <ak@suse.de> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Muli Ben-Yehuda <muli@il.ibm.com> Cc: "Siddha, Suresh B" <suresh.b.siddha@intel.com> Cc: Arjan van de Ven <arjan@infradead.org> Cc: Ashok Raj <ashok.raj@intel.com> Cc: "David S. Miller" <davem@davemloft.net> Cc: Christoph Lameter <clameter@sgi.com> Cc: Greg KH <greg@kroah.com> Signed-off-by: Jan Engelhardt <jengelh@gmx.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-22Intel IOMMU: DMAR fault handling supportKeshavamurthy, Anil S
MSI interrupt handler registrations and fault handling support for Intel-IOMMU hadrware. This patch enables the MSI interrupts for the DMA remapping units and in the interrupt handler read the fault cause and outputs the same on to the console. Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com> Cc: Andi Kleen <ak@suse.de> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Muli Ben-Yehuda <muli@il.ibm.com> Cc: "Siddha, Suresh B" <suresh.b.siddha@intel.com> Cc: Arjan van de Ven <arjan@infradead.org> Cc: Ashok Raj <ashok.raj@intel.com> Cc: "David S. Miller" <davem@davemloft.net> Cc: Christoph Lameter <clameter@sgi.com> Cc: Greg KH <greg@kroah.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-22Intel IOMMU: Intel iommu cmdline option - forcedacKeshavamurthy, Anil S
Introduce intel_iommu=forcedac commandline option. This option is helpful to verify the pci device capability of handling physical dma'able address greater than 4G. Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com> Cc: Andi Kleen <ak@suse.de> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Muli Ben-Yehuda <muli@il.ibm.com> Cc: "Siddha, Suresh B" <suresh.b.siddha@intel.com> Cc: Arjan van de Ven <arjan@infradead.org> Cc: Ashok Raj <ashok.raj@intel.com> Cc: "David S. Miller" <davem@davemloft.net> Cc: Christoph Lameter <clameter@sgi.com> Cc: Greg KH <greg@kroah.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-22Intel IOMMU: Avoid memory allocation failures in dma map api callsKeshavamurthy, Anil S
Intel IOMMU driver needs memory during DMA map calls to setup its internal page tables and for other data structures. As we all know that these DMA map calls are mostly called in the interrupt context or with the spinlock held by the upper level drivers(network/storage drivers), so in order to avoid any memory allocation failure due to low memory issues, this patch makes memory allocation by temporarily setting PF_MEMALLOC flags for the current task before making memory allocation calls. We evaluated mempools as a backup when kmem_cache_alloc() fails and found that mempools are really not useful here because 1) We don't know for sure how much to reserve in advance 2) And mempools are not useful for GFP_ATOMIC case (as we call memory alloc functions with GFP_ATOMIC) (akpm: point 2 is wrong...) With PF_MEMALLOC flag set in the current->flags, the VM subsystem avoids any watermark checks before allocating memory thus guarantee'ing the memory till the last free page. Further, looking at the code in mm/page_alloc.c in __alloc_pages() function, looks like this flag is useful only in the non-interrupt context. If we are in the interrupt context and memory allocation in IOMMU driver fails for some reason, then the DMA map api's will return failure and it is up to the higher level drivers to retry. Suppose, if upper level driver programs the controller with the buggy DMA virtual address, the IOMMU will block that DMA transaction when that happens thus preventing any corruption to main memory. So far in our test scenario, we were unable to create any memory allocation failure inside dma map api calls. Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com> Cc: Andi Kleen <ak@suse.de> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Muli Ben-Yehuda <muli@il.ibm.com> Cc: "Siddha, Suresh B" <suresh.b.siddha@intel.com> Cc: Arjan van de Ven <arjan@infradead.org> Cc: Ashok Raj <ashok.raj@intel.com> Cc: "David S. Miller" <davem@davemloft.net> Cc: Christoph Lameter <clameter@sgi.com> Cc: Greg KH <greg@kroah.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-10-22Intel IOMMU: Intel IOMMU driverKeshavamurthy, Anil S
Actual intel IOMMU driver. Hardware spec can be found at: http://www.intel.com/technology/virtualization This driver sets X86_64 'dma_ops', so hook into standard DMA APIs. In this way, PCI driver will get virtual DMA address. This change is transparent to PCI drivers. [akpm@linux-foundation.org: remove unneeded cast] [akpm@linux-foundation.org: build fix] [bunk@stusta.de: fix duplicate CONFIG_DMAR Makefile line] Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com> Cc: Andi Kleen <ak@suse.de> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Muli Ben-Yehuda <muli@il.ibm.com> Cc: "Siddha, Suresh B" <suresh.b.siddha@intel.com> Cc: Arjan van de Ven <arjan@infradead.org> Cc: Ashok Raj <ashok.raj@intel.com> Cc: "David S. Miller" <davem@davemloft.net> Cc: Christoph Lameter <clameter@sgi.com> Cc: Greg KH <greg@kroah.com> Signed-off-by: Adrian Bunk <bunk@stusta.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>