ARM SMMU versions 1 and 2

From OSDev.wiki
Jump to navigation Jump to search

The ARM SMMU specification outlines an IOMMU architecture for ARM processors, similar to Intel's VT-d. The SMMU mediates accesses to RAM by devices, and can translate addresses generated by DMA engines so as to correct them as the OS requires.

This page is current with versions 1 and 2 of the SMMU specification. This page does not serve as an exhaustive description of the SMMU. It is meant to serve as a refresher to jog the memory of people who have already read the SMMU specification.

A common 's' in a register name, as in SMMU_sCR0 implies that versions of this register exist for both secure and non-secure mode and that both versions are being talked about in that instance.

Terminology:

  • Just in general, the SMMU spec refers to DMA engines and MCUs on the chipset as being "upstream" from the SMMU and it refers to RAM as being "downstream" from the SMMU.
  • The "Stream mapping table" is a term used to refer to the combined working of the SMMU_SMRn + SMMU_S2CRn registers. Do not confuse the SMMU_SMRn ("Stream matching registers") with the umbrella term "Stream mapping registers". ARM defines Stream mapping as (Section 2.3.2) "the process of mapping a StreamID to the associated Stream-to-Context register, SMMU_S2CRn".
  • "Contexts" are synonymous with Intel's "domain" terminology used in the VT-d specification; they refer to a set of IO-page tables for translating IO-Addresses into output addresses on the bus/memory controller.
  • Transactions: Any write to or read from the SMMU by any agent is an SMMU transaction.
    • Client transactions: Client transactions are requests sent through the SMMU which can be potentially translated. These are generally bus messages.
    • Configuration transactions: Configuration transactions are accesses to the SMMU register interface to program the SMMU's behaviour (using the SMMU_* registers).
  • StreamIDs: These are used to identify the source of a transaction. They identify a device uniquely, or otherwise a set of devices working together deliberately to appear to be the same device.
    • A single device may also pretend to be multiple devices by using multiple StreamIDs.
    • An example of a StreamID might be a PCI Bus+Slot+Function combination.

Usage model

Contrast against Intel VT-d

Both SMMU and VT-d support 1st-stage translation. 1st-stage translation can be used:

  • To isolate devices and ensure that they don't access memory beyond what they are expected to (i.e, beyond the scatter gather list that has currently been programmed into the device).

VT-d 1st-stage translation can be used:

  • To isolate guest VMs and ensure that they don't touch memory outside of what they have been allocated as their guest physical address space (IPA addrspace).

Only SMMU supports 2nd-stage translation. 2nd stage translation can be used:

  • When passing a device through to a guest VM, to translate the IPA addresses that the guest will program into the device, into their correct PAs.

Only VT-d supports PASID 2nd-stage translation. PASID translation can be used:

  • To enable a device to directly walk the page tables of userspace processes, and translate virtual addresses into physical addresses.
    • For example, the virtual address of a graphics texture buffer can be sent to the GFX card directly, and the VT-d DHRU will be able to walk the page tables of the sending process and translate the virtual address on its own.
  • PASID translation is then chained into the standard translation flow to isolate the device's address accesses in the standard way that an IOMMU is normally used.

VT-d only supports PCI devices in its current form (at the time of writing) while SMMU supports multiple buses but leaves it up to the OEM implementation to determine how to map a bus technology's device IDs onto the identifier nomenclature ("StreamIDs") employed by the SMMU specification. For example, for the SMMU, an OEM is likely to map the 16-bit PCI Bus+Slot+Function onto the 15/16-bit StreamID space provided by the SMMU.

Secure/Non-secure mode split and SSD

The SMMU splits its registers for use between secure and non-secure mode; both modes can use the SMMU at the same time and there is register banking for some of the registers to facilitate this. Because both modes can use the SMMU at the same time, the SMMU also needs to know which devices have been partitioned into secure-mode devices and which are left for non-secure mode. In particular, secure mode devices can generate client transactions which access both secure memory and non-secure memory whereas non-secure mode devices can only access non-secure memory.

There is a distinction to be kept in mind about the difference between the SMMU's SSD feature (Security-State Determination) and the marking of a transaction by an upstream bus technology/device as either secure or non-secure. A non-secure device may maliciously mark its transactions as being secure, so the SSD feature which sits within the SMMU is meant to thwart that. When an upstream device is determined to be SSD-non-secure, the SMMU will never generate a downstream transaction which is marked secure. When an upstream device is determined to be SSD-secure, the SMMU may generate a downstream transaction which is either secure or non-secure.

  • The SMMU may not have a programmable SSD register space, in which case the SSD determination is done via some [implementation defined] mechanism attached to the SMMU.
  • The SMMU SSD register space is present if SMMU_IDR1.SSDTP == 0b01 or 0b11.
  • When this field has either of those values, the SSD space is present in the SMMU.
    • When the SSD space is present in the SMMU, the SMMU_SSDRn registers are present.
    • These SMMU_SSDRn registers are bitfields, 32 bits each. If bit<N> is 1 then device<N> is non-secure, else it is secure.

The Stream Mapping table is global and shared between Secure and non-secure mode. Secure mode must decide how many Stream Mapping register groups it will expose for non-secure mode's use. ARM recommends that secure mode software leaves at least one Stream mapping register group for use by non-secure software.

Having an SMMU_S2CR which has been allocated to secure be configured to mode point to a context bank which is non-secure is UNPREDICTABLE. Having an SMMU_S2CR which has been allocated to non-secure mode be configured to point to a context bank which is secure will trigger an Unimplemented Context Bank SMMU Fault. 'This proviso doesn't make much sense unless SMMU_CBn_* registers are mode-specific'.

Translation, bypass and faulting

The SMMU will do one of 3 things with any client transaction that is sent to it:

  • Translate the transaction using translation tables.
  • Allow the transaction to bypass translation, and just modify the attributes of the transaction (read/write, shareability, caching, etc).
  • Generate an SMMU fault.

Setup/Initialization related information

SSD knowledge

The SSD register space, if available, tells the OS which devices are secure vs non-secure. For a secure-mode OS, it allows the OS to set which StreamIDs are secure/non-secure. A Secure-mode OS should set these registers up. A non-secure mode OS may choose to read these registers to determine which devices are under its purvue.

StreamIDs

The number of supported streamID bits is determined by reading SMMU_IDR0.NUMSIDB[3:0].

  • Ranges from 0-15.
  • Can be 0 in the case of an SMMU which is dedicated to handling transactions from a single upstream StreamID.

For SMMUv2:

  • Support for 16-bit streamIDs exists if SMMU_IDR0.EXIDS==1 AND SMMU_IDR0.NUMSIDB==15.
  • 16-bit Stream IDs are enabled by setting SMMU_sCR0.EXIDENABLE==1.

Furthermore, if the OS intends to use Stream Indexing instead of Stream Matching, then the number of StreamIDs is further constrained by the number of SMMU_S2CRn registers available. The number of SMMU_S2CRn registers is directly determined by the number of SMMU_SMRn registers (Section 2.3.2, "Stream indexing": This means the maximum StreamID is determined by the number of implemented SMMU_S2CRns). See #Stream Matching Registers for how to determine the number of SMMU_S2CRn registers by inferring it from the number of SMMU_SMRn regs.

The OS should determine the max number of StreamIDs based on the Stream Mapping technique it intends to use, and set up metadata accordingly.

Stream Matching Registers

SMMUv1 supports up to 128 SMMU_SMRs. SMMUv2 may optionally support the Extended Stream Matching Extension which provides up to 1024 SMMU_SMRs.

The number of SMMU_SMRn regs determines the number of SMMU_S2CRn regs (Section 2.3.2, "Stream matching": the optional Extended Stream Matching Extension, that provides up to 1024 Stream Match Registers and associated Stream-to-Context registers).

  • Support for Stream Matching is indicated via SMMU_IDR0.SMS == 1.
  • Support for Extended Stream Matching is indicated via SMMU_IDR0.EXSMRGS == 1.
  • Support for compressed Indexing (which is a subfeature under Stream Matching) is indicated via a secure read of SMMU_IDR2.COMPINDEXS == 1. A non-secure read of this bit only states whether or not compressed indexing is available for use by non-secure software.
  • The number of Stream Mapping Register Groups is determined by reading SMMU_IDR0.NUMSMRG.
  • It is possible for an implementation not to support Stream Matching, but still allow software to reference the Stream Mapping Table (which is the combination of the SMMU_SMRn and SMMU_S2CRn register sets).
    • In such a case, the SMMU_SMRn register group will be RES0.

ARM found it important to mention that if the SMMU support Extended Stream Matching and the extension is not enabled, the (up to) 128 SMMU_SMRs that are exposed, map onto the final 128 SMMU_SMRs provided by the extension. Since it's apparently important, I'll mention it too.

Number of supported Contexts

The max number of supported contexts is implied by the max number of SMMU_S2CRn registers (Section 2.3.4: "The number of SMMU_SBARn regs matches the number of entries in the translation context bank table, and is IMPLEMENTATION DEFINED. SMMU_IDR1.NUMCB specifies the number of implemented SMMU_CBARn regs.")

  • Multiple SMMU_SMRn regs may cause multiple StreamIDs to map to the same Context when Stream Matching is used.
  • Each incoming StreamID maps directly to a specific Context when Stream Indexing is used.

Support for stage1 and stage2

  • Support for stage1 translation is indicated via SMMU_IDR0.S1TS.
  • Support for stage2 translation is indicated via SMMU_IDR0.S2TS.
  • Support for stage1 followed by stage2 translation is indicated via SMMU_IDR0.NTS.
    • If SMMU_IDR0.NTS is set, then both SMMU_IDR0.S1TS and SMMU_IDR0.S2TS must be set.

Stream Mapping Table

Section 2.3.10: "On reset, no initialization is performed on the Stream mapping table entries.". Ergo the OS must explicitly initialize all the stream mapping reg groups "before the SMMU is enabled" [sic].

Client transaction flowchart

Image

Synopsis:

Do not allow a single StreamID to map to multiple contexts. Multiple stream IDs can however, map to a single context.

  • Client transactions are enabled with the "Client Port Disable" bit: SMMU_sCR0.CLIENTPD==0.
  • If SMMU_sCR0.CLIENTPD==1, then bypass is assumed for all incoming transactions.
    • Bypass attribute transformation is done based on the values in SMMU_sCR0.
  • If SMMU_sCR0.CLIENTPD==0, client transaction processing is enabled, and:
    • Stream Mapping table is searched.
      • Stream Mapping table search is done using either Stream Matching, Compressed Stream Indexing or Stream Indexing.
    • If Stream Mapping search yields no results then either:
      • [if SMMU_sCR0.USFCFG==1 -- USFCFG means "Unidentified Stream Fault Config"] An Unidentified Stream SMMU fault is generated.
      • [if SMMU_sCR0.USFCFG==0] The unidentifiable stream ID is handled by bypassing it and applying the attribute transformation specified in SMMU_sCR0.
    • If Stream mapping table search yields multiple results:
      • [if SMMU_sCR0.SMCFCFG==1 -- SMCFCFG means "Stream Match Conflict Fault Config"] A Stream Match Conflict SMMU Fault is generated.
      • [if SMMU_sCR0.SMCFCFG==0 ] The conflict is handled via bypass and applying the attribute transformation specified in SMMU_sCR0 (just like the Unidentified Stream case).
    • If Stream Mapping table search yields a single result ('This is the desired normal case').
      • The Stream Mapping table lookup will identify one of the 128 SMMU_S2CRn Stream to Context Registers.
      • The S2C register will determine which of the 3 handling mechanisms will be used to deal with the incoming client transaction (translation, bypass or fault).
        • [if SMMU_S2CRn.Type == Fault] An Invalid Context SMMU fault is generated.
        • [if SMMU_S2CRn.Type == bypass] The client transaction is handled via bypass and the attributes specified in SMMU_S2CRn are applied to the transaction.
        • [if SMMU_S2CRn.Type == Translation] The client transaction is handled by translation and:
          • The attributes in SMMU_CBARn (stage 1 Context Bank Attribute Reg) and SMMU_SBA2Rn (stage 2 Context Bank Attribute Reg) are 'optionally' applied to the output transaction which is generated downstream.
          • [if SMMU_CBn_SCTLR.M==1 for the particular stage in question] then translation is done for that stage using the configuration in SMMU_CBARn.

Flow for Translation-type scenarios

Stream Mapping

As stated before on this page, Stream Mapping is defined by ARM as "'the process of mapping a StreamID to the associated Stream-to-Context register'".

When a client transaction comes in from upstream, if client transaction processing is enabled (SMMU_sCR0.CLIENTPD==0), the SMMU must decide what SMMU context to use to translate the transaction into the output transaction which will eventually be emitted downstream by the SMMU. The process of determining which Context bank should be used to handle client transactions sent by a particular upstream StreamID is an "associative search" [sic] which uses one of 3 methods: Stream Matching, Compressed Stream Indexing or Stream Indexing.

The Stream Mapping table is global and shared between Secure and non-secure mode. Secure mode must decide how many Stream Mapping register groups it will expose for non-secure mode's use. ARM recommends that secure mode software leaves at least one Stream mapping register group for use by non-secure software. To set the number of Stream mapping regs exposed to non-secure mode, use SMMU_SCR1.NSNUMSMRGO. The SMMU will intelligently deduce the number of SMR regs to tell non-secure mode about, and will report the appropriate number to a non-secure reader through SMMU_IDR0.NUMSMRG.

Retrieved from "https://osdev.wiki/wiki/ARM_SMMU_versions_1_and_2?oldid=3809"