| Ticket Name: TDA2EVM5777: Accessing 4GiB DDR Range Starteware | |
| Query Text: | |
| Part Number: TDA2EVM5777 Other Parts Discussed in Thread: TDA2, SYSBIOS We are using the TDA2xx chip with Vision SDK 3.03 software. Through starterware software we need to access 4GiB of DDR space from the A15 cores and max 2GiB space for DSP cores and 1.5GiB for IPU cores. Our current MLO is configured to access only 1GiB of space. Script used to create current MLO is: gmake -s sbl BOOTMODE=sd CORE=a15_0 SBL_BUILD_MODE=prod SBL_OPT_MODE=high EMIFMODE=SBLLIB_DUAL_EMIF_2X512MB OPPMODE=opp_nom I have two main questions with some addditional secondary questions: Question 1: How do we build an MLO that enables 4GiB? Question 2: Once we do get 4GiB enabled, how do we ensure A15 source code (using library starterware) will allow a15 to access beyond the 2GiB? Thank you, More details and secondary questions below: The above MLO gmake script configures the DMM and MA_MPU LISA_MAP_0 and LISA_MAP_1 registers with 0x80640300U. By updating that value to 0x80740300U would all cores have full access to the first 2GiB of DDR space in interleaved mode or is there anything else that would be necessary? For the A15 to access the additional 2GiB space the A15 core needs to use a 40b address. Under Linux that requires enabling the ARM LPAE support. How does that translate to non-linux starterware software? How would the A15 cores access all 4GiB range and IPU/DSP cores access the 2GiB range in the same application app? | |
| Responses: | |
| Hi, Our expert for this is not in office this week. Please expect delay in response, we will get back to you by early next week. Regards, Ankur | |
| Hello Ankur, Thank you for your reply, looking forward to hearing back when the expert returns. We made some progress in the MLO (Question 1) and continuing to learn more about the A15 memory access (Question 2). Here are additional details/questions, in addition to our previous post. Thank you for your help! For the MLO we followed the example of what Linux does and configured MA_MPU_LISA_MAP_0 = 0x0 MA_MPU_LISA_MAP_1 = 0x0 MA_MPU_LISA_MAP_2 = 0x80740300U MA_MPU_LISA_MAP_3 = 0xFF020100U DMM_LISA_MAP_0 = 0x80740300U DMM_LISA_MAP_1 = 0xFF020100U DMM_LISA_MAP_2 = 0x0 DMM_LISA_MAP_3 = 0x0 With this updated change, this does allow us to access beyond the 1GiB space up until the last 16MiB of the 2nd 1GiB range. (Due to the 0xFF020100U configuration that seems to be correct.) In addition to the previous two questions (Question 1 and 2 in previous post), below are new Questions Question 3: Is there a reason why we would need to have the 16MiB exclusion when not using Linux? Based on further reading, in order to access the extended 2GiB of physical memory the a15 needs to set TTBCR to enable LPAE? The provided MMUA15Init() function calls MMUA15InitASM() which seems to set the TBCR EAE bit to non-zero. MMUA15InitASM: mov r0, #1 @ TTBR0 used and desc uses Short format lsl r0, r0, #31 @ Set TTBCR.EAE bit mcr p15, #0, r0, c2, c0, #2 @ write r0 to TTBCR isb @ flush instruction pipeline @ isb makes sure cp15 changes @ are visible to all subsequent @ instructions bx r14 .endfunc Question 4: Does the MMUA15InitASM() routine therefore enable LPAE? Question 5: If yes, then do we only need to create First Level Descriptor Tables that point to the extended 2GiB physical addresses? And if yes, to access all of the 4GiB space from an A15 what would be the recommended configuration for the first level descriptors? Question 6: This is an Application level function, is there no additional requirement in the MLO for accessing the extended memory? Additionally we've seen that in Linux it appears that for LPAE mode a cache option is changed from (DCACHE_WRITEBACK & ~TTB_SECT_XN_MASK) to DCACHE_WRITEALLOC. Question 7: What is the difference in cache management when using LPAE? Thank you. | |
| Hi, You can use 3 GB DDR within 32 bit address range by using address space from 0x2000_0000 to 0x4000_0000 and 0x6000_0000 to 0x8000_0000 in addition to 2 GB from 0x8000_0000. To use 40 bit addresses you have to use linux, starterware doesn't support the same. Regards, Rishabh | |
| Hello Rishabh, Thank you for confirming the 3GiB limit in starterware. Is there a plan to eventually support 4GiB in starterware/pdk? Also, I have some follow-up questions on your suggested mapping of the two 512MiB ranges below. Thanks again for your help! How To Configure A15 Access of the 3GiB Would accessing the extended 1GiB be done through the A15 MMU and virtually mapping the suggested ranges to extended DDR space? If so, the MMU first level descriptors define a 1GiB block which won't work for the suggested mapping of two 512MiB ranges… So does that mean the second level descriptors are the only way to map the two 512MiB blocks? Are there examples that do this type of mapping? When to Configure Access When does the MMU need to be configured If we wish to initialize global variables in the extended DDR space? Since the MLO loads the app would the MMU init need to be inside the MLO before the RPRCs are loaded? Thank you | |
| Hi, There are no plans to support 40 bit addresses in PDK and hence 4 GB DDR cannot be supported. Yes second level descriptor is the way to map 512 MB. Please note that you need to have 512 entries of 2 MB each in second level table. Unfortunately we don't have an example which uses 2nd level descriptor, you need to refer to header file for the same. From MLO perspective you don't need to add MMU configuration. MLO checks the App Image for valid DDR addresses which you will need to update, current valid address is >=0x8000_0000. Regards, Rishabh | |
| Hi, I haven't heard back from you, I'm assuming you were able to resolve your issue. If not, just post a reply below (or create a new thread if the thread has locked due to time-out). Regards, Rishabh | |
| Hello Rishabh, Another task at work took my attention last week. Thank you for the above replies they have helped better understand how to access the additional 1GiB of space. After reviewing the 2nd level descriptor functions they seemed very similar to 1st level and I am programming them with very similar methods. I have kept the same mmuA15DescriptorAttrs configurations in 2nd level descriptors as 1st level and are now trying to correctly map the suggested two 512MiB ranges to their correct physical addresses. I checked the MA_PRIORITY register to confirm the value of HIMEM_INTERLEAVE_UN and I saw it was set to 0. Under this configuration would the extended 1GiB physical addresses be: EMIF1_SDRAM_CS0 = 0x02_0000_0000 0x02_3FFF_FFFF EMIF2_SDRAM_CS0 = 0x03_C000_0000 0x03_FFFF_FFFF The above is for the full extra 2GiB so are any two 512MiB sections in that range good to map to or are there any restrictions? We'll want to use interleaving for this extended 1GiB space. Is enabling HIMEM_INTERLEAVE_UN the only requirement or are there any other configuration registers for the extended DDR? Once interleaving is enabled, would the physical address range to map to be 0x02_0000_0000 0x02_3FFF_FFFF? Thank you. | |
| Hi, The idea is to have two 512 MB blocks. First block will be 0x2000_0000 to 0x3FFF_FFFF and second block is 0x6000_0000 to 0x7FFF_FFFF. Then you configure MA_LISA_MAP registers for A15 access path to these areas. My suggestion is not to have EMIF interleaving enabled at first. Once you have the system working using single EMIF then you can go for EMIF interleaving. Regards, Rishabh | |
| Hello Rishabh, Thank you again for your reply and suggestions. I configured a LISA_MAP register with the goal of mapping the first 512MiB block using the register value: 0x20500140. However any attempt to read any value from 0x2000.0000 - 0x3FFF.FFFF results in a system halt. I've tried this with and without enabling the MMU with the second level descriptors. I'm still confirming I'm programming the second level descriptor correctly, below is an example of 1 out of 256 entries I created for the first 512MiB block. MMUA15InitDescAttrs(&gAttrs); gAttrs.descriptorType = MMU_A15_DESCRIPTOR_TYPE_BLOCK; gAttrs.attrIndx = MMU_A15_ATTR_INDEX_2; gAttrs.nonSecure = MMU_A15_NON_SECURE_ENABLE; gAttrs.accPerm = MMU_A15_ACC_PERM_RW_ANY_PL; MMUA15SetSecondLevelDesc(&gMmuTable, 0x20000000, 0x20000000 , &gAttrs); I used the case examples from the TDA2 TRM under the section "Addressing Management with LISA" to help derive the LISA register value. The second level descriptor header looked very similar to first level so I'm programming them with similar approach and since the function returned STW_SOK that seemed OK. Is there anything incorrect that you can see or any suggestions for how to debug the halt? Thank you | |
| Hi, Sorry for the delayed response. Can you please see below suggestions: 1. EMIF 1 is MMU mapped to 0x8000_0000 to 0xFFFF_FFFF (2 GB), 2. EMIF 2 DDR is MMU mapped to 0x0000_0000 to 0x3FFF_FFFF (1 GB) – I have used this as you are currently not using GPMC or PCIE in your system. 3. All the DDR regions have been marked cacheable. (This can be modified based on the regions of DDR which are non-cached as well.) /*********************************************** | |
| * MMU Configuration * | |
| ***********************************************/ | |
| var Mmu = xdc.useModule('ti.sysbios.family.arm.a15.Mmu'); | |
| Mmu.enableMMU = true; | |
| // descriptor attribute structure | |
| var tab_desc = new Mmu.DescriptorAttrs(); | |
| Mmu.initDescAttrsMeta(tab_desc); | |
| tab_desc.type = Mmu.DescriptorType_TABLE; // Table descriptor | |
| Mmu.setFirstLevelDescMeta(0x00000000, 0x2C0000000, tab_desc); | |
| Mmu.setFirstLevelDescMeta(0x40000000, 0x2C0000000, tab_desc); | |
| Mmu.setFirstLevelDescMeta(0x80000000, 0x2C0000000, tab_desc); | |
| Mmu.setFirstLevelDescMeta(0xC0000000, 0x2C0000000, tab_desc); | |
| /****************************************************************************** | |
| * | |
| * SYS/BIOS assigns the following defaults to MAIR0 ATTR0, ATTR1 and ATTR2: | |
| * | |
| * ATTR0 -> 0x44 (mark memory region as non-cacheable normal memory) | |
| * ATTR1 -> 0x00 (mark memory region as device memory, i.e. strongly | |
| * ordered and non-cacheable) | |
| * ATTR2 -> 0xFF (mark memory region as normal memory, RW cacheable and | |
| * RW allocate) | |
| ****************************************************************************** | |
| */ | |
| // descriptor attribute structure | |
| var ddr_attr = new Mmu.DescriptorAttrs(); | |
| Mmu.initDescAttrsMeta(ddr_attr); | |
| ddr_attr.type = Mmu.DescriptorType_BLOCK; // BLOCK descriptor | |
| ddr_attr.shareable = 2; // sharerable | |
| ddr_attr.attrIndx = 2; // Cached, normal memory | |
| // Set the descriptor for each entry in the address range - EMIF 2 (1 GB) | |
| for (var i=0x00000000; i < 0x40000000; i = i + 0x00200000) { | |
| // Each 'BLOCK' descriptor entry spans a 2MB address range | |
| Mmu.setSecondLevelDescMeta(i, i + 0x300000000, ddr_attr); | |
| } | |
| // Set the descriptor for each entry in the address range - EMIF 1 (1 GB) | |
| for (var i=0x80000000; i < 0xC0000000; i = i + 0x00200000) { | |
| // Each 'BLOCK' descriptor entry spans a 2MB address range | |
| Mmu.setSecondLevelDescMeta(i, i, ddr_attr); | |
| } | |
| // Set the descriptor for each entry in the address range - EMIF 1 (1 GB) | |
| for (var i=0xC0000000; i < 0x100000000; i = i + 0x00200000) { | |
| // Each 'BLOCK' descriptor entry spans a 2MB address range | |
| Mmu.setSecondLevelDescMeta(i, i, ddr_attr); | |
| } | |
| // descriptor attribute structure | |
| var attrs0 = new Mmu.DescriptorAttrs(); | |
| Mmu.initDescAttrsMeta(attrs0); | |
| attrs0.type = Mmu.DescriptorType_BLOCK; // BLOCK descriptor | |
| attrs0.shareable = 2; // sharerable | |
| attrs0.attrIndx = 2; // Cached, normal memory | |
| // Set the descriptor for each entry in the address range | |
| // Set OCMC memories as shareable and cacheable in case Board package is used | |
| for (var i=0x40000000; i < 0x40600000; i = i + 0x00200000) { | |
| // Each 'BLOCK' descriptor entry spans a 2MB address range | |
| Mmu.setSecondLevelDescMeta(i, i, attrs0); | |
| } | |
| attrs0.type = Mmu.DescriptorType_BLOCK; // BLOCK descriptor | |
| attrs0.noExecute = true; // not executable | |
| attrs0.accPerm = 0; // read/write at PL1 | |
| attrs0.shareable = 2; // shareable | |
| attrs0.attrIndx = 1; // strongly ordered and non-cacheable | |
| // Set the descriptor for each entry in the address range | |
| // Remaining OCMC region configured as non-cacheable, device memory. | |
| for (var i=0x40600000; i < 0x60000000; i = i + 0x00200000) { | |
| // Each 'BLOCK' descriptor entry spans a 2MB address range | |
| Mmu.setSecondLevelDescMeta(i, i, attrs0); | |
| } | |
| Regards, Rishabh | |
| Hello Rishabh Thank you so much for the detailed suggestions. I translated the sys bios functions as best I could to use the PDK functions. After updating the LISA_MAP registers with what you suggest (2GiB on Emif 1, and 1 GiB on EMIF2) I get a full system halt as soon as I enable the MMU. Always at MMUA15Enable() I get a halt. I've started pulling up sys bios documentation to better understand your suggestion but I also have some followup questions. 1. I tried to match the configurations for the descriptor attributes but I'm not familiar with sys bios and couldn't find information on values that get set as part of Mmu.initDescAttrsMeta. I used MMUA15InitDescAttrs() and then updated any specific attributes like you manually did. Is what I'm using below correct? 2. For the EMIF2 configuration I see you're mapping to physical addresses starting at 0x3_0000_0000 but I wasn't sure why. The TRM I have lists that address as RESERVED. Can you explain why use that address? 3. BTW based on you using the full lower 1GiB address range for Emif 2 our max DDR range should be 3.5 GiB if we also use the initially suggested 512MiB range starting at 0x6000_0000 correct? /*********************************************************** ***********************************************************/ MMUA15InitDescAttrs(&table_gAttrs); table_gAttrs.descriptorType = MMU_A15_DESCRIPTOR_TYPE_TABLE; // Set level one descriptor virtualAddr = 0x00000000, phyAddr = 0xc0000000; MMUA15SetFirstLevelDesc(&gMmuTable, virtualAddr, phyAddr, &table_gAttrs); virtualAddr = 0x40000000, phyAddr = 0xc0000000; MMUA15SetFirstLevelDesc(&gMmuTable, virtualAddr, 0xc0000000, &table_gAttrs); virtualAddr = 0x80000000, phyAddr = 0xc0000000; MMUA15SetFirstLevelDesc(&gMmuTable, virtualAddr, phyAddr, &table_gAttrs); virtualAddr = 0xc0000000, phyAddr = 0xc0000000; MMUA15SetFirstLevelDesc(&gMmuTable, virtualAddr, 0xc0000000, &table_gAttrs); cacheEnabled = CACHEA15GetEnabled(); // In case cache is disabled, invalidate and enable it if (CACHE_A15_TYPE_ALL != cacheEnabled) { CACHEA15InvalidateL1DAll(); CACHEA15InvalidateL1IAll(); CACHEA15Enable(CACHE_A15_TYPE_L1); } // Initialize MMU module MMUA15Init(); // Set MAIR to inner and outer cacheable MMUA15SetMAIR(&gMmuTable, MMU_A15_ATTR_INDEX_2, 0xFF); // Initialize descriptor attributes MMUA15InitDescAttrs(&gAttrs); // Set level one descriptor attributes gAttrs.descriptorType = MMU_A15_DESCRIPTOR_TYPE_BLOCK; gAttrs.attrIndx = MMU_A15_ATTR_INDEX_2; gAttrs.shareable = 2; ///MMU_A15_SHAREABLE_OUTER gAttrs.nonSecure = MMU_A15_NON_SECURE_ENABLE; gAttrs.accPerm = MMU_A15_ACC_PERM_RW_ANY_PL; // Set the descriptor for each entry in the address range - EMIF 2 (1 GB) for (t=0x00000000; t < 0x40000000; t = t + 0x00200000) { // Each 'BLOCK' descriptor entry spans a 2MB address range MMUA15SetSecondLevelDesc(&gMmuTable, t, t + 0x300000000, &gAttrs); } // Set the descriptor for each entry in the address range - EMIF 1 (1 GB) for (t=0x80000000; t < 0xC0000000; t = t + 0x00200000) { // Each 'BLOCK' descriptor entry spans a 2MB address range MMUA15SetSecondLevelDesc(&gMmuTable, t, t, &gAttrs); } // Set the descriptor for each entry in the address range - EMIF 1 (1 GB) for (t=0xC0000000; t < 0x100000000; t = t + 0x00200000) { // Each 'BLOCK' descriptor entry spans a 2MB address range MMUA15SetSecondLevelDesc(&gMmuTable, t, t, &gAttrs); } // Set OCMC memories as shareable and cacheable in case Board package is used for (t=0x40000000; t < 0x40600000; t = t + 0x00200000) { // Each 'BLOCK' descriptor entry spans a 2MB address range MMUA15SetSecondLevelDesc(&gMmuTable, t, t, &gAttrs); } MMUA15SetMAIR(&gMmuTable, MMU_A15_ATTR_INDEX_1, 0x00); gAttrs.attrIndx = MMU_A15_ATTR_INDEX_1; gAttrs.accPerm = MMU_A15_ACC_PERM_RW_PL1; gAttrs.shareable = 2; ///MMU_A15_SHAREABLE_OUTER gAttrs.noExecute = 1; ///Enable // Remaining OCMC region configured as non-cacheable, device memory. for (t=0x40600000; t < 0x60000000; t = t + 0x00200000) { // Each 'BLOCK' descriptor entry spans a 2MB address range MMUA15SetSecondLevelDesc(&gMmuTable, t, t, &gAttrs); } MMUA15Enable(&gMmuTable); Thank you | |
| Hi, BIOS and PDK both have very similar implementation. MMUA15InitDescAttrs() usage is correct. Also the configuration I have shared was validated for 3 GB usage some time back. I would suggest you to directly use this instead of trying to map 0x6000_0000. Can you first validate this using SYSBIOS and then try to translate it to PDK for quicker progress. Regards, Rishabh | |
| Hello Rishabh, I completely agree and was using your suggestion directly with only 3GB. The question about 3.5GB was just for determining max potential for planning purposes. I'm not at all familiar with sys bios including its build process so I'm not sure how quick that may actually be. Using linux we've already validated the hardware so ultimately we just need to have the 3GB work in PDK. Since the system halts only when enabling the MMU it seemed the issue was MMU configuration but I thought I implemented the pdk version same as the posted sysbios version. Based on the software I provided is there any mistakes you can identify? Thank you | |
| Hi, Sorry for the delayed response. I did not find any obvious issue with your configuration. Are you familiar with Processor SDK Vision? If yes then you can directly try the cfg I shared. Regards, Rishabh | |