arm-software / bsa-acs

Arm SystemReady : BSA Architecture Compliance Suite

Base System Architecture - Architecture Compliance Suite

Base System Architecture

Base System Architecture (BSA) specification describes a hardware system architecture based on the Arm 64-bit architecture. System software such as operating systems, hypervisors, and firmware rely on this. It addresses PE features and key aspects of system architecture.

For more information, download the BSA specification

BSA - Architecture Compliance Suite

BSA Architecture Compliance Suite (ACS) is a collection of self-checking, portable C-based tests. This suite includes a set of examples of the invariant behaviors that are provided by the BSA specification, so that you can verify if these behaviour have been interpreted correctly.

Most of the tests are executed from UEFI (Unified Extensible Firmware Interface) Shell by executing the BSA UEFI shell application. A few tests are executed by running the BSA ACS Linux application which in turn depends on the BSA ACS Linux kernel module. The tests can also be executed in a Bare-metal environment. The initialization of the Bare-metal environment is specific to the environment and is out of scope of this document.

Release details

• Code quality: v1.0.8
• The tests are written for version 1.0 (c) of the BSA specification.
• The tests can be run at both the Pre-Silicon and Silicon level.
• For complete coverage of the BSA rules, availability of an Exerciser is required for Exerciser tests to be run during verficiation at Pre-Silicon level.
• The compliance suite is not a substitute for design verification.
• To review the BSA ACS logs, Arm licensees can contact Arm directly through their partner managers.
• To know about the BSA rules not implemented in this release, see the Test Scenario Document.

GitHub branch

• To pick up the release version of the code, checkout the corresponding tag from the main branch.
• To get the latest version of the code with bug fixes and new features, use the main branch.

Additional reading

• For information about the implementable BSA rules test algorithm and for unimplemented BSA rules, see the arm BSA Test Scenario Document.
• For information on test category(UEFI, Linux, Bare-metal) and applicable systems(IR,ES,SR,Pre-Silicon), see the arm BSA Test Checklist.
• For details on the design of the BSA ACS, see the arm BSA Validation Methodology Document.
• For details on the BSA ACS UEFI Shell Application and Linux Application see the arm BSA ACS User Guide.
• For details on the BSA ACS Bare-metal support, see the
  • arm BSA ACS Bare-metal User Guide.
  • Bare-metal Code.
    Note: The Bare-metal PCIe enumeration code provided as part of the BSA ACS should be used and should not be replaced. This code is vital in analyzing of the test result.

Running Exerciser tests for complete coverage

Exerciser is a client device wrapped up by PCIe Endpoint. This device is created to meet BSA requirements for various PCIe capability validation tests. Running the Exerciser tests provides additional test coverage on the platform.

Note: To run the exerciser tests on a UEFI Based platform with Exerciser, the Exerciser PAL API's need to be implemented. For details on the reference Exerciser implementation and support, see the Exerciser.md and Exerciser_API_porting_guide.md

ACS build steps - UEFI Shell application

Prebuilt images

Prebuilt images for each release are available in the prebuilt_images folder of the main branch. You can choose to use these images or build your own image by following the steps below. If you choose to use the prebuilt image, see the Test suite execution section below for details on how to run the application.

1. Building from source

Before you start the ACS build, ensure that the following requirements are met.

• Any mainstream Linux-based OS distribution running on a x86 or AArch64 machine.
• git clone the EDK2 tree. Recommended edk2 tag is edk2-stable202402
• git clone the EDK2 port of libc to local <edk2_path>.
• Install GCC-ARM 13.2 toolchain.
• Install the build prerequisite packages to build EDK2.
  Note:
  
• The details of the packages are beyond the scope of this document.

1.1 Target Platform

To start the ACS build for platform using ACPI table, perform the following steps:

1. cd local_edk2_path
2. git submodule update --init --recursive
3. git clone https://github.com/ARM-software/bsa-acs.git ShellPkg/Application/bsa-acs
4. Add the following to the [LibraryClasses.common] section in ShellPkg/ShellPkg.dsc

     BsaValLib|ShellPkg/Application/bsa-acs/val/BsaValLib.inf
     BsaPalLib|ShellPkg/Application/bsa-acs/pal/uefi_acpi/BsaPalLib.inf

5. Add the following in the [components] section of ShellPkg/ShellPkg.dsc

     ShellPkg/Application/bsa-acs/uefi_app/BsaAcs.inf

To start the ACS build for platform using Device tree, perform the following steps:

1. cd local_edk2_path
2. git submodule update --init --recursive
3. git clone https://github.com/ARM-software/bsa-acs.git ShellPkg/Application/bsa-acs
4. Add the following to the [LibraryClasses.common] section in ShellPkg/ShellPkg.dsc

     FdtLib|EmbeddedPkg/Library/FdtLib/FdtLib.inf
     BsaValLib|ShellPkg/Application/bsa-acs/val/BsaValLib.inf
     BsaPalLib|ShellPkg/Application/bsa-acs/pal/uefi_dt/BsaPalLib.inf

5. Add the following in the [components] section of ShellPkg/ShellPkg.dsc

     ShellPkg/Application/bsa-acs/uefi_app/BsaAcs.inf

6. In IR systems, ACS efi application runs on top of efi shell which runs on u-boot as efi payload.

• Below change in edk2 code MdeModulePkg/Library/UefiHiiServicesLib/UefiHiiServicesLib.c is required before compiling for IR system.

     -Status = gBS->LocateProtocol (&gEfiHiiConfigRoutingProtocolGuid, NULL, (VOID **) &gHiiConfigRouting);
     -ASSERT_EFI_ERROR (Status);
     +//Status = gBS->LocateProtocol (&gEfiHiiConfigRoutingProtocolGuid, NULL, (VOID **) &gHiiConfigRouting);
     +//ASSERT_EFI_ERROR (Status);

1.2 Build environment

If the build environment is Linux, perform the following steps:

1. export GCC49_AARCH64_PREFIX= GCC13.2 toolchain path pointing to /bin/aarch64-linux-gnu- in case of x86 machine.
   For an AArch64 build it should point to /usr/bin/
2. export PACKAGES_PATH= path pointing to edk2-libc
3. source edksetup.sh
4. make -C BaseTools/Source/C
5. source ShellPkg/Application/bsa-acs/tools/scripts/acsbuild.sh

1.3 Build output

The EFI executable file is generated at <edk2_path>/Build/Shell/DEBUG_GCC49/AARCH64/Bsa.efi

2. Test suite execution

Prerequisites

• If the system supports LPIs (Interrupt ID > 8192) then Firmware should support installation of handler for LPI interrupts.
  • If you are using edk2, change the ArmGic driver in the ArmPkg to support installation of handler for LPIs.
  • Add the following in edk2/ArmPkg/Drivers/ArmGic/GicV3/ArmGicV3Dxe.c

   - After [#define ARM_GIC_DEFAULT_PRIORITY  0x80]
     +#define ARM_GIC_MAX_NUM_INTERRUPT 16384
   - Change this in GicV3DxeInitialize function.
     -mGicNumInterrupts      = ArmGicGetMaxNumInterrupts (mGicDistributorBase);
     +mGicNumInterrupts      = ARM_GIC_MAX_NUM_INTERRUPT;

The execution of the compliance suite varies depending on the test environment. The following steps assume that the test suite is invoked through the ACS UEFI shell application.

2.1 Silicon

On a system where a USB port is available and functional, perform the following steps:

For IR Systems:

1. Copy 'Bsa.efi' and 'Shell.efi' to a USB Flash drive.
2. Boot the system to U-Boot shell.
3. Plug in the USB flash drive to one of the functional USB ports on the system.
4. To determine the file system number of the plugged-in USB drive, execute command
   usb start
5. Copy the 'Shell.efi' to memory location using the command
   Syntax: fatload usb <dev_num> ${kernel_addr_r} Shell.efi
Eg: fatload usb 0 ${kernel_addr_r} Shell.efi
   This boots the system to UEFI Shell.
6. To determine the file system number of the plugged-in USB drive, execute 'map -r' command.
7. Type 'fs' where '' is replaced by the number determined in step 5.
8. To start the compliance tests, run the executable Bsa.efi with the appropriate parameters.
9. Copy the UART console output to a log file. Note: 'Shell.efi' is available in the pebuilt_images/IR

2.2 Emulation environment with secondary storage

On an emulation environment with secondary storage, perform the following steps:

1. Create an image file which contains the 'Bsa.efi' file. For example:

• mkfs.vfat -C -n HD0 hda.img 2097152
• sudo mount -o rw,loop=/dev/loop0,uid=`whoami`,gid=`whoami` hda.img /mnt/bsa In case loop0 is busy, please specify the one that is free.
• sudo cp "/Bsa.efi" /mnt/bsa/
• sudo umount /mnt/bsa

2. Load the image file to the secondary storage using a backdoor. The steps to load the image file are emulation environment-specific and beyond the scope of this document.
3. Boot the system to UEFI shell.
4. To determine the file system number of the secondary storage, execute 'map -r' command.
5. Type 'fs' where '' is replaced by the number determined in step 4.
6. To start the compliance tests, run the executable Bsa.efi with the appropriate parameters.
7. Copy the UART console output to a log file for analysis and certification.

• For information on the BSA uefi shell application parameters, see the User Guide.

2.3 Emulation environment without secondary storage

On an emulation platform where secondary storage is not available, perform the following steps:

1. Add the path to 'Bsa.efi' file in the UEFI FD file.
2. Build UEFI image including the UEFI Shell.
3. Boot the system to UEFI shell.
4. Run the executable 'Bsa.efi' to start the compliance tests. For details about the parameters,see the User Guide.
5. Copy the UART console output to a log file for analysis and certification.

ACS build steps - Linux application

Certain Peripheral, PCIe and Memory map tests require Linux operating system.This chapter provides information on building and executing these tests from the Linux application.

1. Build steps and environment setup

This section lists the porting and build steps for the kernel module. The patch for the kernel tree and the Linux PAL are hosted separately on linux-acs repo

1.1 Building the kernel module

Prerequisites

• Linux kernel source version 5.11, 5.13, 5.15, 6.0, 6.4, 6.7, 6.8
• Install GCC-ARM 13.2 toolchain.
• Build environment for AArch64 Linux kernel.
  NOTE:
  
• Linux version 6.8 is recommended version.

Porting steps for Linux kernel

1. git clone https://git.gitlab.arm.com/linux-arm/linux-acs.git linux-acs
2. git clone https://github.com/ARM-software/bsa-acs.git bsa-acs
3. git clone https://github.com/torvalds/linux.git -b v6.8
4. export CROSS_COMPILE=<GCC13.2 toolchain path> pointing to /bin/aarch64-linux-gnu-
5. git apply <local_dir>/linux-acs/kernel/src/0001-BSA-ACS-Linux-6.8.patch to your kernel source tree.
6. make ARCH=arm64 defconfig && make -j $(nproc) ARCH=arm64

NOTE: The steps mentions Linux version 6.8, as it is latest version which is verified at ACS end.

1.2 Build steps for BSA kernel module

1. cd <local_dir>/linux-acs/files
2. export CROSS_COMPILE=/bin/aarch64-linux-gnu-
3. export KERNEL_SRC=<linux_kernel_path>
4. ./bsa_setup.sh <local_dir/bsa-acs>
5. ./linux_bsa_acs.sh

Successful completion of above steps will generate bsa_acs.ko

1.3 BSA Linux application build

1. cd /linux_app/bsa-acs-app
2. export CROSS_COMPILE=/bin/aarch64-linux-gnu-
3. make

Successful completion of above steps will generate executable file bsa

2. Loading the kernel module

Before the BSA ACS Linux application can be run, load the BSA ACS kernel module using the insmod command.

shell> insmod bsa_acs.ko

3. Running BSA ACS

shell> ./bsa

• For information on the BSA Linux application parameters, see the User Guide.

ACS build steps - Bare-metal abstraction

The Bare-metal build environment is platform specific.

To provide a baseline, the build steps to integrate and run the Bare-metal tests from UEFI shell are provided in README.md.

For details on generating the binaries to run on Bare-metal environment, refer README.md

ACS build steps - Memory model consistency tests

To evaluate the correctness and consistency of a system's memory model, memory model consistency tests (litmus tests) can be optionally built into BSA UEFI application, the build and run steps are provided in mem_test/README.md.

Security implication

The Arm SystemReady ACS test suite may run at a higher privilege level. An attacker may utilize these tests to elevate the privilege which can potentially reveal the platform security assets. To prevent the leakage of Secure information, Arm strongly recommends that you run the ACS test suite only on development platforms. If it is run on production systems, the system should be scrubbed after running the test suite.

Limitations

• For systems that present firmware compliant to SBBR, BSA depends on SPCR acpi table to get UART information. UEFI console setting must be set to "serial" on these systems.

• ITS test are available only for systems that present firmware compliant to SBBR.

• Some PCIe and Exerciser test are dependent on PCIe features supported by the test system. Please fill the required API's with test system information.

  • pal_pcie_p2p_support : If the test system PCIe supports peer to peer transaction.
  • pal_pcie_is_cache_present : If the test system supports PCIe address translation cache.
  • pal_pcie_get_legacy_irq_map : Fill system legacy irq map

  Below exerciser capabilities are required by exerciser test.

  • MSI-X interrupt generation.
  • Incoming Transaction Monitoring(order, type).
  • Initiating transacions from and to the exerciser.
  • Ability to check on BDF and register address seen for each configuration address along with access type.

BSA ACS version mapping

---

BSA Spec Version	BSA ACS Version	BSA Tag ID	Pre-Si Support
BSA v1.0(c)	v1.0.8	v24.03_REL1.0.8	Yes
BSA v1.0(c)	v1.0.7	v23.12_REL1.0.7	Yes
BSA v1.0(c)	v1.0.6	v23.11_BootFramework	Yes
BSA v1.0(c)	v1.0.6	v23.09_REL1.0.6	Yes
BSA v1.0(c)	v1.0.5	v23.07_REL1.0.5	Yes
BSA v1.0(c)	v1.0.4	v23.03_REL1.0.4	Yes
BSA v1.0	v1.0.3	v23.01_REL1.0.3	No
BSA v1.0	v1.0.2	v22.10_REL1.0.2	No
BSA v1.0	v1.0.1	v22.06_REL1.0.1	No
BSA v1.0	v1.0	v21.09_REL1.0	No

---

License

BSA ACS is distributed under Apache v2.0 License.

Feedback, contributions, and support

• For feedback, use the GitHub Issue Tracker that is associated with this repository.
• For support, send an email to "[email protected]" with details.
• Arm licensees may contact Arm directly through their partner managers.
• Arm welcomes code contributions through GitHub pull requests. See the GitHub documentation on how to raise pull requests.

---

Copyright (c) 2021-2024, Arm Limited and Contributors. All rights reserved.

bsa-acs's Issues

add B_SEC_* tests from SBSA ACS?

SBSA ACS has tests for B_SEC_0[1-5] features. By name they belong to BSA ACS.

They are described only in BSA specification.

Integrate SBSA ACS into BSA ACS

Several tests from SBSA ACS moved to BSA ACS to the level when running sbsa.efi orders to run also BSA ACS.

So maybe it is time to merge two of them into one application?

One binary to test compliance with BSA spec, all levels of SBSA one and in future also other xBSA specs.

BSA test 605: Memory Attribute of DMA failure

Note that this may be related to the Linux kernel as we don’t see this failure in ACS 3.1. ACS 3.1 doesn’t detect any DMA controller.

We got the verbose message below with this failure, which doesn't give us enough information to fix this issue and it may probably be a test suite issue or expected behaviour (as testing this may require a device-specific driver). Could anyone give us more information?

In decode_mem_attr_sh with attr=ff, sh=0
    DMA controler 1: IO Coherent DMA memory should be inner/outer writeback, inner shareable
In decode_mem_attr_sh with attr=ff, sh=0
    DMA controler 0: IO Coherent DMA memory should be inner/outer writeback, inner shareable

• The message shows that pal_dma_mem_get_attrs() gets invalid Memory attributes and Sharability info, so the DMA entry in g_dma_info_table may be invalid. How does sbsa/bsa build the g_dma_info_table? Is the information in the pal_dma_create_info_table from MMU translation table or from APCI table? Where is the pal_dma_create_info_table source code?
• Is the failure a firmware issue or Hardware issue?
• Is there a Linux command to check DMA controller? It looks like all the devices are working fine under OS.

missing \n in info block

During tests of qemu/sbsa-ref I got this weird output:

 GIC_INFO: Number of ITS              :    0

 GTDT don't have el2 virt timer info
 using bsa recommended value 28 TIMER_INFO: Number of system timers  :    0
 WATCHDOG_INFO: Number of Watchdogs   :    1

Looks like forgotten "\n" in some message.

tests 278 and 279 should continue test of all the bdf’s found

278 : Check uniqueness of StreamID
279 : Check Device's ReqID-DeviceID-StreamID

For now, if BSA tests 278 and 279 find a problematic device, it will directly return without checking other devices. It looks like we should change the tests to continue to check other devices.

Fix order of Power and Wakeup tests

Now those tests are in weird order:

      *** Starting Power and Wakeup semantic tests ***
Operating System View:
 503 : Wake from EL0 PHY Timer Int                : Result:  PASS
 504 : Wake from EL0 VIR Timer Int                : Result:  PASS
 505 : Wake from EL2 PHY Timer Int                : Result:  PASS
 501 : Wake from Watchdog WS0 Int                 : Result:  PASS
 502 : Wake from System Timer Int
       Checkpoint --  1                           : Result:  SKIPPED

      *** One or more tests have Failed/Skipped.***

Forever loop when parsing pmu node

Hi,

I just found that we get the forever loop when the pmu-node of device-tree following the requirement of source code.
Because the 'i' variable is used in loop, then the condition in for is wrong
I changed as below and success to parse, but I do not have permission to contribute(push) on this repository.

--- a/platform/pal_uefi_dt/src/pal_pe.c
+++ b/platform/pal_uefi_dt/src/pal_pe.c
@@ -359,7 +359,7 @@ pal_pe_data_cache_ops_by_va(UINT64 addr, UINT32 type)
 VOID
 pal_pe_info_table_pmu_gsiv_dt(PE_INFO_TABLE *PeTable)
 {
-  int i, offset, prop_len;
+  int i, offset, prop_len, dt_idx;
   UINT64 dt_ptr = 0;
   UINT32 *Pintr;
   int index = 0;
@@ -379,12 +379,12 @@ pal_pe_info_table_pmu_gsiv_dt(PE_INFO_TABLE *PeTable)
 
   Ptr = PeTable->pe_info;
 
-  for (i = 0; i < (sizeof(pmu_dt_arr)/PMU_COMPATIBLE_STR_LEN); i++) {
+  for (dt_idx = 0; dt_idx < (sizeof(pmu_dt_arr)/PMU_COMPATIBLE_STR_LEN); dt_idx++) {
 
       /* Search for pmu nodes*/
-      offset = fdt_node_offset_by_compatible((const void *)dt_ptr, -1, pmu_dt_arr[i]);
+      offset = fdt_node_offset_by_compatible((const void *)dt_ptr, -1, pmu_dt_arr[dt_idx]);
       if (offset < 0) {
-          bsa_print(ACS_PRINT_DEBUG, L"  PMU compatible value not found for index:%d\n", i);
+          bsa_print(ACS_PRINT_DEBUG, L"  PMU compatible value not found for index:%d\n", dt_idx);
           continue; /* Search for next compatible item*/
       }
 
@@ -463,7 +463,7 @@ pal_pe_info_table_pmu_gsiv_dt(PE_INFO_TABLE *PeTable)
               }
           }
           offset =
-              fdt_node_offset_by_compatible((const void *)dt_ptr, offset, pmu_dt_arr[i]);
+              fdt_node_offset_by_compatible((const void *)dt_ptr, offset, pmu_dt_arr[dt_idx]);
       }
   }
 }

Best Regards,
Hung Huynh

Test case 830 hangs in an endless loop with any GCC optimization enabled

The BSA (and SBSA equivalent) both hang in an endless loop unless compiler optimization is completely disabled. These tests use a very questionable programming technique of taking the address of a C label and having the exception handler return to that address. With optimization enabled GCC is free to do whatever it wants with the code flow including possibly duplicating code paths, etc.

The general GCC issue is discussed in this thread for instance:

https://stackoverflow.com/questions/6127563/label-moving-after-optimization-is-turned-on-in-gcc

At a minimum I would suggest specifically disabling optimization in bsa-acs/uefi_app/BsaAcs.inf.

       Received exception of type: 0
       Calculated config address is 69400004 pal_mmio_read Address = 69400004  Data = 100004 
 pal_mmio_write Address = 69400004  Data = 100004 

       Calculated config address is 69400034 pal_mmio_read Address = 69400034  Data = 48 

       Calculated config address is 69400048 pal_mmio_read Address = 69400048  Data = 424010 

       Calculated config address is 69400050 pal_mmio_read Address = 69400050  Data = 2000 
 pal_mmio_write Address = 69400050  Data = 2000 

       Calculated config address is 69400034 pal_mmio_read Address = 69400034  Data = 48 

       Calculated config address is 69400048 pal_mmio_read Address = 69400048  Data = 424010 

       Calculated config address is 69400050 pal_mmio_read Address = 69400050  Data = 2000 
 pal_mmio_write Address = 69400050  Data = 82000 

       Calculated config address is 69400004 pal_mmio_read Address = 69400004  Data = 100004 
 pal_mmio_write Address = 69400004  Data = 100004 

       Received exception of type: 0
       Calculated config address is 69400004 pal_mmio_read Address = 69400004  Data = 100004 
 pal_mmio_write Address = 69400004  Data = 100004 

       Calculated config address is 69400034 pal_mmio_read Address = 69400034  Data = 48 

       Calculated config address is 69400048 pal_mmio_read Address = 69400048  Data = 424010 

       Calculated config address is 69400050 pal_mmio_read Address = 69400050  Data = 2000 
 pal_mmio_write Address = 69400050  Data = 2000 

       Calculated config address is 69400034 pal_mmio_read Address = 69400034  Data = 48 

       Calculated config address is 69400048 pal_mmio_read Address = 69400048  Data = 424010 

       Calculated config address is 69400050 pal_mmio_read Address = 69400050  Data = 2000 
 pal_mmio_write Address = 69400050  Data = 82000 

       Calculated config address is 69400004 pal_mmio_read Address = 69400004  Data = 100004 
 pal_mmio_write Address = 69400004  Data = 100004 ```

Do not inform that system does not have GICv2m

Can we agree that GICv2m is distant past and just do not print that we skip tests related to it?

Now on GICv3+ system:

      ***  Starting GIC tests ***
Operating System View:
 201 : Check GIC version                          : Result:  PASS
 202 : Check GICv2 Valid Configuration
       Checkpoint --  1                           : Result:  SKIPPED
 203 : If PCIe, GICv3 then ITS, LPI               : Result:  PASS
 204 : Check GICv3 Security States                : Result:  PASS
 205 : Non-secure SGIs are implemented            : Result:  PASS
 206 : Check PPI Assignments for OS               : Result:  PASS

Hypervisor View:
 226 : Check PPI Assignments Hypervisor
       EL2-Virtual timer interrupt 28 not received     : Result:  PASS

       No GICv2m, Skipping all GICv2m tests

      *** Starting ITS tests ***

Cleaner output on GICv3+ system:

      ***  Starting GIC tests ***
Operating System View:
 201 : Check GIC version                          : Result:  PASS
 203 : If PCIe, GICv3 then ITS, LPI               : Result:  PASS
 204 : Check GICv3 Security States                : Result:  PASS
 205 : Non-secure SGIs are implemented            : Result:  PASS
 206 : Check PPI Assignments for OS               : Result:  PASS

Hypervisor View:
 226 : Check PPI Assignments Hypervisor
       EL2-Virtual timer interrupt 28 not received     : Result:  PASS


      *** Starting ITS tests ***

System hangs at test 861 : PCIe Unaligned access

In the verbose messages (the outputbsa -v 1), test 861 keeps printing the same message for checking bdf 000000.

[ 601.380221] 861 : PCIe Unaligned access START
[ 601.386824]
[ 601.386824] Calculated config address is 28c0600010
[ 601.386824] The BAR value of bdf 060000 is 3011
[ 601.386824] Calculated config address is 28c0000010
[ 601.386824] The BAR value of bdf 000000 is 0
[ 601.386824] Calculated config address is 28c0000010
[ 601.386824] The BAR value of bdf 000000 is 0
[ 601.386824] Calculated config address is 28c0000010
[ 601.386824] The BAR value of bdf 000000 is 0
[ 601.386824] Calculated config address is 28c0000010

It looks like the problem is that there is no break in while loop in

bsa-acs/test_pool/pcie/operating_system/test_os_p061.c

Line 50 in 1cc33fe

while (count--) {

The same test (405) in sbsa doesn't have this issue because it has a condition check to break the loop in https://github.com/ARM-software/sbsa-acs/blob/5ccf09073bd4c17cab96bc338e2a3a314bf3a078/test_pool/pcie/test_p005.c#L84

Therefore, we may need to make the change below to fix this issue.

1. Move the line "next_bdf:" to above the line "while (count--) {"
2. Remove "count--;" for the places that would run "goto next_bdf;"

test_os_m002.c is not buildable

/home/marcin/devel/linaro/sbsa-qemu/code/edk2/ShellPkg/Application/bsa-acs/test_pool/memory_map/operating_system/test_os_m002.c: In function ‘payload’:
/home/marcin/devel/linaro/sbsa-qemu/code/edk2/ShellPkg/Application/bsa-acs/test_pool/memory_map/operating_system/test_os_m002.c:89:18: error: storing the address of local variable ‘exception_taken_n’ in ‘branch_to_test’ [-Werror=dangling-pointer=]
   89 |   branch_to_test = &&exception_taken_n;
      |   ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~
/home/marcin/devel/linaro/sbsa-qemu/code/edk2/ShellPkg/Application/bsa-acs/test_pool/memory_map/operating_system/test_os_m002.c:106:1: note: ‘exception_taken_n’ declared here
  106 | exception_taken_n:
      | ^~~~~~~~~~~~~~~~~
/home/marcin/devel/linaro/sbsa-qemu/code/edk2/ShellPkg/Application/bsa-acs/test_pool/memory_map/operating_system/test_os_m002.c:31:14: note: ‘branch_to_test’ declared here
   31 | static void *branch_to_test;
      |              ^~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[1]: *** [GNUmakefile:991: /home/marcin/devel/linaro/sbsa-qemu/code/Build/Shell/DEBUG_GCC49/AARCH64/ShellPkg/Application/bsa-acs/uefi_app/BsaAcs/OUTPUT/test_pool/memory_map/operating_system/test_os_m002.obj] Błąd 1
make[1]: Opuszczenie katalogu '/home/marcin/devel/linaro/sbsa-qemu/code/Build/Shell/DEBUG_GCC49/AARCH64/ShellPkg/Application/bsa-acs/uefi_app/BsaAcs'

18:19 marcin@puchatek:code$ aarch64-linux-gnu-gcc --version
aarch64-linux-gnu-gcc (GCC) 12.2.1 20220819 (Red Hat Cross 12.2.1-1)
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

output is nightmare

Current output of BSA ACS in unreadable mess. Default run of tool should output just ID/name of test and it's result:

      ***  Starting GIC tests ***                                                                                 
Operating System:                                                                                                 
 201 : B_GIC_01: Check GIC version                 : Result:  PASS
 202 : B_GIC_02: Check GICv2 Valid Configuration   : Result:  SKIPPED
 203 : B_GIC_03: If PCIe, GICv3 then ITS, LPI      : Result:  FAIL
 204 : B_GIC_04: Check GICv3 Security States       : Result:  PASS
 205 : B_GIC_05: Non-secure SGIs are implemented   : Result:  PASS
 206 : B_PPI_01: Check PPI Assignments for OS      : Result:  PASS

Then second run could be done with increased verbosity level to see warning, debugs etc. information.

For SBSA-ACS I did some serious changes in how output is generated: ARM-software/sbsa-acs#146 - would be nice to see something like that in BSA-ACS as well.

Unify test result info like in sbsa-acs?

SBSA-ACS has val_print_test_end() function to unify output how test block went.

Can you implement it in BSA-ACS too?

ARM-software/sbsa-acs#193

Unable to run explicit test case

On trying to explicitly run test case 839 for non integrated pcie devices,Bsa.efi doesnt run the desired test case but outputs the platform info and skips all tests.Logs are as below

BSA_TestCase_Log.txt
BSA_Test_Log.txt
BSA_acpiview_Log.txt

Fix id in test 354

Should SMMU_01 be B_SMMU_01?

354 : SMMUv3 Integration compliance                  START

       B_SMMU_21, SMMU_01
                                       : Result:  PASS 
         END

PCIE_INFO: Number of BDFs found : A

I was playing with sbsa-ref with more complex PCI Express setup and BSA ACS gave me weird output:

 PCIE_INFO: Number of ECAM regions    :    1 
  Dev bdf 0x300 type 0x10 RP bdf 0x300
  Dev bdf 0x400 type 0x10 RP bdf 0x400
  Dev bdf 0x500 type 0x10 RP bdf 0x500
  Dev bdf 0x600 type 0x10 RP bdf 0x600
  Dev bdf 0x700 type 0x10 RP bdf 0x700
  Dev bdf 0x10000 type 0x1 RP bdf 0x300
  Dev bdf 0x20000 type 0x1 RP bdf 0x400
  Dev bdf 0x30000 type 0x1 RP bdf 0x500
  Dev bdf 0x40000 type 0x1 RP bdf 0x600
  Dev bdf 0x50000 type 0x80 RP bdf 0x700
 PCIE_INFO: Number of BDFs found      :    A

I would expect "10" in this place.

Shell> pci
   Seg  Bus  Dev  Func
   ---  ---  ---  ----
    00   00   00    00 ==> Bridge Device - Host/PCI bridge
             Vendor 1B36 Device 0008 Prog Interface 0
    00   00   01    00 ==> Network Controller - Ethernet controller
             Vendor 8086 Device 10D3 Prog Interface 0
    00   00   02    00 ==> Display Controller - VGA/8514 controller
             Vendor 1234 Device 1111 Prog Interface 0
    00   00   03    00 ==> Bridge Device - PCI/PCI bridge
             Vendor 1B36 Device 000C Prog Interface 0
    00   00   04    00 ==> Bridge Device - PCI/PCI bridge
             Vendor 1B36 Device 000C Prog Interface 0
    00   00   05    00 ==> Bridge Device - PCI/PCI bridge
             Vendor 1B36 Device 000C Prog Interface 0
    00   00   06    00 ==> Bridge Device - PCI/PCI bridge
             Vendor 1B36 Device 000C Prog Interface 0
    00   00   07    00 ==> Bridge Device - PCI/PCI bridge
             Vendor 1B36 Device 000C Prog Interface 0
    00   01   00    00 ==> Mass Storage Controller - Non-volatile memory subsystem
             Vendor 1B36 Device 0010 Prog Interface 2
    00   02   00    00 ==> Serial Bus Controllers - USB
             Vendor 1B36 Device 000D Prog Interface 30
    00   03   00    00 ==> Display Controller - Other display controller
             Vendor 1234 Device 1111 Prog Interface 0
    00   04   00    00 ==> Network Controller - Ethernet controller
             Vendor 8086 Device 10D3 Prog Interface 0
    00   05   00    00 ==> Bridge Device - PCI/PCI bridge
             Vendor 1B36 Device 000E Prog Interface 0
    00   06   08    00 ==> Pre 2.0 device - UNDEFINED
             Vendor 1AF4 Device 1005 Prog Interface 0
Shell> 

~ # lspci -t
-[0000:00]-+-00.0
           +-01.0
           +-02.0
           +-03.0-[01]----00.0
           +-04.0-[02]----00.0
           +-05.0-[03]----00.0
           +-06.0-[04]----00.0
           \-07.0-[05-06]----00.0-[06]----08.0
~ # lspci -nn
00:00.0 Host bridge [0600]: Red Hat, Inc. QEMU PCIe Host bridge [1b36:0008]
00:01.0 Ethernet controller [0200]: Intel Corporation 82574L Gigabit Network Connection [8086:10d3]
00:02.0 VGA compatible controller [0300]: Device [1234:1111] (rev 02)
00:03.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCIe Root port [1b36:000c]
00:04.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCIe Root port [1b36:000c]
00:05.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCIe Root port [1b36:000c]
00:06.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCIe Root port [1b36:000c]
00:07.0 PCI bridge [0604]: Red Hat, Inc. QEMU PCIe Root port [1b36:000c]
01:00.0 Non-Volatile memory controller [0108]: Red Hat, Inc. QEMU NVM Express Controller [1b36:0010] (rev 02)
02:00.0 USB controller [0c03]: Red Hat, Inc. QEMU XHCI Host Controller [1b36:000d] (rev 01)
03:00.0 Display controller [0380]: Device [1234:1111] (rev 02)
04:00.0 Ethernet controller [0200]: Intel Corporation 82574L Gigabit Network Connection [8086:10d3]
05:00.0 PCI bridge [0604]: Red Hat, Inc. Device [1b36:000e]
06:08.0 Unclassified device [00ff]: Red Hat, Inc. Virtio RNG [1af4:1005]

BSA Linux - Test cases like 863 for RCiEP and i-EP should not be run

This issue is created for BSA Linux for a similar reason to #40.

Align tests 602/603/604

Test 601 is aligned with previous ones, 602/603/604 are not, 606 is aligned.

      *** Starting Peripheral tests ***  
Operating System View:
 601 : USB CTRL Interface                         : Result:  PASS 
 602 : Check SATA CTRL Interface             
       Read PCIe_CFG: ECAM Base is zero FF000000     : Result:  PASS 
 603 : Check Arm BSA UART register offsets         : Result:  PASS 
 604 : Check Arm GENERIC UART Interrupt      
    Test Message       : Result:  PASS 
 606 : 16550 compatible UART                 
       Checkpoint --  2                           : Result:  SKIPPED 

PCI Test Implementation

For PCI test cases 804 and 830 but in general for all PCI test cases, I believe using MMIO for accessing config space or memory space is not correct. In these two test cases, PCI allocated memory space is being accessed using straight address deref. This will only work for systems that do not do any translation.

If you look at UEFI spec, PCI bus is supported by implementing Root Bridge I/O Protocol. This protocol provides the abstraction needed to access config and memory areas of PCI controllers and devices.

PCI test cases should use EFI_PCI_IO_PROTOCOL to access config and memory. Which in turn uses the host bridge driver for the hardware in question.

Thanks,

Test case 837 - Is accessing the register through PCIIO protocl needed? If so, a improvement might be needed.

Found a question and a improvement while debugging the failure below:

 837 : Check Config Txn for RP in HB                  START
    BDF 0xXXXXXXXXXX
       Reading Class code using PciIo protocol failed 
       Failed on PE -    0
       PCI_IN_12
       Checkpoint --  1                           : Result:  FAIL 

• https://github.com/ARM-software/bsa-acs/blob/main/test_pool/pcie/operating_system/test_os_p037.c says the 837 test is for requirement PCI_IN_12, but it looks like we just need to write and read the registers through the ECAM. Is there anything related to PCIIO protocol that I overlooked in PCI_IN_12? If not, should we update the test 837 to match the BSA spec? In other words, I’m not sure if firmware hiding a device is a BSA compliance issue.

• if we need to catch the mismatch between ECAM and PCIIO access, I’m thinking about a BSA improvement. When the test runs into a mismatch between ECAM and PCIIO, can BSA continue checking other devices instead of stopping the test? In this case, we will not need to modify and build BSA as you suggested to check other devices.

System hang occurs during SMMU initialization

@semihalf-bernacki-grzegorz confirmed that this issue can be worked around by the patch smmu.patch.txt from @ndhillonm. In the patch, the timeout-related change is needed for preventing system hang. However, we're still having no idea why adding a __sync_synchronize() call can work around this issue. Can we get a BSA expert to check the patch?

The following is the log showing where system hang occurs.

FS4:\EFI\BOOT\bsa\> Bsa.efi

 BSA Architecture Compliance Suite 
    Version 1.0  

 Starting tests with Print level is  3

 Creating Platform Information Tables 
 PE_INFO: Number of PE detected       :   24 
 GIC_INFO: Number of GICD             :    1 
 GIC_INFO: Number of ITS              :    1 
 TIMER_INFO: Number of system timers  :    0 
 WATCHDOG_INFO: Number of Watchdogs   :    1 
 PCIE_INFO: Number of ECAM regions    :    7 
 PCIE_INFO: Number of RCiEP           :   13 
 PCIE_INFO: Number of RCEC            :    0 
 PCIE_INFO: Number of EP              :   41 
 PCIE_INFO: Number of RP              :   25 
 PCIE_INFO: Number of iEP_EP          :    0 
 PCIE_INFO: Number of iEP_RP          :    0 
 SMMU_INFO: Number of SMMU CTRL       :    1 

Test 601 improperly failing USB 1.1 controllers

According to RB_PER_01 & RB_PER_02 of BSA, If the system has a USB2.0/USB3.0 host controller, it needs to be EHCI/XHCI complaint.

But test 601 is just testing all USB controllers to see if they're XHCI or EHCI complaint, improperly failing USB 1.1 controllers that are not complaint with either.

bsa-acs/test_pool/peripherals/operating_system/test_os_d001.c

Line 47 in 79986b0

if ((interface != USB_TYPE_EHCI) && (interface != USB_TYPE_XHCI)) {

PCIe tests can't be run when the PCIe root port is hidden by firmware

We got the error message below on a platform and then all PCIe tests got skipped.

• PCIe Hierarchy fail: RP of bdf 0x4010000 not found Create Bdf table failed.

Although the PCIe root ports on this platform are hidden by firmware, we should still run PCIe tests for the end-point devices. In other words, It is better to see that the PCIe Hierarchy problem causes the FAILED or SKIPPED results with some PCIe tests instead of having an error message during the initialization and no test failure.

StreamID related tests (278 and 279) should not be run if the SoC doens't have SMMU

Now the issue causes false-positive on the systems that don't have SMMU.

Test 828 - power management capability is not requried for BSA complaince

In BSA spec, PCI_IN_05 doesn’t mention that we need to check 7.5.2 (PCI Power Management Capability Structure).
Therefore, we need to remove test 828 : Power management cap rules.

"Invalid IORT node type" written with wrong level

During printing information table I got "Invalid IORT node type" message:

 PCIE_INFO: Number of iEP_EP          :    0 
 PCIE_INFO: Number of iEP_RP          :    0 
 Invalid IORT node type
 SMMU_INFO: Number of SMMU CTRL       :    1 
 Peripheral: Num of USB controllers   :    1 

"-v 1" output:

 Creating SMMU INFO table
  IORT node offset:30, type: 0
  IORT node offset:48, type: 4
  IORT node offset:30, type: 0
  IORT node offset:28, type: 48
 Invalid IORT node type
  IORT node offset:A0, type: 2
  IORT node offset:48, type: 4
  Number of IOVIRT blocks = 3
  ITS Group Num ITS: 1
  ITS ID: 0
  SMMU: Major Rev:3 Base Address:0x60050000
  Number of ID Mappings:2
  input_base:0x0 id_count:0xFFFF
  output_base:0x0 output ref:0x18
  input_base:0x3C02 id_count:0x0
  output_base:0x1 output ref:0xFFFFFFFF

  Root Complex  Segment Num:0
  Number of ID Mappings:1
  input_base:0x0 id_count:0xFFFF
  output_base:0x0 output ref:0x134

 SMMU_INFO: Number of SMMU CTRL       :    1 

I wonder where from it takes those nodes:

[000h 0000   4]                    Signature : "IORT"    [IO Remapping Table]
[004h 0004   4]                 Table Length : 000000DC
[008h 0008   1]                     Revision : 00
[009h 0009   1]                     Checksum : F8
[00Ah 0010   6]                       Oem ID : "LINARO"
[010h 0016   8]                 Oem Table ID : "SBSAQEMU"
[018h 0024   4]                 Oem Revision : 20200810
[01Ch 0028   4]              Asl Compiler ID : "LNRO"
[020h 0032   4]        Asl Compiler Revision : 00000001

[024h 0036   4]                   Node Count : 00000003
[028h 0040   4]                  Node Offset : 00000030
[02Ch 0044   4]                     Reserved : 00000000

[030h 0048   1]                         Type : 00
[031h 0049   2]                       Length : 0018
[033h 0051   1]                     Revision : 00
[034h 0052   4]                     Reserved : 00000000
[038h 0056   4]                Mapping Count : 00000000
[03Ch 0060   4]               Mapping Offset : 00000000

[040h 0064   4]                     ItsCount : 00000001
[044h 0068   4]                  Identifiers : 00000000

[048h 0072   1]                         Type : 04
[049h 0073   2]                       Length : 0058
[04Bh 0075   1]                     Revision : 02
[04Ch 0076   4]                     Reserved : 00000000
[050h 0080   4]                Mapping Count : 00000002
[054h 0084   4]               Mapping Offset : 00000044

[058h 0088   8]                 Base Address : 0000000060050000
[060h 0096   4]        Flags (decoded below) : 00000001
                             COHACC Override : 1
                               HTTU Override : 0
                      Proximity Domain Valid : 0
[064h 0100   4]                     Reserved : 00000000
[068h 0104   8]                VATOS Address : 0000000000000000
[070h 0112   4]                        Model : 00000000
[074h 0116   4]                   Event GSIV : 00000000
[078h 0120   4]                     PRI GSIV : 00000000
[07Ch 0124   4]                    GERR GSIV : 00000000
[080h 0128   4]                    Sync GSIV : 00000000
[084h 0132   4]             Proximity Domain : 00000000
[088h 0136   4]      Device ID Mapping Index : 00000001

[08Ch 0140   4]                   Input base : 00000000
[090h 0144   4]                     ID Count : 0000FFFF
[094h 0148   4]                  Output Base : 00000000
[098h 0152   4]             Output Reference : 00000030
[09Ch 0156   4]        Flags (decoded below) : 00000000
                              Single Mapping : 0

[0A0h 0160   4]                   Input base : 00003C02
[0A4h 0164   4]                     ID Count : 00000000
[0A8h 0168   4]                  Output Base : 00000001
[0ACh 0172   4]             Output Reference : 00000028
[0B0h 0176   4]        Flags (decoded below) : 00000001
                              Single Mapping : 1

[0A0h 0160   1]                         Type : 02
[0A1h 0161   2]                       Length : 003C
[0A3h 0163   1]                     Revision : 00
[0A4h 0164   4]                     Reserved : 00000000
[0A8h 0168   4]                Mapping Count : 00000001
[0ACh 0172   4]               Mapping Offset : 00000028

[0B0h 0176   8]            Memory Properties : [IORT Memory Access Properties]
[0B0h 0176   4]              Cache Coherency : 00000001
[0B4h 0180   1]        Hints (decoded below) : 00
                                   Transient : 0
                              Write Allocate : 0
                               Read Allocate : 0
                                    Override : 0
[0B5h 0181   2]                     Reserved : 0000
[0B7h 0183   1] Memory Flags (decoded below) : 00
                                   Coherency : 0
                            Device Attribute : 0
[0B8h 0184   4]                ATS Attribute : 00000001
[0BCh 0188   4]           PCI Segment Number : 00000000
[0C0h 0192   1]            Memory Size Limit : 00
[0C1h 0193   2]           PASID Capabilities : 0000
[0C3h 0195   1]                     Reserved : 00

[0C8h 0200   4]                   Input base : 00000000
[0CCh 0204   4]                     ID Count : 0000FFFF
[0D0h 0208   4]                  Output Base : 00000000
[0D4h 0212   4]             Output Reference : 00000048
[0D8h 0216   4]        Flags (decoded below) : 00000000
                              Single Mapping : 0

Raw Table Data: Length 220 (0xDC)

    0000: 49 4F 52 54 DC 00 00 00 00 F8 4C 49 4E 41 52 4F  // IORT......LINARO
    0010: 53 42 53 41 51 45 4D 55 10 08 20 20 4C 4E 52 4F  // SBSAQEMU..  LNRO
    0020: 01 00 00 00 03 00 00 00 30 00 00 00 00 00 00 00  // ........0.......
    0030: 00 18 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
    0040: 01 00 00 00 00 00 00 00 04 58 00 02 00 00 00 00  // .........X......
    0050: 02 00 00 00 44 00 00 00 00 00 05 60 00 00 00 00  // ....D......`....
    0060: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
    0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
    0080: 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00  // ................
    0090: FF FF 00 00 00 00 00 00 30 00 00 00 00 00 00 00  // ........0.......
    00A0: 02 3C 00 00 00 00 00 00 01 00 00 00 28 00 00 00  // .<..........(...
    00B0: 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00  // ................
    00C0: 00 00 00 00 00 00 00 00 00 00 00 00 FF FF 00 00  // ................
    00D0: 00 00 00 00 48 00 00 00 00 00 00 00              // ....H.......

Display whole peripherial info by default

There is additional peripheral info printed on higher output levels. Can it be printed on default level?

I mean Network/Storage/Display part:

 Peripheral: Num of USB controllers   :    1
 Peripheral: Num of SATA controllers  :    1
 Peripheral: Num of UART controllers  :    1
 Peripheral: Num of Network ctrl      :    2
 Peripheral: Num of Storage ctrl      :    1
 Peripheral: Num of Display ctrl      :    2

Do not run SMMU v2 tests on SMMU v3 system

GICv2 tests are not run on GICv3 system so let do the same with SMMUv2 tests:

 353 : SMMUv2 unique intr per ctxt bank               START

       Not valid for SMMU v3             
       B_SMMU_19
       Checkpoint --  2                           : Result:  SKIPPED 
         END

Can test 606 check for 16550 and/or pl011 uart?

BSA Spec says:

RB_PER_05 For the purpose of system development and bring up, the base system must include a UART. The UART must be one of:
• The Generic UART as specified in Section B.
• A fully 16550 compatible UART

Section B says:

An instance of the PL011 r1p5 UART will be compliant with this specification.

BSA ACS checks only for 16550 and skips test if it is not present.

PLATFORM_OVERRIDE_TIMEOUT does not work

Hi,

I am trying to use the PLATFORM_OVERRIDE_TIMEOUT to change the timeouts for my platform. However, I noticed that changing this value doesn't actually take effect.

The following diff shows changing the macro value to 1 and then adding a compile error to the #if in pal_interface.h. This should cause the build the fail, but it will still happily compile for the uefi_acpi platform with the "default' timeout values

I would attempt a fix but I'm not sure how the platform overrides are meant to work here

diff --git a/platform/pal_uefi_acpi/include/platform_override.h b/platform/pal_uefi_acpi/include/platform_override.h
index ec00b2c..5c4ec8a 100644
--- a/platform/pal_uefi_acpi/include/platform_override.h
+++ b/platform/pal_uefi_acpi/include/platform_override.h
@@ -36,7 +36,7 @@
 #define PLATFORM_OVERRIDE_PLATFORM_TIMER_GSIV 58

 /* Change OVERRIDE to 1 and define the Timeout values to be used */
-#define PLATFORM_OVERRIDE_TIMEOUT        0
+#define PLATFORM_OVERRIDE_TIMEOUT        1
 #define PLATFORM_OVERRIDE_TIMEOUT_LARGE  0x10000
 #define PLATFORM_OVERRIDE_TIMEOUT_MEDIUM 0x1000
 #define PLATFORM_OVERRIDE_TIMEOUT_SMALL  0x10
diff --git a/val/include/pal_interface.h b/val/include/pal_interface.h
index 623c70c..fff6e49 100644
--- a/val/include/pal_interface.h
+++ b/val/include/pal_interface.h
@@ -61,6 +61,7 @@
   typedef UINT64 addr_t;

 #if PLATFORM_OVERRIDE_TIMEOUT
+#error fail
     #define TIMEOUT_LARGE    PLATFORM_OVERRIDE_TIMEOUT_LARGE
     #define TIMEOUT_MEDIUM   PLATFORM_OVERRIDE_TIMEOUT_MEDIUM
     #define TIMEOUT_SMALL    PLATFORM_OVERRIDE_TIMEOUT_SMALL

BSA Linux 104 : Addressability

This is the same issue as the one with sbsa-acs in ARM-software/sbsa-acs#234

The error message is the following
WARNING:The device with bdf=0xXXXXXXX doesn't support 64 bit addressing and is not behind SMMU. Please install driver for this device and test again.

The root cause is that the ACS Linux kernel doesn't have the device driver for the device with bdf=0xXXXXXXX.

Suggest skipping this test in automation. People who have a need to run this test can still try to find and install the device driver and then manually run it.

If we still need to keep this test in automation, the suggestion would be to add device drivers as many as possible. Otherwise, it is easy to run into this failure when the system has an integrated PCIe device or a PCIe card.

SMMU tests are misleading

Test 352 says:

 352 : SMMU Revision,S-EL2 support Hyp                START

       SMMUv3.1 not providing Stage2 functionality
       Failed on PE -    0
       B_SMMU_18
       Checkpoint --  2                           : Result:  FAIL 
         END

So what exactly it tests (by looking at title)?

• SMMU revision?
• S-EL2 support?
• Stage 2 functionality?

SMMU revision and S-EL2 support are checked in test 306:

 306 : SMMU revision and S-EL2 support                START

       B_SMMU_08
                                       : Result:  PASS 
         END

B_SMMU_18 says:

If Secure-EL2 is not implemented, stage 2 System MMU functionality must be provided by a System MMU compatible with the Arm SMMUv2 specification or Arm SMMUv3 specification.

I wonder should there be separate test for "is there Secure-EL2 implemented" as now both B_SMMU_08 and B_SMMU_18 tests can pass if one or another is provided.

For sbsa-ref we pass B_SMMU_08 as we have SMMU v3.0 implemented. Then we fail B_SMMU_18 because we lack Stage2 support.

Improvements for PCIe tests' message

For tests 831, 832, 833, 836, 837, and 838, it looks like we don’t have messages showing what devices have been checked. We might want to add the message for this.

Test case 606 assuming system clock reference for UART

bsa-acs/test_pool/peripherals/operating_system/test_os_d005.c

Line 131 in 79986b0

counter_freq = val_timer_get_info(TIMER_INFO_CNTFREQ, 0);

In test 606, we perform two checks for the baud rate. The first by grabbing the info from SPCR. The second by trying to compute it via UART registers and the system clock. The issue with this is that the UART doesn’t have to use the system clock as its reference, so if a system is using a different clock reference it will fail this test, even if the system is indeed compliant.

I suggest this second check for the baud rate is removed

802 : ECAM Region accessibility check Failure

There is an assumption in the test suite that the first device will always be at Bus 0 Dev 0 Func 0. However, it is possible to see that the first device in ECAM space is an RP (Bridge device) having a non-zero BDF number, and there is no requirement in the PCIe spec to support this assumption. Therefore, we need to update the code to remove the assumption.

We also confirmed that VMware and most Linux distros are working fine with the case that the first device has a non-zero device number.

[pal_uefi_dt][pal_peripherals] Using wrong variable or special purpose?

Hi ARM's developer,

I'm working with new platform to support Arm System Ready, while running test, I found the strange the source code.
Can you help me explain them?
I expect it use "usb_dt_compatible" instead of "uart_dt_compatible" in L173-L177, or I understand wrong the purpose of those lines?
https://github.com/ARM-software/bsa-acs/blob/main/platform/pal_uefi_dt/src/pal_peripherals.c#L173

Best Regards,
Hung Huynh

Increase a value of PCIE_ECAP_ID_END macro.

PCIE_ECAP_ID_END is defined as 0x29

bsa-acs/test_pool/pcie/operating_system/test_os_p009.c

Line 28 in 726b990

#define PCIE_ECAP_ID_END 0x29

However according to "PCI Code and ID Assignment Specification Revision 1.11" values up to 0x2C are allowed.

BSA wakeup test failure (503, 504, 505) with ES ACS 1.1

Use the same system and same firmware to run both ES ACS 1.0 and ES ACS 1.1 BSA wakeup tests, but got a different result. Offline checked with @chetan-rathore. With ES ACS 1.1, it looks like PE is taking some time to enter the WFI state (low-power mode) or not entering it, so caused the failures. Supposedly, PE should enter the WFI state and EL0/EL2 Timers will wake it up. Need to take a further look to check the difference between ES ACS 1.0 and 1.1 and if PE needs more time to enter the WFI state.

ES ACS 1.0 BSA result - All PASS

 503 : Wake from EL0 PHY Timer Int                : Result:  PASS 
 504 : Wake from EL0 VIR Timer Int                : Result:  PASS 
 505 : Wake from EL2 PHY Timer Int                : Result:  PASS 

ES ACS 1.1 BSA result - All FAIL

 503 : Wake from EL0 PHY Timer Int           
       Failed on PE -    0
       Checkpoint --  1                           : Result:  FAIL 
 504 : Wake from EL0 VIR Timer Int           
       Failed on PE -    0
       Checkpoint --  1                           : Result:  FAIL 
 505 : Wake from EL2 PHY Timer Int           
       Failed on PE -    0
       Checkpoint --  1                           : Result:  FAIL 

ES ACS 1.1 BSA result (with verbosity INFO message)- All FAIL

 503 : Wake from EL0 PHY Timer Int                    START

       Failed on PE -    0
       B_WAK_01, B_WAK_02, B_WAK_03, B_WAK_04, B_WAK_05                     
       B_WAK_06, B_WAK_07, B_WAK_10, B_WAK_11
       Checkpoint --  1                           : Result:  FAIL 
         END

 504 : Wake from EL0 VIR Timer Int                    START

       Failed on PE -    0
       B_WAK_01, B_WAK_02, B_WAK_03, B_WAK_04, B_WAK_05                     
       B_WAK_06, B_WAK_07, B_WAK_10, B_WAK_11
       Checkpoint --  1                           : Result:  FAIL 
         END

 505 : Wake from EL2 PHY Timer Int                    START

       Failed on PE -    0
       B_WAK_01, B_WAK_02, B_WAK_03, B_WAK_04, B_WAK_05                     
       B_WAK_06, B_WAK_07, B_WAK_10, B_WAK_11
       Checkpoint --  1                           : Result:  FAIL 
         END

el2 virt timer message needs \n

sbsa-ref again:

 Creating Platform Information Tables
 PE_INFO: Number of PE detected       :    2
 GIC_INFO: Number of GICD             :    1
 GIC_INFO: Number of ITS              :    1

 GTDT don't have el2 virt timer info
 using bsa recommended value 28 TIMER_INFO: Number of system timers  :    0
 WATCHDOG_INFO: Number of Watchdogs   :    1

Looks like there should be end of line before TIMER_INFO

test_os_p030.c

In the 'esr' function of this test, ELR register is being set to an instruction to jump to. It is not working for me and I am wondering if this is a bug.
Current instruction is as follows but results in a hang in our system.
"val_pe_update_elr(context, (uint64_t)branch_to_test);"

To get around this, "val_pe_update_elr(context, (uint64_t)(((EFI_SYSTEM_CONTEXT_AARCH64*)context)->ELR + 4));", seems to behave better.

Is this a real issue or is it just our SoC ?

Thanks,

PENDING result from Test 501 : Wake from Watchdog WS0 Int

We run into the failure below on some ARM systems.
501 : Wake from Watchdog WS0 Int : Result: A01F5000

@semihalf-bernacki-grzegorz found the root cause. The following is my understanding of the root cause, feel free to correct me, Greg.

The problem is that the 501 test checks the result too early, so it gets PENDING result (A01F5000). It directly checks the result right after setting the WS0 timer (without waiting for the WS0 timer to get expire). In other words, the problem is that the WS0 timer interrupt service routine hasn’t got a chance to be triggered to update the result.

@semihalf-bernacki-grzegorz is working on a fix to BSA by adding code to wait for the WS0 timer to get expire.

@chetan-rathore if you have any concern about the fix, feel free to let us know.

Test 205 (Non-secure SGIs are implemented) shouldn't check invalid PE cores

Test currently not checking the correct base address of GIC RD of a PE.

ITS test 003 Issue

Hey Guys,
Can you please look at the "test_os_its003.c" ?
In particular, the array 'streamID' which is 32 elements long but it is updated using 'dev_index' variable which clearly goes past 32 if there are enough 'bdf_tbl_ptr->num_entries'.
This causes memory corruption.

Thanks,
Narinder

4KB Granule support test is wrong

On QEMU with 'max' we have interesting bug:

   4 : Check PE 4KB Granule Support
       Failed on PE -    0
       Checkpoint --  1                           : Result:  FAIL

Test checks TGran4 bits [31:28] in ID_AA64MMFR0_EL1 for value 0 (which means "4KB granule supported").

We have 0x0001 there which means "4KB granule supports 52-bit input and output addresses" (Arm arm).

Test cases like 816 for RCiEP and i-EP should not be run

This is the same issue as ARM-software/sbsa-acs#256

The requirements for RCiEP and i-EP (based on B_REP_1 and B_IEP_1 in BSA spec) are referenced in SBSA Level 6 checklist in SBSA spec rather than the BSA checklist. Therefore, we should not test these requirements in BSA.
For example, we should update test case 816: Check MSI and MSI-X support rule to not check RCiEP devices by default.

Failure - BSA 804 : PHB,RP must recognize Txn frm upstream

We run into a BSA 804 PCIe test failure below on an ARM platform.

804 : PHB,RP must recognize Txn frm upstream
Failed. Exception on Memory Access For Bdf : 0x3000000
Failed on PE - 0
Checkpoint -- 1 : Result: FAIL

The problem is that BSA.efi directly accesses an address that falls within the non-prefetchable memory window in the SoC's PCIe bridge registers (Offset 0x20). The SoC vendor expected the BSA.efi to check ACPI _TRA (Address Translation) and get the offset for accessing the correct address, which is how OSes do for accessing the address. Does it make sense to update BSA.efi for using _TRA? Or using the _TRA is kind of workaround for hardware issues? If you think using _TRA is a workaround, could you give more information like a description in the specification to me?

For more information about _TRA, please check https://uefi.org/specs/ACPI/6.4/19_ASL_Reference/ACPI_Source_Language_Reference.html#qwordmemory-qword-memory-resource-descriptor-macro

Test case 606 UART loopback test doesn't mask off MSR delta bits

The check

bsa-acs/test_pool/peripherals/operating_system/test_os_d005.c

Line 175 in ea487a9

if (msr_status != CTS_DCD_EN)

should be changed to
if ((msr_status & 0xF0) != CTS_DCD_EN)

Issue with test_os_its003.c

Hi,
Looking at the code in test_os_its003.c, it seems to assume that there is only one SMMU in the system. If we have a system like the one attached in the pic where we have multiple SMMU instances and PCIe RCs behind them having same StreamIDs, then, uniqueness of streamID should always be checked in the context of the specific SMMU instance i.e. to put it differently, in the context of a specific output reference.

Improvement for 811: Check RP Byte Enable Rules

This issue is just created for giving more information about the failure that was fixed by 0160970.

The problem with the original test case is that the command register is not suitable for checking the RP Byte Enable Rules as some SoC may have read-only MSE and BME bits and it can only cover 1 byte request. The test case should cover 1, 2, 4 byte requests.
Therefore, the fix in 0160970 is to use Bus number register instead.

List SMMU version in information block

There is GIC version listed at start of app. Would be nice to have SMMU version there as well:

 Creating Platform Information Tables
 PE_INFO: Number of PE detected       :    4 
 GIC INFO: GIC version                :    v3 
 GIC_INFO: Number of GICD             :    1 
 GIC_INFO: Number of GICR RD          :    1 
 GIC_INFO: Number of ITS              :    1 
 TIMER_INFO: Number of system timers  :    0 
 WATCHDOG_INFO: Number of Watchdogs   :    1 
 PCIE_INFO: Number of ECAM regions    :    1 
 PCIE_INFO: Number of BDFs found      :    10
 PCIE_INFO: Number of RCiEP           :    0 
 PCIE_INFO: Number of RCEC            :    0 
 PCIE_INFO: Number of EP              :    4 
 PCIE_INFO: Number of RP              :    5 
 PCIE_INFO: Number of iEP_EP          :    0 
 PCIE_INFO: Number of iEP_RP          :    0 
 SMMU_INFO: Number of SMMU CTRL       :    1 
 Peripheral: Num of USB controllers   :    1 
 Peripheral: Num of SATA controllers  :    1 
 Peripheral: Num of UART controllers  :    1