microsoft / mu_basecore

Project Mu BaseCore

Project Mu Basecore Repository

Host Type & Toolchain	Build Status	Test Status	Code Coverage
Windows_VS2022
Ubuntu_GCC5

This repository is part of Project Mu. Please see Project Mu for details https://microsoft.github.io/mu

For more details about the repository, refer to RepoDetails.md.

Branch Status - release/202311

Status

In Development

Entered Development

2023/12/15

Anticipated Stabilization

Feb 2024

Branch Changes - release/202311

Breaking Changes-dev

• Incomplete

Main Changes-dev

• BPDT is now installed in the config table
• Added more granular variable policy querying
• We now wait for all APs before finishing initialization
• Added MbedTLS to crypto library
• Removed AP waking vector from ResetVector
• Removed AP waking vector logic in SecCore

Bug Fixes-dev

• Applied TCBZ3923 to MbedTLS
• Corrected the description for th MpHandOff header file
• Optimized BmExpandPartitionDevicePath

2311_RefBoot Changes

• Incomplete

2311_CIBuild Changes

• Changed PcdDelayedDispatchMaxEntries token value to get rid of a conflict
• Added missing PCD to MemoryProtectionUnitTestHost.inf
• Applied TCBZ3923 to MbedTLS
• Added MbedTLS to the Markdown Lint exclusion

2311_Rebase Changes

Starting commit: 15df7500b5 ("Removed unused crypto file", 2023-12-15)
Destination Commit from upstream edk2: 8736b8fdca ("RedfishPkg: RedfishDiscoverDxe: Optimize the Redfish Discover flow", 2023-11-22)

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct https://opensource.microsoft.com/codeofconduct/

For more information see the Code of Conduct FAQ https://opensource.microsoft.com/codeofconduct/faq/ or contact [email protected]. with any additional questions or comments.

Contributions

Contributions are always welcome and encouraged! Please open any issues in the Project Mu GitHub tracker and read https://microsoft.github.io/mu/How/contributing/

For documentation:

Copyright & License

Copyright (C) Microsoft Corporation
SPDX-License-Identifier: BSD-2-Clause-Patent

Upstream License (TianoCore)

Copyright (c) 2019, TianoCore and contributors. All rights reserved.

SPDX-License-Identifier: BSD-2-Clause-Patent

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

Subject to the terms and conditions of this license, each copyright holder and contributor hereby grants to those receiving rights under this license a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except for failure to satisfy the conditions of this license) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer this software, where such license applies only to those patent claims, already acquired or hereafter acquired, licensable by such copyright holder or contributor that are necessarily infringed by:

1. their Contribution(s) (the licensed copyrights of copyright holders and non-copyrightable additions of contributors, in source or binary form) alone; or
2. combination of their Contribution(s) with the work of authorship to which such Contribution(s) was added by such copyright holder or contributor, if, at the time the Contribution is added, such addition causes such combination to be necessarily infringed. The patent license shall not apply to any other combinations which include the Contribution.

Except as expressly stated above, no rights or licenses from any copyright holder or contributor is granted under this license, whether expressly, by implication, estoppel or otherwise.

DISCLAIMER

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

mu_basecore's Issues

[Bug]: Char Encoding Check Plugin Is Noisy

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

Currently, Char Encoding Check Plugin prints out a message for each file it scans saying that it passed the check. This can scale to thousands of files and result in CI build logs that are 7/8ths just that message.

Expected Behavior

Instead of printing out a message on success (for every single file in a package), only print out a message on the rare case of failure.

Steps To Reproduce

Run the Char Encoding Check Plugin and view the output.

Build Environment

- OS(s): Any
- Tool Chain(s): Any
- Targets Impacted: Any

Version Information

release/202302

Urgency

Low

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: [GenFw] FindPrmHandler in Elf64Convert.c assumes symbol value is file offset

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

When GenFw is ran with the --prm option, it will attempt to locate the PrmModuleExportDescriptor symbol as part of the ScanSections64 function call. Once found, it will pass its 'st_value' field value into FindPrmHandler as an offset to find the location of PRM handler symbols in the file, as seen here:

mu_basecore/BaseTools/Source/C/GenFw/Elf64Convert.c

Line 991 in 5498883

FindPrmHandler(Sym->st_value);

However, st_value does not necessarily represent the offset of a symbol in the file. It appears to actually represent the VMA of the symbol. For example, on my build, PrmModuleExportDescriptor is placed in the .data section, which has a VMA of 0x6000, but the .data section is actually at file offset 0x7000. So, when st_value is passed into FindPrmHandler, the function begins looking at offset 0x6000 for PRM handlers rather than 0x7000.

A potential fix (which works on my build) is to calculate the symbol offset by first finding the section header it belongs to using the symbol's st_shndx field. Then, subtract the section's address from the symbol's address before finally adding the section offset, like this:

STATIC
Elf64_Addr
GetSymValueOffset (
  Elf_Sym* Sym
  )
{
    Elf_Shdr* shdr = GetShdrByIndex(Sym->st_shndx);
    return (Sym->st_value - shdr->sh_addr) + shdr->sh_offset;
}

Thus, anytime a symbol's value is treated as an offset, this function should be used instead.

Expected Behavior

I expect GenFw, when ran with the --prm option, to locate PRM-related symbols in an ELF file regardless of what their VMA value is.

Steps To Reproduce

1. Write a sample PRM module, and pass the --prm option to GenFw in the sample module's INF file
2. Build UEFI FW using the CLANG100WIN/ARM toolchain
3. View the build log, and observe that GenFw will likely error out as it attempts to read garbage (in my case, it prints error message "FindPrmHandler: Number 255 is too high." as it thought there were over 25,000 PRM Handlers)

Build Environment

- OS(s): Windows 11
- Tool Chain(s): CLANG100WIN, arm-linker
- Targets Impacted: AARCH64 DEBUG (only target tested)

Version Information

Latest

Urgency

Low

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

Maintainer feedback requested

Anything else?

This shows .data has a VMA that is not equal to its file offset:

This shows PrmModuleExportDescriptor has a symbol value of 0x6000 (since it's the first symbol in .data):

And this shows PrmModuleExportDescriptor actually has a file offset of 0x7000 (thus the symbol's value is not its offset):

[Feature]: Support DXE_SMM_DRIVER in PolicyServicePkg

Feature Overview

Currently PolicyServicePkg cannot support DXE_SMM_DRIVER module type, implement the required library instance and driver for supporting the traditional MM.

Solution Overview

1. Add DXE_SMM_DRIVER to LIBRARY_CLASS of MmPolicyLib.inf
2. Refactor the PolicyMm module to have a common entrypoint, and both Standalone MM and Traditional MM entrypoint call the common entrypoint.
3. Add Traditional MM description to Readme.md.

Alternatives Considered

No response

Urgency

Medium

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: BDS infinite retry not functional with inactive boot option at the end of queue

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

When booting with a given boot option list, with the newly introduced PcdSupportInfiniteBootRetries PCD set to true, the boot options should be iterated through indefinitely. However, due to the check for active boot options, the branch-out could cause the logic not going through the newly added wrap around logic and break out of the loop.

Expected Behavior

With the newly introduced PcdSupportInfiniteBootRetries PCD set to true, the boot options should be iterated through indefinitely.

Steps To Reproduce

1. Register several boot options with the last one registered not setting LOAD_OPTION_ACTIVE.
2. Let the BDS boot, make sure the let them all fail in the first pass.
3. System will iterate through the given option list, skip the inactive option and break out of the loop.

Build Environment

- OS(s): Ubuntu 22.04
- Tool Chain(s): GCC5
- Targets Impacted: AARCH64

Version Information

commit a5bf121fa89e0e65c0edfafac6eee88777214eda

Urgency

Medium

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: SnpDxe should execute PxeShutdown before ExitBootServices

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

SnpDxe registers for an ExitBootServices callback and runs the PXE_OPCODE_SHUTDOWN and PXE_OPCODE_STOP commands for any network controllers that the driver is attached to. If the Bus Master Enable bit is cleared by PciBusDxe before this point, then completions returned by the network cards will be unreadable to their drivers during the shutdown sequence.

Expected Behavior

Register the SnpDxe callback for gMuEventPreExitBootServicesGuid instead of gEfiEventExitBootServicesGuid to ensure the correct ordering:

1. ExitBootServices event
2. Network card shutdown sequence is completed
3. BME is disabled on PCI bridges (if configured)

Steps To Reproduce

1. Ensure PcdDisableBMEonEBS is set to TRUE
2. Boot to OS from disk
3. When ExitBootServices is notified, BME will be disabled on PCI bridges before PxeShutdown is issued to any cards

Build Environment

- OS(s): Windows
- Tool Chain(s): GCC5
- Targets Impacted: DEBUG AARCH64

Version Information

Commit: 502c9a9eb29951fbc507514024e3cc574beb50c2

Urgency

Medium

Are you going to fix this?

I will fix it #522

Do you need maintainer feedback?

Maintainer feedback requested

Anything else?

No response

[Bug]: Run BuildEnv Script bring a Error in Line 233

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

Run BuildEnv script in Basetools after make basetools, bring a error of unary operator expected.
Output:
Loading previous configuration from /home/user/mu_basecore/BaseTools/Conf/BuildEnv.sh
Using EDK2 in-source Basetools
WORKSPACE: /home/user/mu_basecore/BaseTools
EDK_TOOLS_PATH: /home/user/mu_basecore/BaseTools
CONF_PATH: /home/user/mu_basecore/BaseTools/Conf
./BuildEnv: line 223: [: !=: unary operator expected
Copying $EDK_TOOLS_PATH/Conf/build_rule.template
to /home/user/mu_basecore/BaseTools/Conf/build_rule.txt
./BuildEnv: line 223: [: !=: unary operator expected
Copying $EDK_TOOLS_PATH/Conf/tools_def.template
to /home/user/mu_basecore/BaseTools/Conf/tools_def.txt
./BuildEnv: line 223: [: !=: unary operator expected
Copying $EDK_TOOLS_PATH/Conf/target.template
to /home/user/mu_basecore/BaseTools/Conf

Expected Behavior

Set the Environment for use BinWrappers

Steps To Reproduce

1. Build BaseTools manually using make.
2. Run BuildEnv script inside BaseTools

Build Environment

- OS(s): Ubuntu 18.04, 20.04, 22.04 can be afected more systems, i have tried in some docker containers
- Tool Chain(s): GCC-5
- Targets Impacted: NO-TARGET

Version Information

Release: 202208

Urgency

Low

Are you going to fix this?

Someone else needs to fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: Override Validation for build_rule.template files

Feature Overview

The OverrideValidation plugin is a very good tool that can help us catch the override module that is required to sync-up during build time, but it does not cover build_rule.template, and some of specific feature may need to override the build_rule.template, it would be great if there is a mechanism that can catch the overridding build_rule.template that needs to sync up with Basetool update.

Solution Overview

If I have an override build_rule.template in my override folder, and when there is a new update in the BaseTools/Conf/build_rule.template of MU_BASECORE, it will throw a build error to remind developer to do sync up with new MU_BASECORE codebase.

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

Maintainer feedback requested

Anything else?

No response

[Bug]: Continue gracefully on unavailable resource spaces

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

On certain platforms, such as ARM based systems, there is not any available IO space.

Currently the PCI driver will exit the resource allocation if any of the requested resources are not available.

Expected Behavior

The request is that, if an endpoint is requesting IO space, there should be a way to continue gracefully without allocating any IO resources.

Steps To Reproduce

Build a system firmware with top of basecore and flash on to a device that has the above behavior on requesting IO spaces.

Once the endpoint requested IO spaces, the subsequent endpoint will no longer be able to allocate any resources.

Build Environment

- OS(s): Ubuntu 22.04 and 20.04
- Tool Chain(s): GCC5
- Targets Impacted: RELEASE, DEBUG

Version Information

Top of release/202302

Urgency

Medium

Are you going to fix this?

Someone else needs to fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: Add unit tests for memory type information changes

Feature Overview

Platforms need assurance that the code in BDS that adjusts memory type information values will function correctly. The request is to add unit tests that:

1. Verify the set of outputs for a given set of inputs
2. Demonstrate the code behaves as expected across a common range of platform input scenarios

Solution Overview

Write unit tests around important algorithms in UefiBootManagerLib related to determining new memory type information values.

Alternatives Considered

No response

Urgency

Medium

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Task]: Provide feedback on OpenSSL 1.1 alternatives

Feature Overview

As noted in the TianoCore edk2 mailing list, OpenSSL 1.1 will reach EOL in September 2023.

[RFC] [staging/CryptoLibrary] Openssl1.1 replacement proposal

Alternatives have been explored with findings summarized in the post and as of today (3/10/2023), a CryptoPkg POC is available for OpenSSL 3.0 and MbedTls 3.0.

Given, CryptoPkg is maintained in this repo and a key dependency, we should evaluate and provide feedback on the work that has been done in TianoCore.

Solution Overview

Evaluate the following two options to provide feedback.

• OpenSSL 3.0 POC
• Mbed TLS

Alternatives Considered

None other than the alternatives explored by TianoCore:

• Intel-IPP: https://software.intel.com/en-us/intel-ipp , https://github.com/intel/ipp-crypto

  • Apache license
  • CON: Only crypto primitive, no certificate, no TLS

• Libsodium: https://doc.libsodium.org/, https://github.com/jedisct1/libsodium

  • ISC license
  • CON: Only crypto primitive, no certificate, no TLS

• BoringSSL: https://github.com/google/boringssl

  • ISC license
  • CON: Google only project. It says: “We don't recommend that third parties depend upon it.”

• WolfSSL: https://www.wolfssl.com/, https://github.com/wolfSSL/wolfssl

  • GPL license
  • CON: GPL License issue.

• BearSSL: https://bearssl.org/

  • MIT license
  • CON: Current version is 0.6. It is now considered beta-quality software.

Urgency

Medium

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

Associated TianoCore Bugzilla items:

1. Openssl3.0: https://bugzilla.tianocore.org/show_bug.cgi?id=3466

2. MbedTls: https://bugzilla.tianocore.org/show_bug.cgi?id=4177

[Feature]: Consider C++ settings needed in Uncrustify config for Google Test adoption

Feature Overview

Originally C++ was not considered when crafting the Uncrustify configuration file.

With the recent adoption of Google Test, C++ should be considered. This tracks the work needed to accommodate C++ needed to better support formatting with Google Test.

Solution Overview

Determine what config items need to be set for reasonable formatting with common Google Test code constructs and make the change to the configuration file.

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

This repo is missing important files

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

[Feature]: Improve performance for NvmExpressDxe driver

Feature Overview

Current driver binding start function for NvmExpressDxe driver is being called multiple times and moving the stall to after the status check improves its performance.

Solution Overview

Moving stall function from before the status check to after the status check.

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

Maintainer feedback requested

Anything else?

No response

[Bug]:New DEBUG message in the wrong place in commit 1236f34

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

A new DEBUG message added by commit 1236f34 is misplaced, and will never be printed.

Expected Behavior

Debug message to be produced when a CRS (unsupported) response occurs.

Steps To Reproduce

Code inspection

Build Environment

- OS(s):ALL
- Tool Chain(s):ALL
- Targets Impacted:ALL

Version Information

1236f343699d0635a68583294eaacd3a7067adaa

Urgency

Low

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: Having to carry override in the unit test dsc

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

Currently MU carries an override in the unit test dsc where the newly added libraries from edk2 cannot be ported to BASECORE, to prevent the consumers to break pipelines. This is due to the consumer dsc did not properly scope their library class for host applications but they already implemented their own instance. Thus a normal override behavior from dsc inclusion did not work.

Expected Behavior

Project MU should carries less override and maintain parity with edk2.

Steps To Reproduce

N/A

Build Environment

N/A

Version Information

N/A

Urgency

Medium

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

The expected resolution is to switch over to the common instance and fixing the consumer library usage. Will revisit this issue once the consumer library migration is done.

ETA: next week.

USB enumeration updates

A couple months ago we made a change that stopped USB enumeration in case a malformed descriptor is found which can be seen here.

When ingested into platforms we got several USB enumeration errors due to bad descriptors which led to boot failures. As a temporary fix we reverted the change but a more robust failure method for malformed descriptors need to exist. This issue tracks progress for that effort.

[Feature]: Improve NVMe controller init robustness

Feature Overview

Some SSD devices have been observed to stop responding during boot after initially succeeding during initialization. When the failure occurs, the following message is observed from NvmeExpressPassThru:

NvmExpressPassThru: Timeout occurs for an NVMe command.
[ref]

Consequently, NvmeControllerInit() is called and the following assert is encountered:

ASSERT ((Private->Cap.Mpsmin + 12) <= EFI_PAGE_SHIFT);
[ref]

It is suspected that the Cap.Mpsmin value is invalid. In any case, this can cause an assertion to stop DEBUG builds and reset on RELEASE builds preventing recovery from UEFI shell or USB boot.

Solution Overview

• Perform a cfg read of DID/VID in NvmeControllerInit() to verify the device is accessible
• Return EFI_DEVICE_ERROR instead of asserting if the Cap.Mpsmin value is invalid

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: Access Denied Seen on Some Windows Systems with BaseTools bin 2022.08.4_1

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

Builds have failed on local systems and build agents after updating to 2022.08.4_1 by encountering a "Access Denied" error.

For example, the error was raised when running GenSec -s EFI_SECTION_PE32 -o <path>/.../<guid>1.pe32

The issue can be worked around on affected systems by excluding the source directory in Windows Defender.

Expected Behavior

A regression in Windows Defender compatibility is not present or readily understood.

Steps To Reproduce

1. Build a platform based on mu_basecore v2022080002.0.1
2. See if the error repros when running GenSec on the system

• Note: It has been reported to not repro on all systems

Build Environment

N/A - Basetools bin release

Version Information

Tag: v2022080002.0.1

Urgency

Medium

Are you going to fix this?

Someone else needs to fix it

Do you need maintainer feedback?

Maintainer feedback requested

Anything else?

No response

[Bug]: ARM Platforms Do Not Sync RP/RO/XP Bits to GCD Initially Leading to Access Flag Fault

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

Recently, EDK2 updated so that ARM would use TT_AF (the access flag in the page table) to represent EFI_MEMORY_RP (read protected memory). This enforces an access flag fault when software attempts to read a page without the access flag set. This is one option allowed by the Arm ARM; previously EDK2 was using TT_AF in the other allowed way, which used TT_AF as an accessed bit, i.e. a page would not have this bit set unless the page had been accessed.

Project Mu added a change to properly sync the GCD with the page table on ARM, as is done on the x86 side, but was never added to the Arm side due to an oversight upstream. This allowed all memory regions to have the capability of setting EFI_MEMORY_RP as an attribute in the GCD, which they did not have previously. When Arm's CpuDxe syncs with the GCD, it attempts to set all ranges of System Memory as EFI_MEMORY_RP by default, because that is how they start out in the page table (this is a change in behavior on the upstream side, previously the access flag is set for all ranges). Because the Capabilities of each GCD range now supports EFI_MEMORY_RP getting set, it would also get marked in the GCD as EFI_MEMORY_RP. Previously, the Arm implementation would silently drop the EFI_MEMORY_RP bit being passed in as an attribute to set because it was not in the capability set of the GCD region.

When the USB stack runs on platforms that use NonDiscoverablePciDeviceIo to manage the MMIO allocations, NonCoherentPciIoAllocateBuffer gets a new page, saves off the existing GCD descriptor, which contains EFI_MEMORY_RP for this range, updates the region in the GCD (which in turn updates the page table) to not include EFI_MEMORY_RP (implicitly when setting a cacheability attribute). When the memory is freed through NonCoherentPciIoFreeBuffer, the saved off GCD descriptor is fetched and it resets the memory range to have the old memory attributes, which includes EFI_MEMORY_RP. When SetMemorySpaceAttributes() is called, the GCD is updated to the old value, but this time it sees that an attribute (EFI_MEMORY_RP) is being added, so it calls into CpuDxe to update the page table, causing the actual page to get TT_AF set to 0.

When freeing a memory pool, the core code attempts to read the page (to set some metadata about the status of the pool, return it to the free list, etc.), which then triggers the access flag fault.

This did not fail on debug builds, because in debug builds we set the CLEAR_MEMORY_ENABLED bit, which, on every conversion of memory type (say in allocating a page) and on every pool allocate and free, causes us to call CLEAR_MEMORY_ENABLED(), which clears the EFI_MEMORY_RP bit (setting the TT_AF flag), so that future accesses of this do not get an access flag fault. We should be unsetting the RP bit on every free, as the core code will fail when attempting to give it out.

The issue is in Arm's MMU mgmt code, specifically GetNextEntryAttribute where it has EntryAttribute = Entry & TT_ATTR_INDX_MASK, which evaluates to 00011100, but TT_AF is BIT10 (1000000000) so it always gets masked off which in the old implementation worked, because we were only using it to tell if a page had been accessed, so the GCD didn't need to know about it. This should be TT_ATTRIBUTE_MASK so that we pick up TT_AF as well as the RO and XP equivalent bits in the page table entry that we were missing on initial GCD/page table syncing prior to this.

Expected Behavior

We should not hit an access flag fault when freeing a page.

Steps To Reproduce

Run an ARM platform using a nondiscoverable PCI device with PcdDebugPropertyMask with the CLEAR_MEMORY_ENABLED bit set to 0 with the latest mu_basecore release.

Build Environment

- OS(s): Ubuntu 20.04
- Tool Chain(s): GCC5
- Targets Impacted: RELEASE

Version Information

Tag: v2022080002.0.1

Urgency

High

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: Support host based unit test on ARM systems

Feature Overview

Today's host based unit test is heavily based on x86 architecture. As more targets are for ARM/AARCH64 systems nowadays, there is more need for running host based unit tests on the same architecture as the targets for better test fidelity.

However, the current host based unit test framework mainly builds for x86 systems and would introduce build breaks if we want to compile host based unit tests for AARCH64. i.e. this structure does not exist on ARM systems: https://github.com/microsoft/mu_basecore/blob/3f022dad7ac0035cfe3ed49a12403a7314445383/MdePkg/Test/UnitTest/Include/Library/UnitTestHostBaseLib.h#L88C20-L88C20.

Solution Overview

Plum through the build break when building on host based unit on ARM system and start to deploy the existing host based unit tests to ARM builds (Windows and Linnux).

There might be further tuning for plugins to make the entire flow functional.

Alternatives Considered

No response

Urgency

Medium

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: ShellPkg/Application/Shell/Shell.c - File device path type is incorrect

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

Commit 5637d61 introduced the following code:

    if ((FileDevicePath->Type == MEDIA_VENDOR_DP) && (FileDevicePath->SubType == MEDIA_FILEPATH_DP)) {
      StartupScriptPath = StrnCatGrow (&StartupScriptPath, &Size, ((FILEPATH_DEVICE_PATH *)FileDevicePath)->PathName, 0);
      PathRemoveLastItem (StartupScriptPath);
    } else {
      StartupScriptPath = StrnCatGrow (&StartupScriptPath, &Size, L"\\", 0);
    }

Expected Behavior

The device path type should be MEDIA_DEVICE_PATH and the subtype should be MEDIA_FILEPATH_DP.

Steps To Reproduce

Code review

Build Environment

N/A

Version Information

Commit: 5637d61fa49747c0ccb4b5a77ef0959cae3176f4

Urgency

Medium

Are you going to fix this?

Someone else needs to fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: EDK Build Does Not Give an Error on Invalid Library Class Mapping

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

A single library instance for a given library class (as defined in the instance INF file) can be mapped to other library class strings in a DSC and an error is not raised.

Expected Behavior

If a library instance with a well-defined library class, as defined in the INF file, is mapped to a different library class in a DSC, a user visible error (or warning) should be raised.

Steps To Reproduce

1. Find a module in a DSC file
2. Add a <LibraryClasses> section
3. Change the class name for a library used by the module to an invalid name

---

Note: I haven't experimented with this, but I'd expect a higher scoped library class assignment needs to resolve correctly so the build finds a library to link so the build works.

For example, the library class is correctly specified in the overall [LibraryClasses] section of the DSC but invalid in a more specific override section (like the <LibraryClasses> override area).

Build Environment

- OS(s): Windows 11 (tested)
- Tool Chain(s): VS2022 (tested)
- Targets Impacted: All

Version Information

Branch: release/202208

Urgency

Medium

Are you going to fix this?

Someone else needs to fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug] VS2022 ARM64 build is broken due a linker error in DxeCore

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

Compile the Hyper-V UEFI code using the latest VS2022. It gives a linker error

INFO - "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.35.32215\bin\Hostx86\arm64\lib.exe" /NOLOGO /LTCG /OUT:d:\a\1\s\Build\MsvmAARCH64\RELEASE_VS2022\AARCH64\MsGraphicsPkg\MsUiTheme\Pei\MsUiThemePpi\OUTPUT\MsUiThemePpi.lib @d:\a\1\s\Build\MsvmAARCH64\RELEASE_VS2022\AARCH64\MsGraphicsPkg\MsUiTheme\Pei\MsUiThemePpi\OUTPUT\object_files.lst
INFO - DxeCore.lib(HeapGuard.obj) : error LNK2001: unresolved external symbol memset
INFO - d:\a\1\s\Build\MsvmAARCH64\RELEASE_VS2022\AARCH64\MdeModulePkg\Core\Dxe\DxeMain\DEBUG\DxeCore.dll : fatal error LNK1120: 1 unresolved externals

Expected Behavior

Compiling and linking should work.

Steps To Reproduce

Compile Mu_Basecore for ARM64 with VS2022

Build Environment

- OS(s):Windows
- Tool Chain(s):VS2022
- Targets Impacted: RELEASE

Version Information

Release/202208

Urgency

Medium

Are you going to fix this?

Someone else needs to fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

Enable CLANGPDB for AARCH64 builds

Currently CLANGPDB is only supported for X86 & X64 builds. Because many of the ARM specific files are written in a GCC centric way, having AARCH64 support in clang would allow for more portable use of this codebase for AARCH64 builds.

I have done some experimenting and found the following issues so far.

1. Various .S files use the .type and .size assembly directives in the format for ELF images and so compiling for COFF format does not work.
2. OpenSSL tries to pull in Windows.h which is not found currently and maybe shouldn't be used.

Other issues I have solved locally and will put into a PR.

[Task]: Remove Mu delta in UnitTestFrameworkPkgHost.dsc.inc

Request Description

#354 introduced a Mu change to the UefiBootServicesTableLib in UnitTestFrameworkPkgHost.dsc.inc. The file should get back to parity with edk2 when possible.

Packages consuming the DSC include file should ideally override libraries in a [LibraryClasses.common.HOST_APPLICATION] section since that is what is used in the include DSC file and doing so will allow the package to override libraries specified in the include file.

Are you going to make the change?

Someone else needs to make the change

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: Cannot generate overridevalidation for specific file not contained in a package path

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

`

python MU_BASECORE\BaseTools\Plugin\OverrideValidation\OverrideValidation.py -w . -t .\MU_BASECORE\BaseTools\Conf\tools_def.template
`
generates

Traceback (most recent call last): File "MU_BASECORE\BaseTools\Plugin\OverrideValidation\OverrideValidation.py", line 695, in <module> rel_path = Paths.TargetPath[Paths.TargetPath.find(pkg_path):] TypeError: must be str, not NoneType

Because pathtool.GetContainingPackage(Paths.TargetPath) returns None since the file is not contained in a Pkg.

Expected Behavior

Expected behavior would be to return a override string, similar to the below

#Override : 00000002 | BaseTools/Conf/tools_def.template | 9d937cc53fc946b30252a6d2646c439c | 2023-02-08T23-13-48 | e1b1a48

Steps To Reproduce

In a project, run the following to observe the error

python MU_BASECORE\BaseTools\Plugin\OverrideValidation\OverrideValidation.py -w . -t .\MU_BASECORE\BaseTools\Conf\tools_def.template

Build Environment

- OS(s): Windows
- Tool Chain(s): n/a
- Targets Impacted:

Version Information

edk2-pytool-extensions 0.20.1
edk2-pytool-library    0.14.0

Urgency

Low

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

202208 Branch Changes section in ReadMe.rst needs to be updated

These sections need to be filled in - https://github.com/microsoft/mu_basecore/blob/release/202208/ReadMe.rst:

Branch Changes - release/202208

Breaking Changes-dev
None

Main Changes-dev
None

Bug Fixes-dev
None

2208_RefBoot Changes
Incomplete

[Feature]: Uncrustify settings enforcing no black line after a conditional

Feature Overview

A lot of times the code line following a conditional is directly tied to the outcome of the check and it makes sense not to have blank lines inserted.

But if the next line is starting an unrelated task that has it's own comments, not having a blank line to help highlight the comments can make readability harder.

For instance, this code snippet is the original prior to running uncrustify:

 1:  // On error, skip writing the data
 2:  if (!EFI_ERROR (Status)) {
 3:
 4:    // Write depex as a series of hex bytes
 5:    for (x = 0; x < DepexSize; x++) {
 6:      MessageAscii (MsgBuffer, "%02X", Depex[x]);
 7:    }
 8:
 9:    // Free memory provided by the ReadSection call
10:    FreePool (Depex);
11:  }

Running uncrustify removes line 3, pushing the comment directly up under the IF statement making the comment hard to distinguish:

 1:  // On error, skip writing the data
 2:  if (!EFI_ERROR (Status)) {
 4:    // Write depex as a series of hex bytes
 5:    for (x = 0; x < DepexSize; x++) {
 6:      MessageAscii (MsgBuffer, "%02X", Depex[x]);
 7:    }
 8:
 9:    // Free memory provided by the ReadSection call
10:    FreePool (Depex);
11:  }

Solution Overview

Can we have the uncrustify.cfg file updated to remove the enforcement of no blank line after a conditional?

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

Maintainer feedback requested

Anything else?

No response

[Bug]: CodeQL Build Plugin Linux Error

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

The following error is observed if running the CodeQL plugin (--codeql) on a Linux host:

AttributeError: property 'suffix' of 'PosixPath' object has no setter

Expected Behavior

The plugin should run successfully.

Steps To Reproduce

1. Open mu_basecore in the Fedora 37 Dev container
2. Update pip modules per pip-requirements.txt
3. Run stuart_update -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 --codeql
4. Run ``stuart_ci_build -c .pytool/CISettings.py TOOL_CHAIN_TAG=GCC5 -p MdePkg -t DEBUG --codeql`

Error should appear.

Build Environment

- OS(s): Fedora 37
- Tool Chain(s): GCC5
- Targets Impacted: DEBUG, RELEASE

Version Information

Tag: v2022080001.2.0

Urgency

Low

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: Support stuart building on Windows ARM systems

Feature Overview

With better support of Windows ARM ecosystem, more dependencies required by stuart build there were not available a few years ago are ready for consumption. This issue is created to support native stuart build on Windows ARM systems.

Solution Overview

The main pieces of this support will include:

1. EDK2 pytool change to support VS discovery on ARM systems
2. BaseTools change to support VS toolchain query and native builds of BaseTools
3. IASL builds inside ARM environments
4. Reuse of x86 version of nasm (at least for now)

The changes will be validated on with integration test on Q35 and SBSA virtual platforms.

Alternatives Considered

No response

Urgency

Medium

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: AARCH64 LD Can Misinterpret Multiple common-page-size Parameters

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

It has been observed that on aarch64 package builds, GenFw can fail at this line:

mu_basecore/BaseTools/Source/C/GenFw/Elf64Convert.c

Line 1540 in 1bd4b3b

Error (NULL, 0, 3000, "Invalid", "WriteSections64(): %s AARCH64 small code model requires identical ELF and PE/COFF section offsets modulo 4 KB.",

.

What appears to be happening is that ld is interpreting the multiple common-page-size parameters passed to it in a different order, sometimes at 0x20, sometimes at 0x1000, which breaks GenFw's expectation of alignment. It does not appear to be valid to pass common-page-size to ld multiple times.

There are complicating factors: the same toolchain will work on the same code, then it will break the next day, with no code changed. Manually editing tools_def.txt (or tools_def.template and regenerating tools_def.txt) to set all common-page-size calls for AARCH64 GCC to 0x1000 causes the issue to go away, however, on a system that is not experiencing the issue, setting all the common-page-size parameters to 0x20 does not repro the issue.

The issue does repro occasionally across multiple machines, both local dev boxes of different devs (and on different packages) and on pipelines. So far, this has only been observed in non-public code; I am attempting to get a public repro, however, with the unpredictability of getting a repro, I have been unsuccessful so far.

At the very least, not passing common-page-size multiple times would seem to be a step in the right direction, although further investigation here seems warranted.

Expected Behavior

I expect GenFw to not fail and properly be able to convert ELF files to PE/COFF.

Steps To Reproduce

Unfortunately, this is not easily reproducible, but the signature is clear.

Build Environment

- OS(s): Linux
- Tool Chain(s): GCC
- Targets Impacted: AARCH64

Version Information

Version v2022080001.4.2

Urgency

High

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: Split AssertLib Functionality From DebugLib

Feature Overview

Today, assert functionality lives inside of DebugLib, which is mixing two separate features into one library and requires every DebugLib implementation to change in response to changes to assert functionality.

Solution Overview

A cleaner and more maintainable architecture would be to split the assert functionality into an AssertLib with a well defined interface that can be consumed by DebugLib.

Alternatives Considered

No response

Urgency

Medium

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: UefiDevicePathLib Should Not Support PEIMs

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

MdePkg/Library/UefiDevicePathLib/UefiDevicePathLib.inf had PEIM added to the LIBRARY_CLASS list as of commit acf286f.

This library class instance includes code such as that in DevicePathUtilitiesDxeSmm.c which directly links against gBS which is provided by UefiBootServicesTableLib and not available in PEI.

Expected Behavior

UefiDevicePathLib should be refactored for PEIM support or drop PEIM from the support library class types.

Steps To Reproduce

Review the module.

Build Environment

- OS(s): All
- Tool Chain(s): All
- Targets Impacted: All

Version Information

Commit: acf286f

Urgency

Low

Are you going to fix this?

Someone else needs to fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

BmFindBootOptionInVariable() does not check if input pointer is NULL

From MdeModulePkg\Library\UefiBootManagerLib\BmBoot.c

UINTN
BmFindBootOptionInVariable (
  IN  EFI_BOOT_MANAGER_LOAD_OPTION  *OptionToFind
  )
{
  EFI_STATUS                    Status;
  EFI_BOOT_MANAGER_LOAD_OPTION  BootOption;
  UINTN                         OptionNumber;
  CHAR16                        OptionName[BM_OPTION_NAME_LEN];
  EFI_BOOT_MANAGER_LOAD_OPTION  *BootOptions;
  UINTN                         BootOptionCount;
  UINTN                         Index;

  OptionNumber = LoadOptionNumberUnassigned;

  //
  // Try to match the variable exactly if the option number is assigned
  //
  if (OptionToFind->OptionNumber != LoadOptionNumberUnassigned) {
    UnicodeSPrint (
      OptionName,
      sizeof (OptionName),
      L"%s%04x",
      mBmLoadOptionName[OptionToFind->OptionType],
      OptionToFind->OptionNumber
      );

OptionToFind should be checked for being NULL before being dereferenced in the function.

Found in #182

[Bug]: New linker failures observed with Visual Studio 17.5

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

No linker issues when building with VS2022 toolchain 17.4.

Expected Behavior

17.5 does not have linker regressions to 17.4.

Steps To Reproduce

1. Install Visual Studio 17.5.1.
2. Perform a clean build of the code at b4a47df (current tip of release/202208)
3. A linker error should result

Build Environment

- OS(s): Windows 11 (reproduced on 10.0.20348 and 10.0.22623)
- Tool Chain(s): VS2022
- Targets Impacted: DEBUG, RELEASE, NOOPT

Version Information

Commit: b4a47dfdec9603eb824b1cb962835ea06c16f201

Urgency

High

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

Appears related to the update from Visual Studio 17.4 to 17.5 (not OS version or a particular commit).

[Feature]: BaseTools/OverrideValidation.py: Do not print override mismatches in log

Feature Overview

OverrideValidation.py currently prints track tag mismatches in the build log.

Depending on the number of track tags, the output can be verbose and confusing to those that are unaware track tags may not be found.

Example of output for a track tag not satisfied:

INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - fatal: bad object 72230d089f08059b7ebbed4e19cb3bcc80b637d6
INFO - ------------------------------------------------
INFO - --------------Cmd Output Finished---------------
INFO - --------- Running Time (mm:ss): 00:00 ----------
INFO - ----------- Return Code: 0x00000080 ------------
INFO - ------------------------------------------------
INFO - Override diff since last update at commit 72230d089f08059b7ebbed4e19cb3bcc80b637d6
INFO - At Line 14: #Track : 00000002 | MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf | 38e88ecb08567395e2ea74b980835ac0 | 2023-02-24T16-23-58 | 72230d089f08059b7ebbed4e19cb3bcc80b637d6
INFO - Inf Override Hash Error: MISMATCH, expecting 29ef137293a9c0deccef93e1b5b73afcc63a7a3b, has 363e254ecb2e8334d7efdbe363172897
INFO - Cmd to run is: git diff 3305879b26b74402e43cbd0abf8b69ff73912421 D:\a\1\s\MU_BASECORE\MdeModulePkg\Core\PiSmmCore\PiSmmIpl.inf

Solution Overview

Make the user aware of two outcomes involving a track tag:

1. If a track tag is not found for a given resource (e.g. INF file) at all, write an ERROR level message indicating that no track tags resolved.
2. If a track tag is found for a given resource, only write information about unresolved track tags at DEBUG level.
   • Since there is no action a normal user needs to take in this case.

Alternatives Considered

Leave as-is. Messages can be confusing to those debugging logs without extensive background in how OverrideValidation works.

Urgency

Low

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

Maintainer feedback requested

Anything else?

No response

[Feature]: Add ability to indefinitely retry boot options in the system.

Feature Overview

Add ability to enable the Bds phase to indefinitely retry all EFI_LOAD_OPTIONS without forcing the user into a setup utility.

Solution Overview

Feature should only be available only for platforms that need it.
Ideal scenario is a PCD that is default disabled. When BootBootOptions() is called, should stay inside of function retrying boot options, to prevent unloading and reloading EfiLoadOptions from memory.

Alternatives Considered

Investigated trying to implement through PlatformBootManagerLib or UefiBootManagerLib, but the high level calls for handling LoadOptions are all in BdsEntry.

Urgency

Low

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: Keyboard and Mouse Fail to Work on KBL

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

The following change causes loss of device functionality (USB keyboard and mouse at FW front page) on KBL:

7f4eca4

Expected Behavior

Device functionality should not be regressed.

Steps To Reproduce

1. Build a KBL platform with 7f4eca4
2. Boot to a firmware UI
3. Try to use USB keyboard and mouse

Build Environment

- OS(s): Windows 11
- Tool Chain(s): All
- Targets Impacted: DEBUG, RELEASE

Version Information

Commit: 7f4eca4cc2e01d4160ef265f477f9d098d7d33df

Urgency

High

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: Provide a hook API when stack overflow occur with stack protection enabled

Feature Overview

[Background]
Currently we have EFI_CPU_ARCH_PROTOCOL.RegisterInterruptHandler() for DXE_DRIVER and EFI_SMM_CPU_SERVICE_PROTOCOL.RegisterExceptionHandler() for DXE_SMM_DRIVER to register EXCEPT_IA32_STACK_FAULT type that can perform some customized actions when stack overflow occurred with stack protection enabled, but it has a dependency that the customized code can only get performed after these protocols get installed, it is the gap we want to resolve.

[Feature request]
When stack overflow occurred in any protected modules types(ex: DXE_DRIVER, DXE_CORE, SMM_CORE, DXE_SMM_DRIVER...), it should provide a hook API or mechanism that allow platform to add customized code (ex: report the error to somewhere as telemetry), the customized should not have dependency on protocols.

Solution Overview

Below is just a proposal for reference, it is ok to have any other kind of methods to achieve it.
[Proposal]
Implement a MU version of BaseStackCheckLib instance and implement below code in BaseStackCheckMsft.c, use MSFT VS compiler to compile this C code that has an customized hook API inside __report_gsfailure.

NORETURN VOID __cdecl
__report_gsfailure (
  void
  )
{
  DEBUG ((DEBUG_ERROR, "(__report_gsfailure) STACK PROTECTION : Violation detected\n"));

  CustomizedHook ();  // API provided by another new added library class
  CpuDeadLoop ();
}

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

Maintainer feedback requested

Anything else?

No response

[Feature]: Support MdePkg & MdeModulePkg AARCH64 Visual Studio compilation

Feature Overview

Current CI build commands:

Tool Chain	Package / Build Command
	MdeModulePkg
GCC5	stuart_ci_build -c .pytool/CISettings.py -p MdeModulePkg -t DEBUG,NOOPT -a IA32,X64,ARM,AARCH64 TOOL_CHAIN_TAG=GCC5 CODE_COVERAGE=TRUE CC_HTML=TRUE
VS2022	stuart_ci_build -c .pytool/CISettings.py -p MdeModulePkg -t DEBUG,NOOPT -a IA32,X64 TOOL_CHAIN_TAG=VS2022
	MdePkg
GCC5	stuart_ci_build -c .pytool/CISettings.py -p MdePkg,UefiCpuPkg -t DEBUG,RELEASE,NO-TARGET,NOOPT -a IA32,X64,ARM,AARCH64 TOOL_CHAIN_TAG=GCC5 CODE_COVERAGE=TRUE CC_HTML=TRUE
VS2022	stuart_ci_build -c .pytool/CISettings.py -p MdePkg,UefiCpuPkg -t DEBUG,RELEASE,NO-TARGET,NOOPT -a IA32,X64 TOOL_CHAIN_TAG=VS2022

Request is to update VS2022 row commands to include AARCH64.

• Note: Other packages would be a bonus but MdeModulePkg and MdePkg are key downstream dependencies.
• Note: ARM could also be supported but the request for AARCH64 is prioritized.

Solution Overview

Enabling support might require changes within the package or in out of package tools.

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

If Project Mu consumers are using VS2022 for ARM builds (they are), we need to catch issues like #307 in this repo first.

[Feature]: Introduce a callback mechanism for policies beyond protocol/PEIM publish

Feature Overview

Issues raised by @os-d & @kuqin12, the policy service currently has limited ability to notify simple consumers of a finalized policy. They would have to set up a notification and check each time if the policy is finalized. This is unintuitive and complicated for consumers at the benefit of making the middle providers more complicated. It also overloads the use of the protocol/PEIM publishing. The other existing solution would be to use a different GUID for the finalized vs non-finalized policy but that is difficult to maintain as it requires a strict ordering for potentially generic components and is difficult to trace.

Solution Overview

The solution would be to make the following changes.

1. Only publish the PEIM/Protocol when the policy is finalized to simplify end consumers.

2. Introduce a proper callback system to allow interim publishers to get notified by non-finalized policies.

Alternatives Considered

We also discussed keeping the protocol publishing on initial policy availability, but this adds complexity to the end consumers which should ideally be as simple as possible.

Urgency

Medium

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

Support for CLANGPDB in AARCH64

To allow for more portable compiling and support using COFF based debug and development tools, support for CLANGPDB in AARCH64 would be largely beneficial.

I have experimented and got this working for MdeModulePkg but with a fair number of hacks, so far there are a following issues.

1. The toolsdef does not contains definitions for CLANGPDB in AARCH64
2. Many of the .S files and macro definitions use the ELF format of the .type and .size assembly directives
3. Various compiler warning issues.

The commit for this experiment can be found here: cfernald@9587e16

[Feature]: Improve USB descriptor robustness

Feature Overview

Commit 714d41b, stopped USB enumeration in the case a malformed descriptor is found. This change was made because access exceptions occurred in some cases when attempting to communicate with the downstream device because the driver has invalid interface descriptor structures associated with the device.

The change has been found to regress end-user functionality of some USB devices.

Solution Overview

An exact solution is not being proposed at the moment. This feature request tracks a more robust solution that may still allow broader device compatibility while preventing the subsequent access violation issues the original change intended to solve.

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: Supporting stuart build for Linux ARM

Feature Overview

Related to #305, this feature request is to add support for stuart build on Linux ARM, which involves building binaries for external dependencies and pipeline setup.

Solution Overview

1. Adding basetool builds for base core.
2. Updating pipeline to run Linux ARM builds.

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

I will implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: NvmExpressDxe and NvmExpressPei are missing 'volatile' for pointer to struct located in memory updated by Nvme controller

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

From NvmExpressDxe\NvmExpressPassthru.c, function NvmExpressPassThru:

  while (EFI_ERROR (gBS->CheckEvent (TimerEvent))) {
    if (Cq->Pt != Private->Pt[QueueId]) {
      Status = EFI_SUCCESS;
      break;
    }
  }

Before this 'while' loop, the code has initiated an NVMe transaction via a submission queue.
The 'while' loop is looking for a state change of 'Cq->Pt' indicating that the transaction is complete.

'Cq' is a pointer to 'NVME_CQ' which is a structure that is updated by the Nvme controller. This structure should have the 'volatile' keyword to ensure that the memory is re-read within the 'while' loop.
Without 'volatile', the compiler is only reading the 'Cq->Pt' once outside of the 'while' loop and then doing register compares within the 'while' loop.

It has been observed (on AARCH64 target hardware) that in some cases, 'Cq->Pt' has not yet been updated by the NVMe controller by the time the code reaches the 'while' loop. In those cases, the 'while' loop will only terminate due to timeout because the external memory is no longer being read within the 'while' loop.

A similar code construct is in NvmExpressPei\NvmExpressPassThru.c and there may be other areas where 'volatile' is missing.

Expected Behavior

Using 'volatile' with declarations of pointer to NVME_CQ results in code that re-reads the memory within the 'while' loop ensuring that the updated state of 'Cq->Pt' is detected.

Steps To Reproduce

The problem is intermittent and may depend on underlying hardware and toolchain optimization flags.

Issue any commands/procedures to read from an NVMe drive.
Observe NVMe command timeout errors through logs and addition of debug prints in NvmExpressPassThru.c.
Code inspection and analysis of assembly listing of execution binaries with and without 'volatile' keyword.

Build Environment

- OS(s): Windows 11 WSL2
- Tool Chain(s): GCC5
- Targets Impacted: ALL

Version Information

Release/202208

Urgency

Medium

Are you going to fix this?

Someone else needs to fix it

Do you need maintainer feedback?

Maintainer feedback requested

Anything else?

There may be other modules besides NvmExpressDei and NvmExpressPei that are missing the 'volatile' keyword.

Openssl disclosed vulnerabilities

Summary

Context: There was a set of new OpenSSL CVEs publicly announced on 07 February 2023. Of the 8 new CVEs announced, 4 affect the OpenSSL 1.1.1 branch.

Details

CVE-2023-0286 X.400 address type confusion in X.509 GeneralName [HIGH severity]

CVE-2023-0215 Use-after-free following BIO_new_NDEF [MODERATE severity]

CVE-2022-4450 Double free after calling PEM_read_bio_ex [MODERATE severity]

CVE-2022-4304 Timing Oracle in RSA Decryption [MODERATE severity]

Impact

CVE-2023-0286: This vulnerability may allow an attacker to pass arbitrary pointers to a memcmp call, enabling them to read memory contents or enact a denial of service

CVE-2023-0215: Use-after-free with internal pointers after certain conditions occur.

CVE-2022-4450: Can lead to a crash.

CVE-2022-4304: Could be sufficient to recover a plaintext across a network in a Bleichenbacher style attack.

[Bug]: SMMUv3 IORT HTTU Override flag needs to be expanded

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

The flag for HTTU override in an SMMUv3 node in the IORT table is currently define in MU_BASECORE/MdePkg/Include/IndustryStandard/IoRemappingTable.h:63 as:

#define EFI_ACPI_IORT_SMMUv3_FLAG_HTTU_OVERRIDE     BIT1

Expected Behavior

The length of this field is actually 2 bits. Possible values are:

0b0000: Hardware update of the Access flag and dirty state are not supported.
0b0001: Support for hardware update of the Access flag for Block and Page descriptors.
0b0010: As 0b0001, and adds support for hardware update of the Access flag for Block and Page descriptors. Hardware update of dirty state is supported.

For more info see:

• Arm System Memory Management Unit Architecture Specification
• Hardware Updates to Access Flag and Dirty State in the Armv8.1-A architecture

Steps To Reproduce

N/A

Build Environment

N/A

Version Information

v2023020002.1.4

Urgency

Low

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

Maintainer feedback requested

Anything else?

No response

[Bug]: MAT Logic Incorrectly Reports Runtime Images

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

To create the MAT, we call into a function CoreGetMemoryMapWithSeparatedImageSection() which will get the memory map (and thus its required size) but update that size to be the size of the map plus the amount of extra space necessary to break up the memory map. The extra space in this case is indicated by a field called CodeSegmentMax which is set to be the most number of code segments found in a single image loaded. Let’s say we found an image with 2 code sections, making CodeSegmentMax = 2. With two code segments, here are a couple of ways an image could be laid out:

Image 1: Data | Code | Data | Code | Data
Image 2: Data | Code | Code | Data

So, if a single memory map descriptor is the same address range as a singe image, that one map descriptor could become at most 5, leading to the calculation (2 * CodeSegmentMax + 1) * NumberOfImageRecords.

After returning the required pool size (MemoryMapSize) the next call will be with a pool of that size. When we call into get the memory map, MemoryMapSize will be updated to be the actual size of the memory map and we make sure that size is what we expect before calling into SplitTable(). Within SplitTable(), MemoryMapSize is equal to the size of the actual memory map fetched, but the buffer containing the memory map is much larger as described above. We set IndexOld to be the last descriptor in the memory map fetched from CoreGetMemoryMap() and IndexNew to the last descriptor in the entire buffer. We work our way backwards through the descriptors generated from CoreGetMemoryMap() and write new descriptors to the end of the entire buffer. We call into GetMaxSplitRecord() which calculates how many images might be contained in a single memory map descriptor. Let’s say one descriptor range maps to one loaded image which has 2 code sections, so GetMaxSplitRecord() will return 4 (we subtract 1 because IndexNew is already pointing to a valid descriptor).

Let’s zoom into those 5 currently empty descriptors:

| | | | | |
^
IndexNew

Let’s say the code/data sections are laid out like Image 2 shown above. We call into SplitRecord() which will get the image covering the original descriptor range then call into SetNewRecord() to start populating the descriptors. There’s a stray portion of SplitRecord() which makes sure there isn’t any lingering memory that hasn’t been covered by one of the new descriptors which we can ignore in this case. In the case of #2 above, we will write 4 descriptors but only return the value 3.

When we get back to SplitTable(), MaxSplitRecordCount = 4 and RealSplitRecordCount = 3 and those 5 descriptors look like this:

| d1 | c1 | c2 | d2 | |
^
IndexNew

We then do a mem copy with a destination of IndexNew + MaxSplitRecordCount – RealSplitRecordCount = IndexNew + 1 and a source of IndexNew and a length of 3 which would result in the following:

| d1 | d1 | c1 | c2 | |
^
IndexNew

And this is clearly not right because we have a duplicated the first data section and overwritten the last data section. Index 0 will be overwritten, but Index 4 is now a garbage memory descriptor which will be captured when we copy all the new records to the front of the buffer and update memory map size to be the new number of entries.

Expected Behavior

The expected MAT Table output is defined in the UEFI Spec

Steps To Reproduce

Load at least one image with more than one code section. Debug print the entire memory map output by SplitTable() and there will be at least one garbage entry and at least one missing entry.

Build Environment

- OS(s): ANY
- Tool Chain(s): VS2022 or VS2019
- Targets Impacted: ANY

Version Information

Commit: 3e1cb8d

Urgency

High

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Bug]: False negative on Google Tests death test

Is there an existing issue for this?

• I have searched existing issues

Current Behavior

When performing death tests, the outcome is inconsistent and could have false negative results.

A detailed unit test usage is outlined in #393:

One test would fail silently on an access violation, although that due to a unit test bug. To prevent this, one should always populate the matching string. However, such string is not connected to the existing DebugLib.h implementation.

Expected Behavior

A detailed unit test usage is outlined in #393.

But to mitigate the false negative unit tests, we should properly support the asserts for Google Test usage.

Steps To Reproduce

See #393 for full code.

Build Environment

- OS(s): Linux and Windows
- Tool Chain(s): VS2022 and GCC11
- Targets Impacted: x64

Version Information

Top of release/202208

Urgency

Medium

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response

[Feature]: xdrlib deprecation and removal in python 3.13

Feature Overview

Python 3.11.3 has started to report warnings in regards to the xdrlib.

MU_BASECORE\BaseTools\Scripts\BinToPcd.py:16: DeprecationWarning: 'xdrlib' is deprecated and slated for removal in Python 3.13

This needs to be addressed prior to 3.13 release, and we need to be aware that this will may cause issues supporting older code-bases on newer python tools.

Solution Overview

BinToPcd.py is the only xdrlib usage, currently.

Alternatives Considered

No response

Urgency

Low

Are you going to implement the feature request?

Someone else needs to implement the feature

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

No response