trailofbits / rpcinvestigator

Exploring RPC interfaces on Windows

RPC Investigator

Please read our blog post announcement.

RPC Investigator (RPCI) is a .NET/C# Windows Forms UI application that provides an advanced discovery and analysis interface to Windows RPC endpoints. The tool provides a visual interface around the existing core RPC capabilities of the NtApiDotNet platform, including:

• Enumerating all active ALPC RPC servers
• Parsing RPC servers from any PE file
• Parsing RPC servers from processes and their loaded modules, including services
• Pulling symbol information from a Symbol Server
• Exporting RPC server definitions as serialized .NET objects for your own scripting

Beyond these core features, RPCI provides additional capabilities:

• The Client Workbench allows you to create and execute an RPC client binary on-the-fly by right-clicking on an RPC server of interest. The workbench has a C# code editor pane that allows you to edit the client in real time and observe results from RPC procedures executed in your code.
• Discovered RPC servers are organized into a searchable library, allowing you to pivot RPC server data in useful ways, such as searching all RPC procedures for all servers for interesting routines through a customizable search interface.
• The RPC Sniffer tool adds visibility into RPC-related ETW data to provide a near real-time view of active RPC calls. By combining ETW data with RPC server data from NtApiDotNet, we can build a more complete picture of ongoing RPC activity.

Common Workflows

There are several workflows that the RPC Investigator supports:

• Auditing
  • Enumerating all active ALPC RPC servers across all processes that are communicating with an ALPC endpoint
  • Enumerating all RPC servers running in a Windows service
  • Loading offline RPC servers defined in a PE file (such as an EXE or DLL)
• Interactive
  • Client Workbench: Automatically generate RPC client code that can be customized and used to call into any RPC service.
  • RPC Sniffer: Realtime monitor of RPC-related Event Tracing for Windows (ETW) data.

Example Workflow: Analyzing the Task Scheduler RPC

In this example, we'll be inspecting the Windows Task Scheduler RPC service, which is used to manage and execute scheduled tasks. We'll find the service, generate client code, and then customize the client to interact with one of the exposed procedures.

First, load the Windows services list by clicking File -> Load From Service. This opens a new service list window:

Find the Schedule service, which is the Windows Task Scheduler, select the service and click Go.

You will be prompted prior to RPCI loading all associated RPC DLLs. Click Yes to continue. Once loaded, you will see a list of all RPC servers discovered across all modules loaded in the service process. The Windows Task Scheduler RPC server has an Interface ID of 86D35949-83C9-4044-B424-DB363231FD0C. Find the row within the list that has this Interface ID, which should have a running service named Task Scheduler, right-click on the row and select New Client.

The left portion of the client window shows RPC server metadata and command line output from the client code. The right side shows two tabs:

• Client Code - Auto generated C# client code that can be customized to interact with one or more procedures.
• Procedures - List of exposed RPC procedures.

In this example we'll be calling the SchRpcHighestVersion procedure. This method accepts a single argument, out int version, which, after calling the procedure, will contain the highest Task Scheduler protocol version supported by the RPC interface. The high 16-bits are the major version and the low 16-bits are the minor version.

To call this procedure:

1. In the Client Code window, find the Run method, which is the main entry point for the RPC client.

2. Edit the Run method body to call the procedure:

   public async Task<bool> Run()
   {
       int version;
       int status = SchRpcHighestVersion(out version);
       if (status == 0) {
           long major = (version & 0xffff0000) >> 16;
           long minor = version & 0x0000ffff;
           Console.WriteLine("highest supported RPC version: {0}.{1}", major, minor);
       } else {
           Console.WriteLine("call to SchRpcHighestVersion failed with error: {0:X}", status);
       }
       return true;
   }

3. After adding this code, run the client by clicking the Run button. This will compile the C# code and then execute the Run method.

   • You will see a popup box with any compilation errors if the client code could not be compiled.

If compilation is successful, you will see something similar to the following in the Output box:

> Run() output:
highest supported RPC version: 1.6

Configuration

The Rpc Investigator has several configuration settings.

Setting	Description	Default
dbghelp.dll	File location of the dbghelp.dll module	Find latest version within installed Windows Kits.
Symbol Path	Path to Windows symbols, which can be a symbol server or local directory	Default public Windows Server: srv*c:\symbols*https://msdl.microsoft.com/download/symbols
Trace Level	The logging trace level	info

The configuration settings can be modified within the application through the Edit -> Settings menu.

Troubleshooting

After its initial release, RPC Investigator was converted from a .NET Framework application to a .NET 7 application. If you run into build issues, make sure Visual Studio is up-to-date. Wiping all build output folders prior to building from the .NET Framework version is a good idea.

Also, due to the move from the insecure BinaryFormatter class to protobuf-net, RPC libraries generated with the .NET framework version are incompatible with the .NET version.

If you're experiencing random crashes in RPC Investigator, you might find a solution in asking your administrator to tweak your EDR. We have found that some EDRs do not behave sanely with JIT'ed languages.

Development Environment

1. Install Visual Studio Community 2022, make sure to select the .NET Desktop Development workflow.
2. Download and install the latest Windows 10 SDK. Perform a full installation so that .NET 4.8.1 and Debugging Tools are installed.
3. Open the Solution and verify that the projects loaded correctly. If there is an error about missing .NET 4.8.1 Targeting Pack, download and install .NET Framework 4.8.1 Developer Pack and then restart Visual Studio.

Further Reading

Because Windows RPC has been a popular research topic for well over a decade, there are too many related resources and research efforts to name here. We've listed a few below that we encountered while building this tool:

• https://clearbluejar.github.io/posts/surveying-windows-rpc-discovery-tools/
• https://www.powerofcommunity.net/poc2019/James.pdf
• https://www.tiraniddo.dev/2022/06/finding-running-rpc-server-information.html
• https://clearbluejar.github.io/posts/from-ntobjectmanager-to-petitpotam/
• https://itm4n.github.io/from-rpcview-to-petitpotam/
• https://learn.microsoft.com/en-us/windows/win32/rpc/rpc-security-essentials
• https://www.cyberark.com/resources/threat-research-blog/understanding-windows-containers-communication
• https://github.com/silverf0x/RpcView
• https://github.com/xpn/RpcEnum
• https://github.com/cyberark/RPCMon
• https://github.com/tyranid/WindowsRpcClients

If you're unfamiliar with RPC internals or need a technical refresher, we would recommend one of the authoritative sources on the topic - Alex Ionescu's 2014 SyScan talk in Singapore, All about the RPC, LRPC, ALPC, and LPC in your PC.

rpcinvestigator's Issues

Miscellaneous Release Build Issues

I've been using the RPC sniffer in debug mode without issues and tried using it in release mode. I started recording events and then the app crashed with the following error in the event log

Faulting application name: RpcInvestigator.exe, version: 1.0.0.0, time stamp: 0x91e5952b
Faulting module name: sechost.dll, version: 10.0.22621.608, time stamp: 0x45764b57
Exception code: 0xc0000005
Fault offset: 0x000208a6
Faulting process id: 0x0x2A0
Faulting application start time: 0x0x1D90977038EF350
Faulting application path: C:\Users\Adam\Projects\RpcInvestigator-master\bin\Release\RpcInvestigator.exe
Faulting module path: C:\Windows\System32\sechost.dll
Report Id: 892f9a7b-e2d5-4968-9b79-cb1a3b4cdd08
Faulting package full name: 
Faulting package-relative application ID: 

Application: RpcInvestigator.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.AccessViolationException
   at RpcInvestigator.Util.NativeTraceControl.EnableTraceEx2(IntPtr, System.Guid ByRef, EventControlCode, EventTraceLevel, UInt64, UInt64, UInt32, IntPtr)
   at RpcInvestigator.Util.EtwRealTimeTrace.Start()
   at RpcInvestigator.Windows.Sniffer+<>c__DisplayClass18_0.<startButton_Click>b__0()
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   at System.Threading.Tasks.Task.ExecutionContextCallback(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef)
   at System.Threading.Tasks.Task.ExecuteEntry(Boolean)
   at System.Threading.Tasks.Task.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

I was prompted to run the app as administrator, which I allowed, and I tried again by explicitly running it as administrator. This is happening consistently in release mode.

Setup and configuration instructions / automation

I'm setting up and using Rpc Investigator for the first time, so I'll keep track of what I'm doing and update the README with the instructions. I'll also be looking into automating the process of locating the most recent version of the dbghelp.dll module from installed Windows Kits (v10).

Add ability to specify connection security

Currently, anywhere in the tool where you connect to an endpoint, the default transport security is used. Add a setting that will allow the user to specify an SDDL string. Also, anytime an ALPC port is the target, we should be using the SDDL from RpcAlpcServer.SecurityDescriptor

Incorrect NtApiDotNet.dll search path

The client expects to find NtApiDotNet.dll in System32 instead of inside the RpcInvestigator directory and fails to execute client code.

Create UnitTest with prebuilt ETL files

To create test coverage for more exotic ETW event types (such as those with properties that are arrays), we should create some ETL captures containing those events and then wrap a unit test around it. Use Perfview to create the ETL file, save it to a folder in UnitTests directory, and then subclass the EtwEventParser class to operate on the ETL file.

Make ellipsis node expandable in Sniffer graph view

Empty search disables right-click context menu

Repro:

• In any searchable tab, right-click and enter a search term that causes 0 results
• The right-click context menu is disabled

Integrate RPC ETW event capture

We'd like to be able to consume RPC related ETW events in the tool and then tie them up to known RPC server information inside the library. This could allow us to have an "RPC Wireshark"-like capability.

Convert project from .NET framework to .NET 7 standard

.NET framework has been frozen by Microsoft (no releases past 4.8) and the general guidance is to move away from it to what is now simply called ".net" (sorta like .net standard and .net core but... not)

Investigate ways to optimize memory consumption

Currently, running sniffer for a minute or two fills up a GB of memory.

Improve search experience

Currently search is implemented via right-click context menu on certain tabs and limited to keyword-search only. While I do feel like we should strive to keep the UI clean, simple and intuitive - ie, search is best integrated at points where the data is displayed and not in new windows or numerous UI controls - I think we could improve this.

Automated ALPC port connector/scanner

We could brute-force connect to any ALPC port reported by endpoint mapper, with options to specify security parameters (admin principal, anonymous, local service, etc). Going further, we could even automatically call some simple interface routines, such as those that require no parameters, and output the result.

A note here: I prototyped a scanner and it was problematic - the endpoint mapper will hang reliably on some RPC services (such as those owned by a suspended process). because you cannot safely force-terminate async tasks in .NET (see "cooperative cancellation"), and launching a native thread that you terminate from within a task can easily cause CLR stack corruption, we will need to do something cheesy like write results to a file and read them back in.

As part of this work, we should implement the capability of Get-AccessibleAlpcPort in https://github.com/googleprojectzero/sandbox-attacksurface-analysis-tools/blob/c02ed8ba04324e54a0a188ab9877ee6aa372dfac/NtObjectManager/RpcFunctions.ps1

Column selector clears captured data

Should allow to choose columns after data was captured without clearing the sniffer from the already-captured data.

Update readme and wiki docs

Add filtering options for RpcSniffer

On long captures the graph view can be very busy - add option to filter the results to only include certain processes or interfaces.

RPC sniffer - PID and TID should be in decimal

In the RPC sniffer, it would be helpful if PID and TID are shown in decimal, not hex, to be consistent with other views within the app. Also, I'm seeing a ProcessId and ProcessID column, is one of them a duplicate?

Implement Get-NtAlpcServer

see https://github.com/googleprojectzero/sandbox-attacksurface-analysis-tools/blob/c02ed8ba04324e54a0a188ab9877ee6aa372dfac/NtObjectManager/RpcFunctions.ps1

System.ArgumentException: Unable to sort because the IComparer.Compare()

Howdy!

Amazing tool, thanks for sharing!
I got the following exception when adding/removing columns to an already filtered (on IsServiceRunning field) servers list.

System.ArgumentException: 'Unable to sort because the IComparer.Compare() method returns inconsistent results. Either a value does not compare equal to itself, or one value repeatedly compared to another value yields different results. IComparer: 'System.Collections.Generic.ObjectComparer`1[BrightIdeasSoftware.ICluster]'.'

I couldn't reproduce it yet but I thought I report it anyway...

Rework column selector in Sniffer

• Column selector should also allow removal of default properties like Task, Keywords, etc
• Column selector is ignorant of properties that are struct or array types, some of which are incompatible as a column. for example, a debug RPC event has a property Params that is an array of structs with 4 members - we can't create columns for arbitrary array lengths without severely cluttering things. We might consider intelligently gluing those array entries together into a single column, but then that specializes how columns are presented to the user. This needs more thought.

Investigate "the stub received bad data" in Client Window

Improve use of OwnerDraw

The tab controls are owner-drawn to make them closeable, but a side effect is that control themes are disabled. This makes the control look more boxy and antiquated. Look into how to modernize this approach so that themes are preserved, or at least extend OwnerDraw methods to approximate standard Windows theme.

Investigate "access denied" issues

Example server: c:\windows\system32\lsm.dll,11f25515-c879-400a-989e-b074d5f092fe,1.0,HRESULT Proc1();

This could be a simple matter of "this is expected behavior" or perhaps we need to be connecting with a particular token (for an ALPC port, it would need to match the SDDL of the security descriptor).

Add unit test framework

Add unit test project and use MSTest to run them. Plug this into github's CI yaml

RPC sniffer - "open in library" confusion

In the RPC sniffer, I right-click a row and select "open in library". I was thinking that this would open the endpoint represented in the row I selected within the main window. I'm seeing that multiple servers are opened in the main window, not just the row I selected. This may be a confusion on my part.

Remove assumptions around endpoint mapper

There are places in the code, when we attempt to connect to an interface, that might fail if the particular server is not registered with the endpoint mapper. We should unwind these.

App Hang and crash

When I try to follow the example mentioned in the project -
https://github.com/trailofbits/RpcInvestigator#example-workflow-analyzing-the-task-scheduler-rpc

it hang and finally crash the app.

Improve code editor for new client

The control used is bare-bones, let's investigate a better one.

Also, we could add some helper functions for Run(), such as:

• printing output data as hex
• RpcFunctions.ps1 - function New-RpcContextHandle

In the autogenerated client code, parameters use fully-qualified type names that could be simplified. Example: NtApiDotNet.Ndr.Marshal.NdrContextHandle p could simply be NdrContextHandle p

Consider converting from Windows Forms to WPF

Let's modernize.