maoni0 / mem-doc

This is a document to help with .NET memory analysis and diagnostics.

English | 简体中文

mem-doc

Over the years I've worked with a lot of customers to help them with analyzing GC heap related performance issues. Even though perf issues can and often vary a lot from one to the next, there's enough similarities that I thought it’s worthwhile to write a guideline document to either get folks started on the right path to be efficient at doing .NET memory performance analysis, or give those who are already quite experienced in perf more knowledge and tools to make their jobs easier.

The .NET Memory Analysis document in this repo includes the following –

• How to think about performance work and how to pick the right approaches for memory analysis;
• Enough memory fundamentals to help you assess the memory issues;
• Know when to worry about memory issues and how to analyze them with the right tools;

It's a long document so there’s a specific “how to read this document” section to help folks avoid sections that they might already be familiar with. One of my goals is to include material that’s not easy to find elsewhere so I didn’t spend a lot of content on things that you can find quite easily on your own (eg, if you don’t know what ETW is there’s MSDN documentation and plenty of blogs that talk about it).

As we have more and better tooling I will continue updating this document. But another goal of sharing this document is to have contributors who would like to share their tips and tricks for memory analysis that don’t already exist in this doc. Your contributions are greatly appreciated.

I also included slide decks and video recording links of talks I've done in this repo. In general when I do talks I tend to also talk about things that aren't easily found elsewhere. Currently the following talks are included -

2022 MPLR'22 keynote - "Optimizations in the .NET GC" - slides (pdf)

2022 Internal talk - "Diagnosing Memory Leaks" slides (pdf), video

2021 Internal talk - "A .NET Object - from allocation to collection" (talks about how memory for a .NET object is acquired all the way down to the hardware layer. Since I'm touching many memory related things - the hardware, the OS and the GC - I had to stay very high level): slides(pdf), video

2020 dotnext talk - "What's new in the .NET 5.0 GC" (short high level talk on new features in 5.0): slides (pdf)

2020 dotnetos talk - "What's so hard about pinning" (in-depth talk on how pinning is handled): slides(pdf), video

2019 Prague meetup talk - "What's new in the .NET Core 3.0 GC" (fairly detailed talk on new features in 3.0): slides(pdf), video

Thank you and I hope you find this material helpful.

mem-doc's Issues

Calculating allocations from GCHeapStats events

Is it possible to figure out total allocations purely from looking at GCHeapStats? I would like to be able to use GCHeapStats to know how much memory was allocated during a certain time period.

I am using TraceEventSession to get the data. In particular, I was not expecting to see multiple records with Gen0 unchanged. I was under the impression GCHeapStats happened after each collection therefore I expected Gen0 to always be collected and change.

FinalizationPromotedBytes	FinalizationPromotedCount	Generation	Generation0Bytes	Generation1Bytes	Generation2Bytes	Generation3Bytes	Generation4Bytes	PinnedObjectCount	SinkBlockCount	TotalHeapSizeBytes	TotalPromoted0Bytes	TotalPromoted1Bytes	TotalPromoted2Bytes	TotalPromoted3Bytes	TotalPromoted4Bytes	TotalPromotedBytes
3360	21	2	78729160	3482248	21868696	15269320	0	0	60	119349424	3359000	359864	18712624	4817288	0	27248776
3360	35	2	78729160	3482248	21868696	15038408	0	0	60	119118512	3359000	359864	18712624	5667896	0	28099384
36636	45	2	47662456	4855672	24150720	13164024	0	0	57	89832872	4837904	944528	22425736	5954736	0	34162904

One line not show after markdown be rendered

I found this line not show when the markdown text rendered. Maybe by, I guess the Xml Tag not closed.

mem-doc/doc/.NETMemoryPerformanceAnalysis.md

Line 944 in f2dee5e

<GCEvent GCNumber="9" GCGeneration="0" Reason="AllocSmall">

Should we mark these lines as code block like:

<GCEvent GCNumber="9" GCGeneration="0" Reason="AllocSmall">
<GlobalHeapHistory **FinalYoungestDesired="9,830,400"** NumHeaps="12"/> 

But FinalYoungestDesired="9,830,400" in the code block not bold.

[Question] Server vs Workstation GarbageCollection Benchmark

Hi is there a template to follow to benchmark the performance difference between Server vs Workstation GarbageCollection for aspnet core applications ?
we experienced out-of-memory events with Server GarbageCollection, and using Workstation GarbageCollection "should" reduce memory usage, but need to collect the metrics as some of us insists NOT to use workstation GC
(for context we have 30 aspnet core app in Kubernetes with total of about 8 vCPU)

Is there any plan to improve the translation quality of the Chinese version?

I noticed that some translations were not very accurate or idiomatic. Is there any plan to improve the translation quality of the Chinese version? Or would you mind if someone else submits PRs to help?

GC heap size vs committed

In the last sentence of the following paragraph, I get into some confusion:
https://github.com/Maoni0/mem-doc/blob/master/doc/.NETMemoryPerformanceAnalysis.md#special-handling-of-the-ephemeral-segment

it means this difference can be large, e.g., if there are 32 heaps and each heap has a 50 MB gen0 budget and you look at the heap size right after a GC, the size you see is going to be (32 * 50 = 1.6 GB) less than the committed bytes.

a few lines above it says the gen0 budget will be committed, but my understanding is that the "heap size" term would include "free objects" or in other words all committed memory. So shouldn't the heap size always refer to more than the committed?