.NET and Memory

We recently released a maintenance release for .NET Memory Profiler 3.0 - v3.0.108.

It can be downloaded from http://memprofiler.com/download.aspx.

This release contains a few minor fixes, for example:

When unloading an unsaved project from the projects explorer the profiler did not prompt you to save the changes. 

There was a timing issue in the profiler that could cause the profiler and the profiled process to get stuck. This happened when identifying root references while collecting a heap snapshot under .NET 1.1. It is normally very unlikely that this should occur; we have only had a single report about this, and yet the problem has probably existed since the first release.

The code that handles side-by-side DLLs in the unmanaged resources tracker contained an error. Fixing this error requires some time, so we have only included a temporary solution in this release. The resource  tracker will only track instances that are created by the firstly loaded DLL. This is probably not a big issue, most resource instances will still be tracked correctly, but it might occur in the GdiPlus DLL when working against Microsoft Office components. We are currently working on a better solution.

For more information see the release notes at http://memprofiler.com/releasenotes.htm.

A common question we get about the profiler is how the number presented under the “Other data-><Other>” node in the Native memory tree should be interpreted. This number can often be significant compared to the total memory used by the application.

The “Other data -> <Other>” node represents native memory that the profiler has failed to identify (as mentioned below, data collected by the unmanaged resource tracker is currently not used). This can be memory used by:

• Unmanaged resources
  Unmanaged windows resources such as HBITMAPs, HWNDs and HICONs etc. will consume memory, but the profiler will not be able to identify it.
• Internal memory used by the runtime
  The runtime needs to allocate memory for internal structures. These structures can contain information about loaded modules,  classes and other data needed by the runtime. When attaching, the profiler identifies some of this memory but when doing normal profiling this memory will be unidentified. Additionally, the amount of unidentified native memory is greater when running under the profiler than it is when running the application the normal way. The runtime itself probably consumes more memory when profiling since it cannot perform certain optimizations and it might keep information available for the profiler that could otherwise be discarded. 
• Memory allocated by VirtualAlloc
  If the memory is not sub-allocated by other functions, like HeapAlloc, then it will be presented as <Other>.
• Memory mapped files
  If the file mapping is created without a specified file, then the memory used will be unidentified.
• Committed stack memory
  This memory is allocated using VirtualAlloc and can consume quite a lot of memory if you have many threads in the application. I will discuss the stack memory usage in a future post.

What do I do if a have a high memory usage that has not been identified by the profiler?

First of all you should investigate whether the high memory usage is really a problem. As mentioned above, the amount of unidentified native memory is higher when running under the profiler. This increase can be mitigated by using the “Attach to” command in the profiler. When attaching to a process the profiler will not affect the memory usage of the profiled process at all (however, since it reads a lot of memory pages, it will affect the working set of the process when a snapshot is collected).

If you still see a lot of unidentified memory, and especially if the size of this memory increases, you can continue the investigation by using the unmanaged resources tracker. It can be used to collect more detailed information about the unmanaged memory usage of the application. After you have collected a snapshot, you will be able to see unmanaged resources in the Types/Resources view. Unfortunately, the information collected by the resource tracker is currently only presented in the normal snapshot views, the native memory tree view does not use the information collected by the resource tracker. This will be implemented in a future version.

Unmanaged resources are identified by the  icon in the Types/Resources view  and can be further investigated by double-clicking. Each created resource instance is associated with a creation context. When looking for a leak related to unmanaged resources, the instances associated with the unmanaged interop context () are usually of most interest. The reason for this is that the creation of these instances is normally performed by managed code (using P/Invoke calls) and it it easier to track why they’re created.  Resource instances associated with the “Managed runtime” and the “Other unmanaged” are usually created by the runtime or other unmanaged code, so it might not be possibly to identify why they’re created.

I hope this post has given you some useful information about the native memory view, and some directions on what to do if you have an application with a lot of unidentified native memory. We will continue to work on identifying native memory, and future versions of the profiler will include more detailed information.

Welcome to my blog.  

My name is Andreas Suurkuusk and I am one of the founders of SciTech Software AB. I’m the primary developer of our .NET Memory Profiler, a program that started as a side-project but has now become our main product. In this blog I intend to focus on information about .NET Memory Profiler, comments about the .NET Framework and the runtime, and possibly other development related topics.

I have currently planned for the following categories related to .NET Memory Profiler:

• .NET Memory Profiler Releases
  This category will be used to publish information about all releases of .NET Memory Profiler, including minor bug fix releases. On the download page you are currently able to subscribe to update notifications, and indicate whether you want information about all releases or only major updates. We will remove the option to get notifications about minor updates. Minor updates will only be published in this blog (you can use RSS to get notifications about these).
• .NET Memory Profiler FAQ
  Here I will publish information about frequently asked questions about .NET Memory Profiler.
• .NET Memory Profiler Features
  In this category I will present information about current and upcoming features of .NET Memory Profiler.

Then there will of course be other categories for other topics, but I don’t yet know what I’ll end up writing about, so time will tell what they’ll be.