PerformanceCounterCategory.GetCategories is inconsistent with Perfmon

Question

Okay, So I'm basically trying to create a list of installed Performance Counter Categories, like the one you get in PerfMon. For this I'm using

System.Diagnostics.PerformanceCounterCategory.GetCategories()

which seems like it works, until you inspect the list, and find out that some are missing. The first one I spotted missing was the ReadyBoost Cache. This was because the project was set to compile on "x86". Changing this to "Any CPU" fixed that issue.

However there are still some that are missing, for instance, one of the test machines has a "Authorization Manager Applications" Category (mine doesn't, and nobody seems to know why, or where it comes from) However, on that machine, that Performance Counter Category shows up in PerfMon, but not when invoking the GetCategories() method from C#.

Does anyone know why? Is there a more reliable way to get PerformanceCounterCategories? Is this because I'm using .Net? Is there some native API I can use instead?

EDIT

I'm sorry, I still don't get it. I've written this code to perhaps better illustrate it:

using System;
using System.Diagnostics;
using System.Linq;
using System.Text.RegularExpressions;
using Microsoft.Win32;

namespace PccHack
{
    class Program
    {
        private static readonly Regex Numeric = new Regex(@"^\d+$");
        static void Main(string[] args)
        {
            var pcc1 = PerformanceCounterCategory.GetCategories();
            Console.Out.WriteLine("Getting automatically from the microsoft framework gave {0} results.", pcc1.Count());
            string[] counters;
            using (var regKey = Registry.LocalMachine.OpenSubKey(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Perflib\009"))
            {
                counters = regKey.GetValue("Counter") as string[];
            }
            var pcc2 = counters.Where(counter => !Numeric.IsMatch(counter)).ToList();
            pcc2.Sort();
            Console.Out.WriteLine("Getting manually from the registry gave {0} results.", pcc2.Count());
            Console.In.ReadLine();
        }
    }
}

This now gives me 3236 results. Because it gets all the performance counters in the system. So I figure all I need to do is filter out those that are actually performance counters, leaving me with just categories. However there does not seem to be a constructor for the PerformanceCounter which takes just the name(because this is not unique), nor does there seem to be one which takes the index value. I've discovered a Win32 API named Performance Data Helper, but this doesn't seem to have the functionality I want either. So. If I have a Performance Counter Index, how do I, in C# get the PerformanceCounterCategory, for that index? PerfMon does it, so it must be possible. Is there some way to parse the Index "Magic Number" to figure out which is which?

EDIT 2

Okay. So this is doing my head in. The latest version of the code using the three different approaches suggested (.Net / Registry / PowerShell):

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Reflection;
using Microsoft.Win32;
using System.Management.Automation;


namespace PccHack
{
    internal class Program
    {
        private static void Main()
        {
            var counterMap = new Dictionary<string, string>();
            using (var regKey = Registry.LocalMachine.OpenSubKey(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Perflib\009"))
            {
                var counter = regKey.GetValue("Counter") as string[];
                for (var i = 0; i < counter.Count() - 1; i += 2)
                {
                    counterMap.Add(counter[i], counter[i + 1]);
                }
            }

            var pcc1 = PerformanceCounterCategory.GetCategories().Select(o => o.CategoryName).ToList();
            var pcc2 = new List<string>();
            // Get v1 providers
            using (var regKey = Registry.LocalMachine.OpenSubKey(@"SYSTEM\CurrentControlSet\services"))
            {
                foreach (var subKeyName in regKey.GetSubKeyNames())
                {
                    using (var subKey = regKey.OpenSubKey(subKeyName))
                    {
                        if (!subKey.GetSubKeyNames().Contains("Performance")) continue;
                        using (var perfKey = subKey.OpenSubKey("Performance"))
                        {
                            var blah = (string) perfKey.GetValue("Object List");
                            if (blah != null)
                            {
                                pcc2.AddRange(blah.Split(' ').Select(b => counterMap[b]));
                            }
                        }
                    }
                }
            }
            // Get v2 providers
            using (var regKey = Registry.LocalMachine.OpenSubKey(@"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Perflib\_V2Providers"))
            {
                foreach (var subKeyName in regKey.GetSubKeyNames())
                {
                    using (var subKey = regKey.OpenSubKey(subKeyName))
                    {
                        foreach (var perfKeyName in subKey.GetSubKeyNames())
                        {
                            using (var perfKey = subKey.OpenSubKey(perfKeyName))
                            {
                                var blah = (string) perfKey.GetValue("NeutralName");
                                if (blah != null)
                                {
                                    pcc2.Add(blah);
                                }
                            }
                        }
                    }
                }
            }
            var ps = PowerShell.Create();

            ps.AddCommand("Get-Counter").AddParameter("listSet", "*");
            var pcc3 = ps.Invoke().Select(result => result.Members["CounterSetName"].Value.ToString()).ToList();

            pcc1.Sort();
            pcc2.Sort();
            pcc3.Sort();
            Console.Out.WriteLine("Getting automatically from the microsoft framework gave {0} results.", pcc1.Count());
            Console.Out.WriteLine("Getting manually from the registry gave {0} results.", pcc2.Count());
            Console.Out.WriteLine("Getting from PowerShell gave {0} results.", pcc3.Count());
            Console.In.ReadLine();
        }
    }
}

On my machine I get 138 using the .Net framework, 117 by parsing the registry, and 157 by using PowerShell (which is the correct answer).

However depending on the user having installed PowerShell/Windows SDK is not really an option.

Anyone have any ideas at all? Are there some top secret version 3 performance counter categories, hidden somewhere else in the registry, that I need to track down? I've not only run out of ideas to try, I've run out of bad ideas to try as well. Are there any secret command line switches I can use on perfmon, to get it to list all the Categories?

Answer 1

I think you're running into what I would qualify as a .NET Framework bug induced by Perflib v2 counters.

Behind the scenes, PerformanceCounterCategory uses the Registry Functions to get info about the categories (aka objects), instances and counters currently registered with the Performance subsystem. You can verify this by looking at the code for PerformanceCounterCategory with ILSpy.

Counters can become registered through two types of providers: "core"-providers and "extensibility"-providers. These names were invented by me for lack of a better option.

Core-providers are built deep into Windows and interface with the Performance subsystem intimately to provide counters such as those for Process, System, etc. If you're not able to see this kind of counters through PerformanceCounterCategory, it's very likely you have some deep problem with your Windows installation and at least some of these errors in your Event Logs. I assume this is not your case.

Extensibility-providers interface with the Performance subsystem through the documented Perflib interface to provide all the other counters. It's important to note that counters for some Windows features are registered through extensibility-providers, as are counters for major MS products such as SQL Server, .NET Framework, etc. So it's not that core-providers are for everything made by MS and extensibility-providers for 3rd parties.

If you're not able to see through PerformanceCounterCategory counters registered through Perflib, first it may be that their provider is incorrectly configured in the system or that the configuration has been broken. In this case you should have in your Event Log some of the errors defined in Performance Counter Loading or Performance Library Availability sections from these docs. I assume this is not your case.

The second reason is related to how Perflib providers work behind the scenes. Two major steps are required to have their counters registered. First step is to write the configuration for the providers in the Registry using LodCtr.exe. I assume this has been done automatically for you by the installers of the counters you're interesed in, and that the configuration is correct, especially since if there were a problem with this configuration you would likely have some of the aforementioned errors in the Event Log. Second step is to actually register the Perflib provider with the Performance subsystem.

Now we're getting close to the problem. Registration is done very differently for Perflib v1 and v2 providers. For v1 the code for the providers is written in DLLs referenced from the Registry configuration written at step one and loaded by the system itself. Thus, Perflib v1 provider registration happens automatically as the system reads configuration info from the Registry and loads the DLLs. For Perflib v2 providers things are different. The code for the providers is no longer executed directly by the system but by an application/service associated with the providers. So if you write an app that creates custom providers/counters using Perflib v2 your app would also run the code for gathering data for these providers and it would interface with the Performance subsystem in the documented way. Trouble is, the code doing the registration of Perflib v2 providers with the system now must be triggered by the app hosting the provider code (as opposed to being triggered automatically by the system as for Perflib v1). So, for example, if the app is a Windows service and the service has not started yet, the providers would not be registered with the Performance subsystem and their counters would not be visible (yet) through the Registry Functions / PerformanceCounterCategory.

Here is the relevant part of doc describing this self-registration for Perflib v2 providers:

Your provider must call the CounterInitialize and CounterCleanup functions. The CounterInitialize calls the PerfStartProvider function to register the provider and also calls the PerfSetCounterSetInfo function to initialize the counter set. The CounterCleanup calls the PerfStopProvider function to remove the provider's registration.

In conclusion, there are two different ways of listing categories, instances and counters. One is to query the Registry Functions and list all the items registered at query time. The other is to look at the configuration info written in the Registry describing providers regardless of whether or not they are registered with the Performance subsystem at query time.

In practice you would need to use a combo of the two ways because you can only get instances for categories by querying the Registry Functions and you can only get categories and counters for providers that have not been registered yet by querying the configurations written in the Registry.

Unfortunately, PerformanceCounterCategory only queries the Registry Functions and so is not able to get you info about Perflib v2 providers that have not been registered yet. You can see these providers through other means, for example through Performance Monitor MMC (which behind the scenes uses the PDH API, which is able to show a combo of registered and not-yet-registered counters) or typeperf.exe -qx.

You can test the above applies to you with the BranchCache category. The example below was tested on Win 7.

1. Ensure the Windows service with display name BranchCache is started and then run this C# code:

   Debug.WriteLine((new PerformanceCounterCategory("BranchCache")).ReadCategory().Keys.Count);
   

   You should get no error and 21 written to the debug output.

2. Now stop the BranchCache service and run the C# code again. You'll get an exception as the category is no longer registered with the Performance subsystem and so PerformanceCounterCategory fails to find it.

To ensure what I described applies to the counters you're missing via PerformanceCounterCategory.GetCategories(), check that the missing counters are shown by typeperf -qx on categories with names associated with providers configured in Registry somewhere under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Perflib\_V2Providers.

The solution is to write a C# wrapper for the PDH API and get your info that way. This is non-trivial, especially if you're not used to handling native interactions. WMI also seems to be a valid option (I tried a quick listing of performance objects through PowerShell and it seems counters for all providers are returned) but while you don't need to know how to interface with native code you need to know WMI, which is also non-trivial.

Answer 2

Performance counters (and categories) are registered per locale. That is, you can have different names for them depending on the language.

All available performance categories and their counters are registered in the Windows Registry under HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Perflib. You will find a sub key for each available language (e.g. 009 for English).

The PerformanceCounterCategory.GetCategories() method internally works is to first check the "invariant culture" categories. If it finds any it will return this set. So, if due to some error or vendor's oversight a category is only available with one language, you'll not get it depending on your current language setting (either OS or application or both).

I would first check the content of the HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Perflib\<langcode>\Counter keys and see if maybe the missing category is only in one of them. A related issue might be this (Google search), but I haven't checked further.

Frankly, I don't know of any "better" way to get the list of available counters. If your issue is the one described above (or related), I would rather try to see to get the situation fixed.