How to query the mode of the VB6 IDE from C#

Question

I'm a contributer to the open source project Rubberduck, and looking to add support for the standalone VB6 IDE (it currently supports VBA).

One piece of information we need from the IDE is the mode it's currently in - Design, Break or Run. On the VBA side, this is exposed by the extensibility API, however this is unfortunately not the case for VB6. So I'm left trying to find other ways to query this information.

I had thought it would be possible to 'cheat' by simply examining the caption of the main window - it appends the mode to the project name in the title bar. Unfortunately, this is not practical, as the text gets localised for international versions of the IDE.

I think the solution lies in the EbMode function of vba6.dll (the 'a' here is misleading, and exists for compatibility reasons). Googling 'EbMode VB6' shows numerous examples of this in practice.

Hence I tried the following:

public class Test
{
    private const string DllName = "vba6.dll";  // Not considering VB5 for now
    [DllImport(DllName)]
    private static extern int EbMode();

    public EnvironmentMode Mode => (EnvironmentMode)EbMode();
}

public enum EnvironmentMode
{
    Run = 0,
    Break = 1,
    Design = 2
}

This executes and returns a value, unfortunately it always returns 0 (which maps to 'Run'), even when in design or break mode.

My challenge is that the public code I can find is all written for unmanaged code (mostly VB6, and often including inline assembly). Some of these examples apply some kind of patch before calling the function, and I think this may be what I'm missing. But I'm not sure how to proceed in managed code.

Answer 1

So I figured this out. The key was this post, without which I would never have spotted that the values returned by EbMode do not match the values of the EnvironmentMode enum in the VBA extensibility library.

Return values of EbMode are: 0=Design, 1=Run, 2=Break.