make VS Code parse and display the structure of a new language to the outline region of VSC

Question

I'm trying to make VSC display the structure of a document containing a DSL (domain specific language) by adding the language definition to VSC. The structure should appear in VSC "outline view" where all document structure is shown for installed languages (like json, markdown, html, etc.)

The DSL is quite simple and just some elements in capital letters should appear in the outline and remain the hierarchy:

WORD xxx
GRAMMAR xxx
STRUCTURE xxx xxx
     xxx xxx xxx xxx
MEANING xxx xxx xxx 
    SUB_MEANING xxx xxx xxx xxx
        SUB_SUB_MEANING xxx xxx xxx

I followed all hints on stackoverflow that all lead to official documentations of VSC and/or the language server protocol (LSP). However, none helped, not at all -.- Yes, I could use the CodeMap extension, but I don't want to be dependent on that since VSC actually is able to understand new languages. For well known languages, there is no need for creating a dedicated tree view element or something elses, so there MUST be a way to make VSC parse the language structure.

"outline view" in VSC remains empty. I figured out that the installed language support for (e.g.) markdown or json does also not produce any content to "Outline" if one deletes the folder "xxx-language-features" (xxx stands for the language) within the extension folder of VSC. So it seems that I need a language-feature-extension, too.

I went through https://code.visualstudio.com/api/language-extensions/language-configuration-guide and https://microsoft.github.io/language-server-protocol/ and many other stuff including the LSP-example from the Github-Repo of VSC but there is NOTHING that helps on that. I also tried to create a new language by aid of "yo code". Nothing. The LSP-example provided by Microsoft is for plaintext files...how useful is it to create a language server for plaintext?! I would like to have an example on a language. Looking at the compiled files inside extensions doesn't help since they're minified.

There is no complete "how to" on that issue - So, any help is appreciated! How can I tell VSC to parse the document structure into the "outline view"?

Answer 1

You have several ways to solve this, you could:

1. create an IDE-exclusive extension that does all the work (parsing the file, understand the meaning of all tokens and provide the list of symbols, etc.)
2. create a Language Server client extension by using a Language Server that implements the language server protocol as specified by Microsoft. In this case, your VSCode extension will essentially be a "shell" extension launching the language server and communicating with it. What to do with the information returned by the server is already implemented out of the box by the vscode-languageclient package's LanguageClient class.

Option 2. has a huge advantage: it fixes the N*M problem: all editors that support the Language Server Protocol can communicate with your language server so it makes easy to provide language support to several editors (VSCode, Visual Studio, Atom, Vim, emacs, etc., IntelliJ in a limited way using a 3rd party plugin) with minimal effort.

In your question, it is not clear to me if you plan to write an extension that does all the work or if you plan to develop a language server and a language-client extension to go with it.

The answer really depends on your language's (or DSL) complexity. If it has a somewhat complex grammar and you already have a compiler for it, then making a language server makes sense since most of the parsing and logic is already implemented in the compiler. If your language is really as simple as the example you gave and doesn't have a grammar, then it might be overkill.

If you go "the language server" route, then there are libraries in different languages that ease the implementation of the server, they take care of the JSON-RPC part of the LSP and you just have to override the methods defined by the Language Server Protocol (e.g. LSP4J for any JVM language, or vscode-languageserver for NodeJS (despite its name, the package is not specific to VSCode)). In this case, having the Outline View of VSCode filled with your symbols' hierarchy only requires you to implement textDocument/documentSymbol request in your Language server and nothing else in the extension, it's taken care of by the vscode-languageclient package's LanguageClient class!

• If your language client supports hierarchical document symbols: in the language server initialize method, you'll receive InitializeParams with textDocument.documentSymbol.hierarchicalDocumentSymbolSupport == true (VSCode does support this) then you return DocumentSymbol[] (which has a children property, thus providing your client a tree of symbols).
• If your language server client does not support hierarchical document symbols (e.g. Visual Studio), then you return SymbolInformation[], which is a flat list of symbols.

One important thing to take care of: VSCode takes the range attribute of the DocumentSymbol into account when displaying the symbols tree (in Outline view). Your children symbols' range MUST BE included in their parent's range or VSCode will not display any symbol in Outline view, even if your symbols tree structure and selectionRanges are correct. The range is the definition range (the whole definition of your symbol, e.g. for a typescript class, it starts at the keyword class and ends with the closing }) and the selectionRange is often only the symbol's token range (but according to the specification, it can also include the doc comment block and the visibility modifiers, it is the implementor's choice). e.g.

class MyClass {
  /**
   * Some method documentation
   */
  private function myFunction(param1) {
    console.log(param1);
  }
}

In this case, the ranges would be

rangeOfClass = {
  "start": { "line": 0, "character": 0 },
  "end": { "line": 7, "character": 1 }
};
rangeOfMethod = {
  "start": { "line": 4, "character": 2 },
  "end": { "line": 6, "character": 3 }
};

and the selectionRange could be only the name of the symbol

selectionRangeOfClass = {
  "start": { "line": 0, "character": 6 },
  "end": { "line": 0, "character": 13 }
};
selectionRangeOfMethod = {
  "start": { "line": 4, "character": 19 },
  "end": { "line": 4, "character": 29 }
};

or it could include from the documentation, keywords and modifiers and upto the end of the symbol

selectionRangeOfClass = {
  "start": { "line": 0, "character": 0 },
  "end": { "line": 0, "character": 13 }
};
selectionRangeOfMethod = {
  "start": { "line": 1, "character": 2 },
  "end": { "line": 4, "character": 29 }
};

And with that, you'll get a nice Outline:

Answer 2

The outline view is populated by the Document Symbols Request. The same is true for the breadcrumbs navigation, and of course the regular document symbols popup (Go → Go to Symbol in File...).

The hierachy is achieved by using DocumentSymbol.children instead of returning a flat list.