Cloning a sublime text 3 highlighting syntax definition

Question

Is there a simple procedure for deriving your own custom sublime text 3 highlighting definition, from an existing one?

I've installed AAAPackageDev as recommended by the orderly and slightly tedious tutorial for creating a new highlighting syntax definition, but I find that cloning one would get me going much faster. In the very least, I'd like to learn how can I browse the default syntax definitions shipping with sublime.

If you really want to be entirely specific, all I wish to accomplish is a primitive highlighting scheme where any string between a given pair of opening and closing tokens, of my own definition, would get colored. And any string between another different pair of tokens of my own definition, would get colored with a second color. A nicity would be to also have the tokens themselves grayed out.

(Ubuntu 14.04)

Thanks!

Answer 1

If you install PackageResourceViewer you can open any of the built in, compressed resources. Do PackageResourceViewer: Open Resource, then select a language that you're comfortable with. Then select the file with the .tmLanguage extension.

Answer 2

UPDATE 2015-05-10: Sublime Text 3 build 3084 introduces a totally new sublime-syntax format for writing syntax definitions. It's much better than the old system, which Sublime inherited from TextMate. The new system should land in the public ST3 beta builds soon. Since ST3 is the recommended version of Sublime, I'd recommend writing any new highlighters using the new system instead of the system described below.

Here's a crash course in Sublime Text syntax highlighting.

Setup

First, as @lthreed pointed out, you can use PackageResourceViewer to look at the default packages that come with Sublime Text. Those .tmLanguage files are all in plist format, which is extremely difficult to read and understand. PackageDev can convert plist files into the far more readable JSON or YAML formats. When you're learning by looking at the default packages, make sure to convert it to YAML first. Be warned, PackageDev may not convert it perfectly. It doesn't matter. You're just using the code as a reference.

plist is the native format that Sublime understands, but that doesn't mean you should write it like that. I strongly recommend writing your highlighter in YAML and converting it to a plist with PackageDev. Don't write it in JSON. JSON does not support raw strings. All the regexes will have to be double-escaped. This is an absolute nightmare. Just use YAML.

You can start a new syntax definition by opening the command palette (cmd+shift+p on a Mac) and selecting PackageDev: New YAML Syntax Definition. When you're ready to test it, open the command palette and select PackageDev: Convert (YAML, JSON, PList) to... and PackageDev will figure out that you have a YAML file and want to convert to plist. The conversion takes your .YAML-tmLanguage files and spits out a .tmLanguage file that Sublime understand. Put that file inside the /Packages/User directory and Sublime will load it and apply it (you may have to restart).

How Sublime syntax highlighting works

The syntax definition you are writing doesn't color the text directly. It applies scope names to the text. Then the people who write themes like Monokai and Solarized come along and make files that associate the scope names to colors. You can make up your own scope names, but you should stick to the official TextMate scope names. These scope names may not make any sense at all for the code you're matching. That's ok. Just do the best you can. If you have to make up a scope name, use the TextMate scope names as a starting point. For example, instead of string.quoted.double.xxx (where xxx is the file extension of the language you're matching), you could make up a scope name called string.quoted.triple.xxx.

Code Sample

Here's a syntax definition for a made up matt language with file extension .matt. It only has two rules: one for matching a pipe-delimited string, and one for matching a string with more complicated delimiters.

# [PackageDev] target_format: plist, ext: tmLanguage
---
name: Mattlang
scopeName: source.matt
fileTypes: [matt]

patterns:
- include: '#pipe-string'
- include: '#complex-string'

# Rules defined in the repository can reference each other. You can include
# one rule inside another.
repository:
  # This is a rule of the begin-end form. The rule matches a string bounded by
  # pipes, such as |hello there|
  pipe-string:
    # The optional 'name' field lets you apply a single scope to everything,
    # including the begin-end pipes. All the scope names must end with .matt
    name: everything.matt
    # We have to escape the pipe character, because it's a special character in
    # the Oniguruma regex syntax (and most other regex engines).
    begin: \|
    # 'beginCaptures' is required if you want the pipes to be colored differently
    beginCaptures:
      # In regex jargon, the begin pipe is 'captured'. Capture group 0 means the
      # entire match, which in this case is just the pipe.
      '0': {name: entire.begin.match.matt}
    # The optional 'contentName' field lets you apply a scope to all the text
    # between (but not including) the begin-end pipes.
    contentName: stuff.between.the.pipes.matt
    patterns:
    # These rules will only be applied to the text *BETWEEN* the pipes. Sublime
    # will go through the rules from top to bottom and try to match the text, so
    # higher rules have a higher "precedence" and will get matched first.
    # Given the text |hello there|, Sublime will see an 'h' character and move
    # through the rules from top to bottom trying to find a rule that starts
    # with 'h'. The #hell rule will match the 'h' and the rest of the
    # characters. The #hell scope name will be applied to the 'hell' text and 
    # Sublime will resume trying to find the next match at the 'o' character.
    # The 'o' character WILL NOT match #hello. You can think of the matched text
    # as being removed from the stream entirely. The point is: order matters.
    - include: '#hell'
    - include: '#hello'
    - end: \|
    endCaptures:
      '0': {name: entire.end.match.matt}

  # This is the other form of rule you can define. It's extremely simple --
  # just a scope name and a regex pattern to match. Note that these rules will
  # only match text on the same line, unlike begin-end rules, which can cover
  # multiple lines.
  hell:
    name: some.other.scope.matt
    match: hell

  hello:
    name: some.scope.matt
    match: hello

  # This rule matches a string that starts with $!! and ends with !!$,
  # e.g. !!$hello there!!$
  complex-string:
    # I've labeled the capture groups.
    #      |---0---|
    #      |--1-||3|
    begin: (!(!))($)
    #        |2|
    beginCaptures:
      '0': {name: full.match.matt}
      '1': {name: both.exclamation.marks.matt}
      '2': {name: second.exclamation.mark.matt}
      '3': {name: dollar.sign.matt}
    # It's ok to leave out the 'patterns' field. Technically, all you really
    # need is a 'begin' field and an 'end' field.
    end: ((!)!)($)
    endCaptures:
      '0': {name: everything.matt}
      '1': {name: both.exclamation.marks.matt}
      '2': {name: first.exclamation.mark.matt}
      '3': {name: dollar.sign.matt}