Trying to build a C# grammar for bison/wisent

Question

I've never done Bison or Wisent before.
how can I get started?

My real goal is to produce a working Wisent/Semantic grammar for C#, to allow C# to be edited in emacs with code-completion, and all the other CEDET goodies. (For those who don't know, Wisent is a emacs-lisp port of GNU Bison, which is included into CEDET. The Wisent apparently is a European Bison. And Bison, I take it, is a play-on-words deriving from YACC. And CEDET is a Collection of Emacs Development Tools. All caught up? I'm not going to try to define emacs. )

Microsoft provides the BNF grammar for C#, including all the LINQ extensions, in the language reference document. I was able to translate that into a .wy file that compiles successfully with semantic-grammar-create-package.

But the compiled grammar doesn't "work". In some cases the grammar "finds" enum declarations, but not class declarations. Why? I don't know. I haven't been able to get it to recognize attributes. I'm not finding the "debugging" of the grammar to be very easy.

I thought I'd take a step back and try to produce a wisent grammar for a vastly simpler language, a toy language with only a few keywords. Just to sort of gain some experience. Even that is proving a challenge.

I've seen the .info documents on the grammar fw, and wisent, but... still those things are not really clarifying for me, how the stuff really works.

So

Q1: any tips on debugging a wisent grammar in emacs? Is there a way to run a "lint-like" thing on the grammar to find out if there are unused rules, dead-ends stuff like that? What about being able to watch the parser in action? Anything like that?

Q2: Any tips on coming up to speed on bison/wisent in general? What I'm thinking is a tool that will allow me to gain some insight into how the rules work. Something that provides some transparency, instead of the "it didn't work" experience i'm getting now with Wisent.

Q3: Rather than continue to fight this, should I give up and become an organic farmer?

---

ps: I know about the existing C# grammar in the contrib directory of CEDET/semantic. That thing works, but ... It doesn't support the latest C# spec, including LINQ, partial classes and methods, yield, anonymous methods, object initializers, and so on. Also it mostly punts on parsing a bunch of the C# code. It sniffs out the classes and methods, and then bails out. Even foreach loops aren't done quite right. It's good as far as it goes, but I'd like to see it be better. What I'm trying to do is make it current, and also extend it to parse more of the C# code.

Answer 1

You may want to look at the calc example in the semantic/wisent directory. It is quite simple, and also shows how to use the %left and %right features. It will "execute" the code instead of convert it into tags. Some other simple grammars include the 'dot' parser in cogre, and the srecode parser in srecode.

For wisent debugging, there is a verbosity flag in the menu, though to be honest I hadn't tried it. There is also wisent-debug-on-entry which lets you select a action that will cause the Emacs debugger to stop in that action so you can see what the values are.

The older "bovine" parser has a debug mode that allows you to step through the rules, but it was never ported to wisent. That is a feature I have sorely missed as I write wisent parsers.

Answer 2

Regarding Q1: 1st make sure that the wisent parser is actually used:

(fetch-overload 'semantic-parse-stream)

should return wisent-parse-stream.

Run the following elisp-snippet:

(easy-menu-add-item semantic-mode-map '(menu-bar cedet-menu) ["Wisent-Debug" wisent-debug-toggle :style toggle :selected (wisent-debug-active)])
(defun wisent-debug-active ()
  "Return non-nil if wisent debugging is active."
  (assoc 'wisent-parse-action-debug (ad-get-advice-info-field 'wisent-parse-action 'after)))
(defun wisent-debug-toggle ()
  "Install debugging of wisent-parser"
  (interactive)
  (if (wisent-debug-active)
      (ad-unadvise 'wisent-parse-action)
    (defadvice wisent-parse-action (after wisent-parse-action-debug activate)
      (princ (format "\ntoken:%S;\nactionList:%S;\nreturn:%S\n"
             (eval i)
             (eval al)
             (eval ad-return-value)) (get-buffer-create "*wisent-debug*"))))
  (let ((fileName (locate-file "semantic/wisent/wisent" load-path '(".el" ".el.gz")))
    fct found)
    (if fileName
    (with-current-buffer (find-file-noselect fileName)
      (goto-char (point-max))
      (while (progn
           (backward-list)
           (setq fct (sexp-at-point))
           (null
            (or
             (bobp)
             (and
              (listp fct)
              (eq 'defun (car fct))
              (setq found (eq 'wisent-parse (cadr fct))))))))
      (if found
          (eval fct)
        (error "Did not find wisent-parse.")))
      (error "Source file for semantic/wisent/wisent not found.")
      )))

It creates a new entry Wisent-Debug in the Development-menu. Clicking this entry toggles debugging of the wisent parser. Next time you reparse a buffer with the wisent-parser it outputs debug information to the buffer *wisent debug*. The buffer *wisent debug* is not shown automatically but you find it via the buffer menu. To avoid a flooding of *wisent debug* you should disable "Reparse when idle". From time to time you shold clear the buffer *wisent debug* with erase-buffer.