Scala's compiler lexer

Question

I'm trying to get a list of tokens (I'm most interested in keywords) and their positions for a given scala source file.

I think there is a lexer utility inside scala compiler, but I can't find it. Can you point me into the right direction?

Answer 1

A simple lexer for a Scala-like language is provided in a standard library.

A small utility program which tokenizes Scala source using the same lexer as compiler does lives here

Answer 2

Scalariform has an accurate Scala lexer you can use:

import scalariform.lexer._
val tokens = ScalaLexer.rawTokenise("class A", forgiveErrors = true)
val keywords = tokens.find(_.tokenType.isKeyword)
val comments = tokens.find(_.tokenType.isComment)