Detecting variables through a string

Question

I am creating a simple IDE using JTextPane and detecting keywords and coloring them.

Currently, I am able to detect:

1. Comments
2. String Literals
3. Integers & Floats
4. Keywords

The way I detect these types are through regular expressions.

Now, I am trying to detect variables like [int x = 10;] and coloring them a different color.

Currently, I am able to get all data types like int, float char using the following regex:

Pattern words = Pattern.compile(\\bint\\b|\\bfloat\\b\\bchar\\b);
Matcher matcherWords = words.matcher(code);
while (matcherWords.find()) {
    System.out.print(code.substring(matcherWords.start(), matcherWords.end());
    // How to get next word that is a variable?
}

Below is a sample output of my program:

How am I able to detect variables like a, b, c after I can detect int, float, etc?

Answer 1

Try this one:

(?:(?<=int|float|String|double|char|long)(?:\s+[a-zA-Z_$][\w$]*\s*)|(?<=\G,)(?:\s*[a-zA-Z_$][\w$]*\s*))(?=,|;|=)

which means:

• (?<=int|float|String|double|char|long) - positive lookbehind searching for variable type,
• (?:\s+[a-zA-Z_$][\w$]*\s*) - non capturing group: at least one space, followed by valid characters for Java variables, followed by zero or more spaces
• | - or; alternative between maching name after var. type or after comma,
• (?<=\G,) - positive lookbehind for previous match and comma (because other parts match spaces from both sides)
• (?:\s*[a-zA-Z_$][\w$]*\s*) - non capturing group: at least one space, followed by valid characters for Java variables, followed by zero or more spaces
• (?=,|;|=) - positive lookahead for comma, equal sign or semi-colon

it use a \G boundary matching (The end of the previous match), so the alternative, which search names between other names (words beetween spaces or/and commas exactly), will match only if it is after previous match. So it will not match every word beetween commas in Strings for example. Also I added $ in [a-zA-Z_$][\w$]* as it is allowed in variable names however not recommended.

DEMO

And for Java:

 Pattern pattern = Pattern.compile("(?:(?<=int|float|String|double|char|long)(?:\\s+[a-zA-Z_$][\\w$]*\\s*)|(?<=\\G,)(?:\\s*[a-zA-Z_$][\\w$]*\\s*))(?=,|;|=)");

EDIT

You can use (int |float |...) to match names of variables directly using matcher.start() and matcher.end() without spaces, however I would rather use (?:\s*) in every place where space can ocour and then check for redundant spaces during data process, because you never know how much spaces will user type (of course more than one is redundant, but it is still valid!).

Another approuch would be to match spaces but use groups, like:

(?:(?<=int|float|String|double|char|long)(?:\s+)([a-zA-Z_$][\w$]*)(?:\s*)|(?<=\G,)(?:\s*)([a-zA-Z_$][\w$]*)(?:\s*))(?=,|;|=)

DEMO

names are without spaces, but you need to extract them from groups 1 & 2 by matcher.start(group no) and matcher.end(group no).

EDIT2 answer to question from comment

It depends what you want to achieve. If you just want to get variables as Strings, it is enough to use mathod trim() but if you want to get start and end indices of variables in text, to for example highlight it in different colour, it will be better to use for example matcher.start(1) to extract start index of group 1. Consider this example:

import java.io.IOException; import java.util.regex.Matcher; import java.util.regex.Pattern;

public class Test {
    public static void main(String[] args) throws IOException {
        String      text = "int a = 100;\n" +
                "float b = 100.10;\n" +
                "double c - 12.454545645;\n" +
                "long longest dsfsf = 453543543543;\n" +
                "a = d;\n" +
                "char     b = 'a';\n" +
                "String str = \"dfssffdsdfsd\"\n" +
                "int d,f,g;\n" +
                "int a,f,frhg = 0;\n" +
                "String string = \"a,b,c,d,e,f\"";

        Pattern pattern = Pattern.compile("(?:(?<=int|float|String|double|char|long)(?:\\s+)([a-zA-Z_$][\\w$]*)(?:\\s*)|(?<=\\G,)(?:\\s*)([a-zA-Z_$][\\w$]*)(?:\\s*))(?=,|;|=)");
        Matcher matcher = pattern.matcher(text);
        while(matcher.find()){
            System.out.println("trim(): " + text.substring(matcher.start(),matcher.end()).trim()); // cut off spaces by trim() method;

            int group = (matcher.group(1)==null)? 2 : 1; // check which group captured string;
            System.out.println("group(" + group + "): \n\t"  // to extract string by group capturing;
                    + text.substring(matcher.start(group),matcher.end(group))
                    + ",\n\tsubstring(" + matcher.start(group) + "," + matcher.end(group)+")");

        }
    }
}

the output present two approches.