string tokenizer in Java

Question

I have a text file which contains data seperated by '|'. I need to get each field(seperated by '|') and process it. The text file can be shown as below :

ABC|DEF||FGHT

I am using string tokenizer(JDK 1.4) for getting each field value. Now the problem is, I should get an empty string after DEF.However, I am not getting the empty space between DEF & FGHT.

My result should be - ABC,DEF,"",FGHT but I am getting ABC,DEF,FGHT

Answer 1

From StringTokenizer documentation :

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

The following code should work :

String s = "ABC|DEF||FGHT"; String[] r = s.split("\\|"); 

Answer 2

Use the returnDelims flag and check two subsequent occurrences of the delimiter:

String str = "ABC|DEF||FGHT"; String delim = "|"; StringTokenizer tok = new StringTokenizer(str, delim, true);  boolean expectDelim = false; while (tok.hasMoreTokens()) {     String token = tok.nextToken();     if (delim.equals(token)) {         if (expectDelim) {             expectDelim = false;             continue;         } else {             // unexpected delim means empty token             token = null;         }     }      System.out.println(token);     expectDelim = true; } 

this prints

ABC DEF null FGHT 

The API isn't pretty and therefore considered legacy (i.e. "almost obsolete"). Use it only with where pattern matching is too expensive (which should only be the case for extremely long strings) or where an API expects an Enumeration.

In case you switch to String.split(String), make sure to quote the delimiter. Either manually ("\\|") or automatically using string.split(Pattern.quote(delim));