Parsing CSV input with a RegEx in java

Question

I know, now I have two problems. But I'm having fun!

I started with this advice not to try and split, but instead to match on what is an acceptable field, and expanded from there to this expression.

final Pattern pattern = Pattern.compile("\"([^\"]*)\"|(?<=,|^)([^,]*)(?=,|$)");

The expression looks like this without the annoying escaped quotes:

"([^"]*)"|(?<=,|^)([^,]*)(?=,|$)

This is working well for me - either it matches on "two quotes and whatever is between them", or "something between the start of the line or a comma and the end of the line or a comma". Iterating through the matches gets me all the fields, even if they are empty. For instance,

the quick, "brown, fox jumps", over, "the",,"lazy dog"

breaks down into

the quick
"brown, fox jumps"
over
"the"

"lazy dog"

Great! Now I want to drop the quotes, so I added the lookahead and lookbehind non-capturing groups like I was doing for the commas.

final Pattern pattern = Pattern.compile("(?<=\")([^\"]*)(?=\")|(?<=,|^)([^,]*)(?=,|$)");

again the expression is:

(?<=")([^"]*)(?=")|(?<=,|^)([^,]*)(?=,|$)

Instead of the desired result

the quick
brown, fox jumps
over
the

lazy dog

now I get this breakdown:

the quick
"brown
 fox jumps"
,over,
"the"
,,
"lazy dog"

What am I missing?

Answer 1

Operator precedence. Basically there is none. It's all left to right. So the or (|) is applying to the closing quote lookahead and the comma lookahead

Try:

(?:(?<=")([^"]*)(?="))|(?<=,|^)([^,]*)(?=,|$)

Answer 2

(?:^|,)\s*(?:(?:(?=")"([^"].*?)")|(?:(?!")(.*?)))(?=,|$)

This should do what you want.

Explanation:

(?:^|,)\s*

The pattern should start with a , or beginning of string. Also, ignore all whitespace at the beginning.

Lookahead and see if the rest starts with a quote

(?:(?=")"([^"].*?)")

If it does, then match non-greedily till next quote.

(?:(?!")(.*?))

If it does not begin with a quote, then match non-greedily till next comma or end of string.

(?=,|$)

The pattern should end with a comma or end of string.