gawk FS to split record into individual characters

Question

If the field separator is the empty string, each character becomes a separate field

$ echo hello | awk -F '' -v OFS=, '{$1 = NF OFS $1} 1'
5,h,e,l,l,o

However, if FS is a regex that can possibly match zero times, the same behaviour does not occur:

$ echo hello | awk -F ' *' -v OFS=, '{$1 = NF OFS $1} 1'
1,hello

Anyone know why that is? I could not find anything in the gawk manual. Is FS="" just a special case?

I'm most interested in understanding why the 2nd case does not split the record into more fields. It's as if awk is treating FS=" *" like FS=" +"

Answer 1

Interesting question!

I just pulled gnu-awk 4.1.0's codes, I think the answer we could find in the file field.c.

line 371:
 * re_parse_field --- parse fields using a regexp.
 *
 * This is called both from get_field() and from do_split()
 * via (*parse_field)().  This variation is for when FS is a regular
 * expression -- either user-defined or because RS=="" and FS==" "
 */
static long
re_parse_field(lo...

also this line: (line 425):

if (REEND(rp, scan) == RESTART(rp, scan)) {   /* null match */

here is the case of <space>* matching in your question. The implementation didn't increment the nf, that is, it thinks the whole line is one single field. Note this function was used in do_split() function too.

First, if FS is null string, gawk separates each char into its own field. gawk's doc has clearly written this, also in codes, we could see:

line 613:
 * null_parse_field --- each character is a separate field
 *
 * This is called both from get_field() and from do_split()
 * via (*parse_field)().  This variation is for when FS is the null string.
 */
static long
null_parse_field(long up_to,

If the FS has single character, awk won't consider it as regex. This was mentioned in doc too. Also in codes:

#line 667
 * sc_parse_field --- single character field separator
 *
 * This is called both from get_field() and from do_split()
 * via (*parse_field)().  This variation is for when FS is a single character
 * other than space.
 */
static long
sc_parse_field(l

if we read the function, no regex match handling was done there.

In the comments of the function re_parse_field(), and sc_parse_field(), we see do_split invokes them too. It explains why we have 1 in following command instead of 3:

kent$  echo "foo"|awk '{split($0,a,/ */);print length(a)}'
1

Note, to avoid to make the post too long, I didn't paste the complete codes here, we can find the codes here:

http://git.savannah.gnu.org/cgit/gawk.git/