strtok_s behaviour with consecutive delimiters

Question

I'm parsing 3 values in parallel which are separated with a specific separator.

token1 = strtok_s(str1, separator, &nextToken1);
token2 = strtok_s(str2, separator, &nextToken2);
token3 = strtok_s(str3, separator, &nextToken3);

while ((token1 != NULL) && (token2 != NULL) && (token3 != NULL))
{
    //...
    token1 = strtok_s(NULL, separator, &nextToken1);
    token2 = strtok_s(NULL, separator, &nextToken2);
    token3 = strtok_s(NULL, separator, &nextToken3);
}

Suppose '-' is my separator. The behaviour is that a string with no consecutive separators:

1-2-3-45

would effectively result in each of these parts:

1
2
3
45

However, a string with two consecutive separators:

1-2--3-45

will not yield a 0 length string, that one is skipped so that the result is:

1
2
3
45

and not

1
2

3
45

What workaround or strategy would be better suited to obtain all the actual parts, including the 0-length ones? I'd like to avoid re-implementing strtok_s, if possible.

Answer 1

Unfortunately, strtok() ignores empty tokens. Even though you said you wish to avoid doing that, there is no other way but to parse it yourself, using for example strchr() to find the next delimiter and then copying the token to a temporary variable for processing. This way you can handle empty tokens whichever way you please.