Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

multithreading and strtok

I will be continually splitting strings in a multithreaded application, I've read that strtok is not suitable for this, but why?

Should I consider using a semaphore around the portion of my code that calls strtok?

like image 239
andandandand Avatar asked Nov 30 '10 05:11

andandandand


People also ask

Can strtok be used with threads?

Every call to the strtok() function, returns a refrence to a NULL terminated string and it uses a static buffer during parsing. Any subsequent call to the function will refer to that buffer only, and it gets altered.! It is independent of who called it, and thats is the reason for it is not thread safe.

Why is strtok thread unsafe?

The identity of the delimiting character is lost. The strtok() function uses a static buffer while parsing, so it's not thread safe.

What does strtok () do in C?

The C function strtok() is a string tokenization function that takes two arguments: an initial string to be parsed and a const -qualified character delimiter. It returns a pointer to the first character of a token or to a null pointer if there is no token.

Is code thread safe?

A [portion of code] is thread-safe if it behaves correctly when accessed from multiple threads, regardless of the scheduling or interleaving of the execution of those threads by the runtime environment, and with no additional synchronization or other coordination on the part of the calling code.


2 Answers

You should consider not using strtok or strtok_r at all. It's trivial to write your own functions similar to these but better-tailored to the exact way you want to use them, and of course have the caller store all the state and pass a pointer to the state for thread-safety/reentrancy.

As for your question about using a semaphore (or other locking primitive) around calls to strtok, that will not help if you just put it around the actual call. You'd have to hold the lock during the whole process of parsing the string to protect the internal state of strtok. I believe this is what many people refer to as locking code in place of data and it's generally considered A Bad Thing.

like image 176
R.. GitHub STOP HELPING ICE Avatar answered Sep 30 '22 19:09

R.. GitHub STOP HELPING ICE


Use strtok_r() for thread safety.

like image 33
fastcodejava Avatar answered Sep 30 '22 18:09

fastcodejava