How to Split Strings with strtok() in C Programming?

Splitting strings is important for various reasons such as text processing, data analysis or parsing, and the C programming language offers numerous ways to handle this task. However, the most popular C function that allows splitting string into multiple tokens by a specified delimiter is strtok(). Being a library function, it is available in the standard C library that supports string operations.

How Does the strtok() Function Work?

To split a string using strtok(), you first need to understand how it works. The strtok() reads the input string character by character until it comes across a delimiter. Once a delimiter is found, strtok() replaces it with a null character ‘\0’, effectively splitting the original string into two parts. A pointer to the first character of the string is then returned by the function. The strtok() function’s syntax is:

strtok(char* str, const char* delim);

The strtok() function takes two arguments- a pointer to the first token in the set of tokens and a string that holds the delimiter characters. It returns a pointer to the next token, which starts immediately after the delimiter that split the last token. It returns NULL if there are no more tokens.

How to Split Strings with Strtok() in C Programming?

Here is an example of how to use strtok() to split a string:

#include <stdio.h>

#include <string.h>

int main() {

char input[] = "Linux,Hint";

char *token;

token = strtok(input, ",");

while (token != NULL) {

printf("%s\n", token);

token = strtok(NULL, ",");

}

return 0;

}

In the above example, we first declare a pointer to a char. We then use strtok() to split the input string into tokens. The first call of strtok() takes two arguments: the input string and the delimiter string “,”. The first token, “Linux,” is the one that strtok() returns as a pointer to. We then use a while loop to iterate through the input string, using strtok() with a NULL pointer to continue splitting the remaining tokens. The delimiter string is again passed as an argument to strtok() to indicate the delimiter characters.

Output

Limitations of Strtok() Function

1: Modifies the Original String

The strtok() function does not return a copy of the string; instead, it modifies the original string. Therefore, if you need to have an unmodified version of the string after splitting it, you should make a copy of the original string before invoking strtok().

2: Asynchronous

One significant limitation of strtok() is that it can only be used on a single string at a time. In other words, you cannot use it to tokenize multiple strings simultaneously.

3: Limited Capacity

Another potential limitation is that strtok() has a limited capacity to handle repeated delimiters. For example, if your string has consecutive spaces between words, strtok() treats it as a single delimiter. To handle such cases, you’ll have to use a different approach.

Conclusion

The strtok() function is a very useful function for splitting strings in C programming. It takes an input string and a delimiter string and returns a pointer to the first token found in the input string. Strtok() is destructive, thus if you need to preserve the integrity of the original string, it is advised to create a duplicate of it instead. Now that you know how to do it, you can cut any string into smaller pieces for processing.