In C programming language, string tokenization refers to the process of breaking a string into smaller substrings (tokens) based on a specific delimiter. The strtok() function is commonly used for tokenizing strings in C.

Here is an example of how to use strtok() to tokenize a string:

Copy code

#include <string.h> #include <stdio.h> int main() {    char str[] = "Hello, world! How are you?";    char* token;    // Tokenize the string using space as the delimiter    token = strtok(str, " ");    while (token != NULL) {        printf("%s\n", token);        token = strtok(NULL, " ");    }    return 0; }

In this example, the string "Hello, world! How are you?" is tokenized using a space as the delimiter. The function strtok() is called with the first argument as the string to be tokenized and the second argument as the delimiter. The function returns the first token, which is "Hello,". The function is called again with a NULL argument and the delimiter, to get the next token.

It is important to note that strtok() modifies the original string, replacing the delimiter with a null character. Also, strtok() is not thread-safe, so if you need to tokenize strings in a multithreaded environment, you should use a different function such as strtok_r().