Strtok in C can be likened to this specific function. It generates tokens from the input sequence, which are divided by designated separators. The separators are determined by a defined predicate.
Syntax:
Template:
split(Result, Input, Predicate Pred);
Parameters:
Input: A container which will be searched.
Pred: A predicate to identify separators.
This predicate is supposed to return true
if a given element is a separator.
Result: A container that can hold copies of
references to the substrings.
Returns: A reference the result
Time Compelxity: O(n)
Auxilary Space: O(1)
In software development, this method involves dividing a string into substrings and delineating them with separators.
Example:
Input : boost::split(result, input, boost::is_any_of("\t"))
input = "geeks\tfor\tgeeks"
Output : geeks
for
geeks
Explanation: Here in input string we have "geeks\tfor\tgeeks"
and result is a container in which we want to store our result
here separator is "\t".
Tokenize the Provided String Using the boost::split Function
Boost provides robust utilities for integrating established, thoroughly vetted libraries into the C++ standard library. This article focuses on the boost::split function, a feature of the Boost string algorithm library. This library offers various methods for string manipulation such as truncating and substituting. The boost::split function segments a string sequence into tokens and distinguishes them using a specified delimiter. Users need to define the delimiter as the third parameter, typically through a predicate function that returns true if the element is a delimiter.
To detect whitespace in the given text and segment it into tokens, an isspace function object is passed in the subsequent illustration. In order to store the tokenized substrings, a destination sequence container is mandatory for boost::split. It is important to highlight that the function invocation replaces any existing data in the destination container beyond the initial parameter, which is the container itself.
#include <iostream>
#include <string>
#include <sstream>
#include <vector>
#include <boost/algorithm/string/split.hpp>
using std::cout; using std::cin;
using std::endl; using std::string;
using std::vector;
int main() {
string text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit.";
vector<string> words;
boost::split(words, text, isspace);
for (const auto &item : words) {
cout << item << "; ";
}
cout << endl;
return EXIT_SUCCESS;
}
Output:
Lorem; ipsum; ; dolor; sit; amet,; consectetur; adipiscing; elit.;
In the earlier code snippet, the boost::split function saves empty strings when multiple delimiters are nearby. However, if we set the fourth optional argument boost::token compress to true, adjacent delimiters will be merged together. Remember that both these code examples require the presence of Boost libraries on the system to function correctly.
#include <iostream>
#include <string>
#include <sstream>
#include <vector>
#include <boost/algorithm/string/split.hpp>
using std::cout; using std::cin;
using std::endl; using std::string;
using std::vector;
using std::stringstream;
int main() {
string text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit.";
vector<string> words;
boost::split(words, text, isspace, boost::token_compress_on);
for (const auto &item : words) {
cout << item << "; ";
}
cout << endl;
return EXIT_SUCCESS;
}
Output:
Lorem; ipsum; dolor; sit; amet,; consectetur; adipiscing; elit.;
Use stringstream With getline Function to Split the String With Delimiters
To separate text using a particular character as a delimiter, you can employ the getline function along with the stringstream class instead. In this case, there is no need for Boost headers as we are solely relying on STL utilities. Note that this iteration of the code is more extended and entails extra procedures to merge adjacent delimiter characters.
#include <iostream>
#include <string>
#include <sstream>
#include <vector>
using std::cout; using std::cin;
using std::endl; using std::string;
using std::vector;
using std::stringstream;
int main() {
string text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit.";
vector<string> words;
char space_char = ' ';
stringstream sstream(text);
string word;
while (std::getline(sstream, word, space_char)) {
words.push_back(word);
}
for (const auto &item : words) {
cout << item << "; ";
}
cout << endl;
return EXIT_SUCCESS;
}
Output:
Lorem; ipsum; ; dolor; sit; amet,; consectetur; adipiscing; elit.;
Split
For a common use scenario, split algorithms serve as an expansion of the find iterator functionality. These algorithms utilize a find iterator to store all occurrences in the specified container. The segmented substrings should be capable of being saved in the container either as references (like iterator range) or duplicates (like std::string).
There are two algorithms available. The all function in the input identifies all occurrences of a particular string, while split method segments the input into distinct sections.
