mlpack 3.4.2
char_extract.hpp
Go to the documentation of this file.
1
14#ifndef MLPACK_CORE_DATA_TOKENIZERS_CHAR_EXTRACT_HPP
15#define MLPACK_CORE_DATA_TOKENIZERS_CHAR_EXTRACT_HPP
16
17#include <mlpack/prereqs.hpp>
18
19namespace mlpack {
20namespace data {
21
26{
27 public:
29 using TokenType = int;
30
39 int operator()(boost::string_view& str) const
40 {
41 if (str.empty())
42 return EOF;
43
44 const int retval = static_cast<unsigned char>(str[0]);
45
46 str.remove_prefix(1);
47
48 return retval;
49 }
50
56 static bool IsTokenEmpty(const int token)
57 {
58 return token == EOF;
59 }
60};
61
62} // namespace data
63} // namespace mlpack
64
65#endif
The class is used to split a string into characters.
static bool IsTokenEmpty(const int token)
The function returns true if the given token is equal to EOF.
int TokenType
The type of the token which the tokenizer extracts.
int operator()(boost::string_view &str) const
The function extracts the first character from the given string view and removes it from the view.
Linear algebra utility functions, generally performed on matrices or vectors.
Definition: cv.hpp:1
The core includes that mlpack expects; standard C++ includes and Armadillo.