Searched defs:Tokenizer (Results 1 – 2 of 2) sorted by relevance
53 class Tokenizer { class56 explicit Tokenizer(const std::string &str) : str_(str) {} in Tokenizer() function in ark::Tokenizer
53 class Tokenizer { class56 explicit Tokenizer(const std::string &str) : str_(str), pos_(0) {} in Tokenizer() function in panda::Tokenizer