Searched refs:TokenizeSubstring (Results 1 – 2 of 2) sorted by relevance
177 void Tokenizer::TokenizeSubstring(const UnicodeText& unicode_text, in TokenizeSubstring() function in libtextclassifier3::Tokenizer224 TokenizeSubstring(unicode_text, span, &result); in InternalRetokenize()229 TokenizeSubstring(unicode_text, span, &result); in InternalRetokenize()
112 void TokenizeSubstring(const UnicodeText& unicode_text, CodepointSpan span,