Searched refs:TokenizeSubstring (Results 1 – 2 of 2) sorted by relevance
160 void Tokenizer::TokenizeSubstring(const UnicodeText& unicode_text, in TokenizeSubstring() function in libtextclassifier3::Tokenizer207 TokenizeSubstring(unicode_text, span, &result); in InternalRetokenize()212 TokenizeSubstring(unicode_text, span, &result); in InternalRetokenize()
85 void TokenizeSubstring(const UnicodeText& unicode_text, CodepointSpan span,