Searched refs:TokenizeSubstring (Results 1 – 2 of 2) sorted by relevance
227 void TokenizeSubstring(const UnicodeText& unicode_text, CodepointSpan span,
729 TokenizeSubstring(unicode_text, span, &result); in InternalRetokenize()734 TokenizeSubstring(unicode_text, span, &result); in InternalRetokenize()739 void FeatureProcessor::TokenizeSubstring(const UnicodeText& unicode_text, in TokenizeSubstring() function in libtextclassifier::FeatureProcessor