Home
last modified time | relevance | path

Searched defs:token_text (Results 1 – 4 of 4) sorted by relevance

/external/libtextclassifier/native/utils/
Dtokenizer-utils.h79 const std::string token_text = UnicodeText::UTF8Substring(start_it, it); in TokenizeWithFilter() local
96 const std::string token_text = in TokenizeWithFilter() local
/external/libtextclassifier/native/annotator/number/
Dnumber.cc199 bool NumberAnnotator::TryParseNumber(const UnicodeText& token_text, in TryParseNumber()
236 const UnicodeText token_text = in FindAll() local
/external/icing/icing/query/advanced_query_parser/
Dlexer.cc190 std::string token_text(raw_token_text); in ConsumeStringLiteral() local
/external/libtextclassifier/native/annotator/pod_ner/
Dpod-ner-impl.cc359 const std::string token_text = in PrepareText() local