Home
last modified time | relevance | path

Searched refs:match_is_unknown_character (Results 1 – 1 of 1) sorted by relevance

/external/libtextclassifier/native/utils/
Dwordpiece_tokenizer.cc52 int* byte_end, bool* found_match, bool* match_is_unknown_character) { in LongestMatchStartingAt() argument
53 *match_is_unknown_character = false; in LongestMatchStartingAt()
83 *match_is_unknown_character = true; in LongestMatchStartingAt()
173 bool match_is_unknown_character; in TokenizeL2RGreedy() local
177 &match_is_unknown_character); in TokenizeL2RGreedy()
180 if (match_is_unknown_character) { in TokenizeL2RGreedy()