Home
last modified time | relevance | path

Searched refs:MaybeResetTokenAndAddChar (Results 1 – 1 of 1) sorted by relevance

/external/libtextclassifier/native/utils/
Dtokenizer.cc292 auto MaybeResetTokenAndAddChar = in NumberTokenize() local
321 MaybeResetTokenAndAddChar(codepoint_index, NUMERICAL, it); in NumberTokenize()
323 MaybeResetTokenAndAddChar(codepoint_index, TERM, it); in NumberTokenize()
325 MaybeResetTokenAndAddChar(codepoint_index, WHITESPACE, it, in NumberTokenize()