Searched refs:token_end_it (Results 1 – 1 of 1) sorted by relevance
234 auto token_end_it = token_begin_it; in ICUTokenize() local235 std::advance(token_end_it, token_length); in ICUTokenize()239 for (auto char_it = token_begin_it; char_it < token_end_it; ++char_it) { in ICUTokenize()247 context_unicode.UTF8Substring(token_begin_it, token_end_it); in ICUTokenize()255 token_begin_it = token_end_it; in ICUTokenize()