Searched refs:is_token_in_span (Results 1 – 1 of 1) sorted by relevance
351 bool is_token_in_span; in CodepointSpanToTokenSpan() local353 is_token_in_span = codepoint_start < selectable_tokens[i].end && in CodepointSpanToTokenSpan()356 is_token_in_span = codepoint_start <= selectable_tokens[i].start && in CodepointSpanToTokenSpan()359 if (is_token_in_span && !selectable_tokens[i].is_padding) { in CodepointSpanToTokenSpan()