Home
last modified time | relevance | path

Searched refs:CodepointSpanToTokenSpan (Results 1 – 5 of 5) sorted by relevance

/external/libtextclassifier/native/annotator/
Dfeature-processor_test.cc1019 TEST_F(AnnotatorFeatureProcessorTest, CodepointSpanToTokenSpan) { in TEST_F() argument
1025 EXPECT_EQ(TokenSpan(0, 1), CodepointSpanToTokenSpan(tokens, {0, 5})); in TEST_F()
1026 EXPECT_EQ(TokenSpan(1, 2), CodepointSpanToTokenSpan(tokens, {6, 23})); in TEST_F()
1027 EXPECT_EQ(TokenSpan(2, 3), CodepointSpanToTokenSpan(tokens, {24, 29})); in TEST_F()
1028 EXPECT_EQ(TokenSpan(0, 2), CodepointSpanToTokenSpan(tokens, {0, 23})); in TEST_F()
1029 EXPECT_EQ(TokenSpan(1, 3), CodepointSpanToTokenSpan(tokens, {6, 29})); in TEST_F()
1030 EXPECT_EQ(TokenSpan(0, 3), CodepointSpanToTokenSpan(tokens, {0, 29})); in TEST_F()
1033 EXPECT_EQ(TokenSpan(0, 1), CodepointSpanToTokenSpan(tokens, {0, 5}, true)); in TEST_F()
1034 EXPECT_EQ(TokenSpan(1, 2), CodepointSpanToTokenSpan(tokens, {6, 23}, true)); in TEST_F()
1035 EXPECT_EQ(TokenSpan(2, 3), CodepointSpanToTokenSpan(tokens, {24, 29}, true)); in TEST_F()
[all …]
Dfeature-processor.h77 TokenSpan CodepointSpanToTokenSpan(
Dfeature-processor.cc332 TokenSpan CodepointSpanToTokenSpan(const std::vector<Token>& selectable_tokens, in CodepointSpanToTokenSpan() function
402 CodepointSpanToTokenSpan(selectable_tokens, span); in CenterTokenFromClick()
429 CodepointSpanToTokenSpan(selectable_tokens, span); in CenterTokenFromMiddleOfSelection()
Dannotator.cc1478 CodepointSpanToTokenSpan(*tokens, selection_indices); in ModelClassifyText()
/external/libtextclassifier/native/annotator/grammar/
Dgrammar-annotator.cc279 if (const TokenSpan context_span = CodepointSpanToTokenSpan( in ClassifyText()