Home
last modified time | relevance | path

Searched refs:wp_begin_offset (Results 1 – 5 of 5) sorted by relevance

/external/libtextclassifier/native/utils/
Dbert_tokenizer_test.cc83 EXPECT_THAT(results.wp_begin_offset, ElementsAre(0, 1, 2, 4, 12, 15, 18, 21)); in TEST()
99 EXPECT_THAT(results.wp_begin_offset, ElementsAre(0)); in TEST()
Dbert_tokenizer.h52 std::vector<int> wp_begin_offset; member
Dbert_tokenizer.cc124 result.wp_begin_offset.push_back( in TokenizeIntoWordpieces()
/external/tflite-support/tensorflow_lite_support/cc/text/tokenizers/
Dbert_tokenizer.cc66 std::vector<int>& wp_absolute_begin_offset = result.wp_begin_offset; in TokenizeWordpiece()
Dbert_tokenizer.h54 std::vector<int> wp_begin_offset; member