Searched defs:UnicodeCharTokenizer (Results 1 – 2 of 2) sorted by relevance
457 class UnicodeCharTokenizer(TextTensorOperation): class
382 UnicodeCharTokenizer::UnicodeCharTokenizer(bool with_offsets) : data_(std::make_shared<Data>(with_o… in UnicodeCharTokenizer() function in mindspore::dataset::text::UnicodeCharTokenizer