Searched defs:delim_pattern (Results 1 – 7 of 7) sorted by relevance
| /third_party/mindspore/tests/ut/python/dataset/ |
| D | test_text_tokenizer.py | 307 def regex_tokenizer(first, last, expect_str, delim_pattern, keep_delim_pattern): argument
338 …x_tokenizer(first, last, expect_str, expected_offsets_start, expected_offsets_limit, delim_pattern, argument
|
| /third_party/mindspore/mindspore/ccsrc/minddata/dataset/text/kernels/ |
| D | basic_tokenizer_op.cc | 64 std::string delim_pattern = std::string("\\s+|") + kCommonPattern; in BasicTokenizerOp() local
|
| /third_party/mindspore/mindspore/ccsrc/minddata/dataset/include/dataset/ |
| D | text.h | 384 … : RegexTokenizer(StringToChar(delim_pattern), StringToChar(keep_delim_pattern), with_offsets) {} in RegexTokenizer()
|
| /third_party/mindspore/mindspore/ccsrc/minddata/dataset/api/python/bindings/dataset/text/kernels/ir/ |
| D | bindings.cc | 94 …py::init([](const std::string &delim_pattern, const std::string &keep_delim_pattern, const bool &w… in __anonf0f064540b02()
|
| /third_party/mindspore/mindspore/ccsrc/minddata/dataset/text/ir/kernels/ |
| D | text_ir.cc | 318 RegexTokenizerOperation::RegexTokenizerOperation(std::string delim_pattern, std::string keep_delim_… in RegexTokenizerOperation()
|
| /third_party/mindspore/mindspore/dataset/text/ |
| D | transforms.py | 832 def __init__(self, delim_pattern, keep_delim_pattern='', with_offsets=False): argument
|
| /third_party/mindspore/mindspore/ccsrc/minddata/dataset/api/ |
| D | text.cc | 304 RegexTokenizer::RegexTokenizer(const std::vector<char> &delim_pattern, const std::vector<char> &kee… in RegexTokenizer()
|