Searched defs:delim_pattern (Results 1 – 7 of 7) sorted by relevance
| /third_party/mindspore/mindspore-src/source/tests/ut/python/dataset/ |
| D | test_text_tokenizer.py | 327 def regex_tokenizer(first, last, expect_str, delim_pattern, keep_delim_pattern): argument
360 …x_tokenizer(first, last, expect_str, expected_offsets_start, expected_offsets_limit, delim_pattern, argument
|
| /third_party/mindspore/mindspore-src/source/mindspore/ccsrc/minddata/dataset/text/kernels/ |
| D | basic_tokenizer_op.cc | 64 std::string delim_pattern = std::string("\\s+|") + kCommonPattern; in BasicTokenizerOp() local
|
| /third_party/mindspore/mindspore-src/source/mindspore/ccsrc/minddata/dataset/include/dataset/ |
| D | text.h | 734 … : RegexTokenizer(StringToChar(delim_pattern), StringToChar(keep_delim_pattern), with_offsets) {} in RegexTokenizer()
|
| /third_party/mindspore/mindspore-src/source/mindspore/ccsrc/minddata/dataset/api/python/bindings/dataset/text/kernels/ir/ |
| D | bindings.cc | 115 …py::init([](const std::string &delim_pattern, const std::string &keep_delim_pattern, const bool &w… in __anon2bada04b0f02()
|
| /third_party/mindspore/mindspore-src/source/mindspore/python/mindspore/dataset/text/ |
| D | transforms.py | 1556 def __init__(self, delim_pattern, keep_delim_pattern='', with_offsets=False): argument
|
| /third_party/mindspore/mindspore-src/source/mindspore/ccsrc/minddata/dataset/text/ir/kernels/ |
| D | text_ir.cc | 456 RegexTokenizerOperation::RegexTokenizerOperation(std::string delim_pattern, std::string keep_delim_… in RegexTokenizerOperation()
|
| /third_party/mindspore/mindspore-src/source/mindspore/ccsrc/minddata/dataset/api/ |
| D | text.cc | 320 RegexTokenizer::RegexTokenizer(const std::vector<char> &delim_pattern, const std::vector<char> &kee… in RegexTokenizer()
|