Searched defs:test_tokens (Results 1 – 2 of 2) sorted by relevance
45 def test_tokens(data_filename, tokens_filename, verbose=False): function
16 test_tokens = [ variable in TestTokenStream