Searched refs:CachedTokenizer (Results 1 – 1 of 1) sorted by relevance
128 class CachedTokenizer(object): class153 generate_tokens = CachedTokenizer().generate_tokens