Searched refs:encode_tokens (Results 1 – 1 of 1) sorted by relevance
42 def encode_tokens(tokenizer, string, bos=True, device="cpu"): function158 encoded = encode_tokens(tokenizer, args.prompt, bos=True, device="cpu")