Home
last modified time | relevance | path

Searched defs:clean_up_tokenization_spaces (Results 1 – 4 of 4) sorted by relevance

/external/executorch/examples/mediatek/aot_utils/llm_utils/tokenizers_/
Dtokenization_llama.py174 clean_up_tokenization_spaces=False, argument
/external/executorch/examples/mediatek/models/llm_models/weights/Llama-3.2-1B-Instruct/
Dtokenizer_config.json2054 "clean_up_tokenization_spaces": true, boolean
/external/executorch/examples/mediatek/models/llm_models/weights/llama3-8B-instruct/
Dtokenizer_config.json2054 "clean_up_tokenization_spaces": true, boolean
/external/executorch/examples/mediatek/models/llm_models/weights/Llama-3.2-3B-Instruct/
Dtokenizer_config.json2054 "clean_up_tokenization_spaces": true, boolean