| Name | Date | Size | #Lines | LOC | ||
|---|---|---|---|---|---|---|
| .. | - | - | ||||
| test/ | 04-Jul-2025 | - | 128,425 | 128,315 | ||
| TARGETS | D | 04-Jul-2025 | 232 | 9 | 5 | |
| __init__.py | D | 04-Jul-2025 | 0 | 1 | 0 | |
| base64.h | D | 04-Jul-2025 | 6.5 KiB | 216 | 150 | |
| bpe_tokenizer.cpp | D | 04-Jul-2025 | 10.2 KiB | 321 | 196 | |
| bpe_tokenizer.h | D | 04-Jul-2025 | 1.6 KiB | 59 | 36 | |
| targets.bzl | D | 04-Jul-2025 | 2.5 KiB | 97 | 90 | |
| tiktoken.cpp | D | 04-Jul-2025 | 13.7 KiB | 449 | 321 | |
| tiktoken.h | D | 04-Jul-2025 | 2.8 KiB | 94 | 62 | |
| tokenizer.h | D | 04-Jul-2025 | 2 KiB | 86 | 59 | |
| tokenizer.py | D | 04-Jul-2025 | 5.2 KiB | 152 | 103 | |
| utils.py | D | 04-Jul-2025 | 658 | 20 | 11 |