• Home
Name Date Size #Lines LOC

..--

README.mdD12-May-2024647 1310

en-huge.txtD12-May-2024599 KiB22,92822,927

en-medium.txtD12-May-202460 KiB2,1712,170

en-small.txtD12-May-20241,019 4039

en-teeny.txtD12-May-202428 21

en-tiny.txtD12-May-2024108 32

ru-huge.txtD12-May-2024599 KiB12,68612,685

ru-medium.txtD12-May-202460 KiB1,3241,323

ru-small.txtD12-May-20241 KiB1918

ru-teeny.txtD12-May-202442 21

ru-tiny.txtD12-May-2024174 32

zh-huge.txtD12-May-2024599 KiB22,00122,000

zh-medium.txtD12-May-202460 KiB1,4661,465

zh-small.txtD12-May-20241 KiB2928

zh-teeny.txtD12-May-202431 21

zh-tiny.txtD12-May-2024110 43

README.md

1These were downloaded and derived from the Open Subtitles data set:
2https://opus.nlpl.eu/OpenSubtitles-v2018.php
3
4The specific way in which they were modified has been lost to time, but it's
5likely they were just a simple truncation based on target file sizes for
6various benchmarks.
7
8The main reason why we have them is that it gives us a way to test similar
9inputs on non-ASCII text. Normally this wouldn't matter for a substring search
10implementation, but because of the heuristics used to pick a priori determined
11"rare bytes" to base a prefilter on, it's possible for this heuristic to do
12more poorly on non-ASCII text than one might expect.
13