tokenization.txt represents the state of the spec implemented in Tokenizer.java.

To get a diffable version corresponding to the current spec:
lynx -display_charset=utf-8 -dump -nolist http://www.whatwg.org/specs/web-apps/current-work/multipage/tokenization.html > current.txt

tree-construction.txt represents the state of the spec implemented in TreeBuilder.java.

To get a diffable version corresponding to the current spec:
lynx -display_charset=utf-8 -dump -nolist http://www.whatwg.org/specs/web-apps/current-work/multipage/tree-construction.html > current.txt


The text of the files in this directory comes from the WHATWG HTML 5 spec 
which carries the following notice:
© Copyright 2004-2010 Apple Computer, Inc., Mozilla Foundation, and Opera Software ASA.
You are granted a license to use, reproduce and create derivative works of this document.