1Name: sqlite 2URL: http://sqlite.org/ 3Version: 3.7.6.3 4Included In Release: Yes 5Security Critical: Yes 6License: Public domain 7 8Instructions for importing a new release of SQLite from sqlite.org. 9 10Note: our current base version is 3.7.6.3. 11 12First, you need to be on Linux. 13 14# Determine the versions of the release you want and the release we currently 15# have. (See the VERSION file to determine which release we currently have.) 16# You may wish to consult http://www.sqlite.org/changes.html to find out what 17# changes have been made in each release. 18# Note - this is just an example. Always refer to the version above for our 19# real current version. 20# Set some variables to remember the versions, e.g.: 21BASE=3.7.6.3 22LATEST=3.7.6.4 23 24# Get to the src/third_party directory in your Chromium client: 25cd src/third_party 26 27# Download the .tar.gz files for the releases: 28# (If the URL changes you might need to find the new one.) 29# TODO(shess): Rewrite this to track the new naming format. Meanwhile, 30# manually navigate to www.sqlite.org and find downloads, use "legacy" version. 31wget http://www.sqlite.org/sqlite-$BASE.tar.gz 32wget http://www.sqlite.org/sqlite-$LATEST.tar.gz 33 34# Extract the vanilla current and desired versions: 35tar xzf sqlite-$BASE.tar.gz 36tar xzf sqlite-$LATEST.tar.gz 37 38# Use kdiff3 to merge the changes: 39kdiff3 -m sqlite-$BASE sqlite-$LATEST sqlite 40 41# Resolve any conflicts. Figure out if we've got everything we should 42# have (see below), or if we can omit any changes we no longer need. 43 44# Change to the sqlite directory: 45cd sqlite 46 47# Run the google_generate_amalgamation.sh script: 48./google_generate_amalgamation.sh 49 50# Find a sucker. Send review. 51# TODO(shess) Describe an appropriate comment style. Seems like it 52# should at least include the SQLite version number. 53 54-------------------------------------------- 55 56For reference, all of our local patches are also kept as .patch files in the 57sqlite directory. Here is a list of the patches, in the order they should be 58applied to a vanilla SQLite (of the version we currently have) to get, in 59principle, exactly what is checked in: 60 61misc.patch 62preload-cache.patch 63safe-tolower.patch 64fts2.patch 65fts3.patch 66fts3_85522.patch 67icu-regexp.patch 68icu-shell.patch 69attach-integer.patch 70webdb.patch 71test.patch 72mac_time_machine.patch 73system-sqlite.patch 74sqlite-3.7.6.3-fix-out-of-scope-memory-reference.patch 75misalignment.patch 76memcmp.patch 77separate_cache_pool.patch 78recover.patch 79 80So, e.g. you could do this to apply all our patches to vanilla SQLite: 81 82cd sqlite-$LATEST 83patch -p0 < ../sqlite/misc.patch 84patch -p0 < ../sqlite/preload-cache.patch 85patch -p0 < ../sqlite/fts2.patch 86patch -p0 < ../sqlite/fts3.patch 87patch -p0 < ../sqlite/fts3_85522.patch 88patch -p0 < ../sqlite/icu-shell.patch 89patch -p0 < ../sqlite/webdb.patch 90patch -p0 < ../sqlite/test.patch 91patch -p0 < ../sqlite/mac_time_machine.patch 92patch -p0 < ../sqlite/system-sqlite.patch 93patch -p0 < ../sqlite/sqlite-3.7.6.3-fix-out-of-scope-memory-reference.patch 94patch -p0 < ../sqlite/misalignment.patch 95patch -p0 < ../sqlite/memcmp.patch 96patch -p0 < ../sqlite/separate_cache_pool.patch 97patch -p0 < ../sqlite/recover.patch 98 99This will only be the case if all changes we make also update the corresponding 100patch files. Therefore please remember to do that whenever you make a change! 101 102Descriptions of the changes we've made can be found at the bottom of this file. 103 104-------------------------------------------- 105 106How to run the SQLite tests for the Chromium version of SQLite on Linux. 107 108Prerequisties: On my corp Ubuntu 8.04 workstation, I needed to install the 109following packages: 110sudo apt-get install tcl8.4-dev libicu-dev 111 112cd src/third_party/sqlite/src 113mkdir build 114cd build 115make -f ../Makefile.linux-gcc testfixture 116make -f ../Makefile.linux-gcc test > /tmp/test.log 117egrep -v 'Ok$' /tmp/test.log 118# For an ideal test run, you would see: 119# 0 errors out of 57887 tests 120# However, the current situation on my corp Linux Ubuntu 8.04 machine, with 121# test run on a locally mounted directory, is the failure of: 122# "rollback-2.3", "tkt3457-1.4" 123# I do not know why, but it is not related to our fts2.c changes -- I backed 124# them out to check. 125 126Chris Evans <cevans@google.com>, Oct 1, 2009 127 128-------------------------------------------- 129 130As of May 07, 2010, these are our changes from sqlite_vendor: 131 132 - A fix for a crash passing an integer expression to ATTACH / DETACH. See 133 attach-integer.patch 134 - A fix for a crash mis-calling the REGEXP() function of the ICU extension. 135 See icu-regexp.patch 136 - A large number of fts2 robustness fixes against corrupt data in its metadata 137 tables. 138 - fts2.c disables fts2_tokenizer(). 139 - fts3.c disables fts3_tokenizer(). 140 - Tweak to SQLITE_EXTENSION_INIT* in sqlite3ext.h. 141 - That implied a change in src/test_autoext.c for testing. 142 - Added fts.test in tests, modified quick.test. 143 - Modifications to Makefile.linux-gcc and main.mk for compiling 144 SQLite tests. 145 - Compile warning (cast to void* for sqlite3_free) fixed in func.c. 146 - Avoid using tolower() in fts code which causes problem in some locales, see: 147 safe-tolower.patch 148 http://crbug.com/15261 149 http://www.sqlite.org/src/tktview/991789d9f3136a0460dc83a33e815c1aa9757c26 150 - Check that the third argument to memset() is nonzero in expr.c to avoid 151 a linker warning when the compiler can optimize it to a constant zero 152 (e.g. see http://www.sqlite.org/cvstrac/tktview?tn=3765,39) 153 154Changes from Chrome: 155 - I marked all changes I made with "evanm", so you can find them with 156 "grep evanm *". 157 - Most files include sqlite3ext.h with SQLITE_CORE #defined, but two don't: 158 fts2_tokenizer.c and icu.c. Without this #define, the calls in 159 fts2_tokenizer.c try to go through some pointer to the sqlite API instead 160 of calling the functions directly (to work as a loadable module), but then 161 crash (because the other files never initialize that loadable module 162 support). As a hack I #defined it in these files, but it'd be nice to 163 figure out what really ought to happen here (perhaps this file is new and 164 hasn't been tested to verify it works right). Update: Seems this is an 165 issue we get because we're using fts2 instead of fts3. 166 - shell_icu_win.c and shell_icu_linux.c are Chrome-specific files used to load 167 our ICU data. shell.c has been modifed to call into these files. 168 - fts2_icu.c and fts3_icu.c have a critical bug. U8_NEXT is used over 169 a UTF-16 string. It's rep$ by U16_NEXT (jungshik) 170 - Added a new function sqlite3_preload we use to prime the database cache. It 171 allows much faster performance by reading the file in one contiguous 172 operation rather than bringing it in organically, which involves a lot of 173 seeking. This change also required sqlite3PcacheGetCachesize to be compiled 174 even outside SQLITE_TEST. 175 - Added a new function chromium_sqlite3_initialize_win_sqlite3_file() 176 at the end of os_win.c. It allows the Windows-specific Chromium VFS 177 to reuse most of the win32 SQLite VFS. 178 - Added a new function 179 chromium_sqlite3_initialize_unix_sqlite3_file() and made 180 fillInUnixFile() non-static in os_unix.c. It allows the 181 Linux-specific Chromium VFS to reuse most of the unix SQLite VFS. 182 - Exposed three functions that deal with unused file descriptors in 183 os_unix.c, to allow Chromium's Posix VFS implementation in 184 WebKit/WebCore/platform/sql/chromium/SQLiteFileSystemChromiumPosix.cpp 185 to correctly implement the "unused file descriptors" logic in the 186 xDlOpen() method. The new functions are 187 chromium_sqlite3_get_reusable_file_handle(), 188 chromium_sqlite3_update_reusable_file_handle() and 189 chromium_sqlite3_destroy_reusable_file_handle(). Also, added the 190 chromium_sqlite3_fill_in_unix_sqlite3_file() function that calls 191 fillInUnixFile(), which will be made static again as soon as a 192 WebKit patch using the new function lands. 193 - From mac_time_machine.patch: 194 When __APPLE__ and when creating a -journal file with any unix-type vfs, 195 determine if the database for which the journal is being created has been 196 excluded from being backed up using Apple's Time Machine and if so then also 197 exclude the journal. These changes were made in pager.c with includes of 198 Apple interfaces being made in sqliteInt.h. In order to eliminate a symbol 199 conflict with an Apple library after amalgamation it was also necessary to 200 rename fts3_porter.c's 'cType' to 'vOrCType'. 201 - fts3_85522.patch allows fts3 to work if PRAGMA is not authorized. 202 - src/recover.c file implements a virtual table which can read 203 through corruption. 204 - Enable the macro 'SQLITE_TEMP_STORE=3' for Android. 205 - memcmp.patch backports ASAN-related fixes from SQLite trunk. 206