• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1 //! Incremental BLOB I/O.
2 //!
3 //! Note that SQLite does not provide API-level access to change the size of a
4 //! BLOB; that must be performed through SQL statements.
5 //!
6 //! There are two choices for how to perform IO on a [`Blob`].
7 //!
8 //! 1. The implementations it provides of the `std::io::Read`, `std::io::Write`,
9 //!    and `std::io::Seek` traits.
10 //!
11 //! 2. A positional IO API, e.g. [`Blob::read_at`], [`Blob::write_at`] and
12 //!    similar.
13 //!
14 //! Documenting these in order:
15 //!
16 //! ## 1. `std::io` trait implementations.
17 //!
18 //! `Blob` conforms to `std::io::Read`, `std::io::Write`, and `std::io::Seek`,
19 //! so it plays nicely with other types that build on these (such as
20 //! `std::io::BufReader` and `std::io::BufWriter`). However, you must be careful
21 //! with the size of the blob. For example, when using a `BufWriter`, the
22 //! `BufWriter` will accept more data than the `Blob` will allow, so make sure
23 //! to call `flush` and check for errors. (See the unit tests in this module for
24 //! an example.)
25 //!
26 //! ## 2. Positional IO
27 //!
28 //! `Blob`s also offer a `pread` / `pwrite`-style positional IO api in the form
29 //! of [`Blob::read_at`], [`Blob::write_at`], [`Blob::raw_read_at`],
30 //! [`Blob::read_at_exact`], and [`Blob::raw_read_at_exact`].
31 //!
32 //! These APIs all take the position to read from or write to from as a
33 //! parameter, instead of using an internal `pos` value.
34 //!
35 //! ### Positional IO Read Variants
36 //!
37 //! For the `read` functions, there are several functions provided:
38 //!
39 //! - [`Blob::read_at`]
40 //! - [`Blob::raw_read_at`]
41 //! - [`Blob::read_at_exact`]
42 //! - [`Blob::raw_read_at_exact`]
43 //!
44 //! These can be divided along two axes: raw/not raw, and exact/inexact:
45 //!
46 //! 1. Raw/not raw refers to the type of the destination buffer. The raw
47 //!    functions take a `&mut [MaybeUninit<u8>]` as the destination buffer,
48 //!    where the "normal" functions take a `&mut [u8]`.
49 //!
50 //!    Using `MaybeUninit` here can be more efficient in some cases, but is
51 //!    often inconvenient, so both are provided.
52 //!
53 //! 2. Exact/inexact refers to whether or not the entire buffer must be
54 //!    filled in order for the call to be considered a success.
55 //!
56 //!    The "exact" functions require the provided buffer be entirely filled, or
57 //!    they return an error, whereas the "inexact" functions read as much out of
58 //!    the blob as is available, and return how much they were able to read.
59 //!
60 //!    The inexact functions are preferable if you do not know the size of the
61 //!    blob already, and the exact functions are preferable if you do.
62 //!
63 //! ### Comparison to using the `std::io` traits:
64 //!
65 //! In general, the positional methods offer the following Pro/Cons compared to
66 //! using the implementation `std::io::{Read, Write, Seek}` we provide for
67 //! `Blob`:
68 //!
69 //! 1. (Pro) There is no need to first seek to a position in order to perform IO
70 //!    on it as the position is a parameter.
71 //!
72 //! 2. (Pro) `Blob`'s positional read functions don't mutate the blob in any
73 //!    way, and take `&self`. No `&mut` access required.
74 //!
75 //! 3. (Pro) Positional IO functions return `Err(rusqlite::Error)` on failure,
76 //!    rather than `Err(std::io::Error)`. Returning `rusqlite::Error` is more
77 //!    accurate and convenient.
78 //!
79 //!    Note that for the `std::io` API, no data is lost however, and it can be
80 //!    recovered with `io_err.downcast::<rusqlite::Error>()` (this can be easy
81 //!    to forget, though).
82 //!
83 //! 4. (Pro, for now). A `raw` version of the read API exists which can allow
84 //!    reading into a `&mut [MaybeUninit<u8>]` buffer, which avoids a potential
85 //!    costly initialization step. (However, `std::io` traits will certainly
86 //!    gain this someday, which is why this is only a "Pro, for now").
87 //!
88 //! 5. (Con) The set of functions is more bare-bones than what is offered in
89 //!    `std::io`, which has a number of adapters, handy algorithms, further
90 //!    traits.
91 //!
92 //! 6. (Con) No meaningful interoperability with other crates, so if you need
93 //!    that you must use `std::io`.
94 //!
95 //! To generalize: the `std::io` traits are useful because they conform to a
96 //! standard interface that a lot of code knows how to handle, however that
97 //! interface is not a perfect fit for [`Blob`], so another small set of
98 //! functions is provided as well.
99 //!
100 //! # Example (`std::io`)
101 //!
102 //! ```rust
103 //! # use rusqlite::blob::ZeroBlob;
104 //! # use rusqlite::{Connection, DatabaseName};
105 //! # use std::error::Error;
106 //! # use std::io::{Read, Seek, SeekFrom, Write};
107 //! # fn main() -> Result<(), Box<dyn Error>> {
108 //! let db = Connection::open_in_memory()?;
109 //! db.execute_batch("CREATE TABLE test_table (content BLOB);")?;
110 //!
111 //! // Insert a BLOB into the `content` column of `test_table`. Note that the Blob
112 //! // I/O API provides no way of inserting or resizing BLOBs in the DB -- this
113 //! // must be done via SQL.
114 //! db.execute("INSERT INTO test_table (content) VALUES (ZEROBLOB(10))", [])?;
115 //!
116 //! // Get the row id off the BLOB we just inserted.
117 //! let rowid = db.last_insert_rowid();
118 //! // Open the BLOB we just inserted for IO.
119 //! let mut blob = db.blob_open(DatabaseName::Main, "test_table", "content", rowid, false)?;
120 //!
121 //! // Write some data into the blob. Make sure to test that the number of bytes
122 //! // written matches what you expect; if you try to write too much, the data
123 //! // will be truncated to the size of the BLOB.
124 //! let bytes_written = blob.write(b"01234567")?;
125 //! assert_eq!(bytes_written, 8);
126 //!
127 //! // Move back to the start and read into a local buffer.
128 //! // Same guidance - make sure you check the number of bytes read!
129 //! blob.seek(SeekFrom::Start(0))?;
130 //! let mut buf = [0u8; 20];
131 //! let bytes_read = blob.read(&mut buf[..])?;
132 //! assert_eq!(bytes_read, 10); // note we read 10 bytes because the blob has size 10
133 //!
134 //! // Insert another BLOB, this time using a parameter passed in from
135 //! // rust (potentially with a dynamic size).
136 //! db.execute(
137 //!     "INSERT INTO test_table (content) VALUES (?1)",
138 //!     [ZeroBlob(64)],
139 //! )?;
140 //!
141 //! // given a new row ID, we can reopen the blob on that row
142 //! let rowid = db.last_insert_rowid();
143 //! blob.reopen(rowid)?;
144 //! // Just check that the size is right.
145 //! assert_eq!(blob.len(), 64);
146 //! # Ok(())
147 //! # }
148 //! ```
149 //!
150 //! # Example (Positional)
151 //!
152 //! ```rust
153 //! # use rusqlite::blob::ZeroBlob;
154 //! # use rusqlite::{Connection, DatabaseName};
155 //! # use std::error::Error;
156 //! # fn main() -> Result<(), Box<dyn Error>> {
157 //! let db = Connection::open_in_memory()?;
158 //! db.execute_batch("CREATE TABLE test_table (content BLOB);")?;
159 //! // Insert a blob into the `content` column of `test_table`. Note that the Blob
160 //! // I/O API provides no way of inserting or resizing blobs in the DB -- this
161 //! // must be done via SQL.
162 //! db.execute("INSERT INTO test_table (content) VALUES (ZEROBLOB(10))", [])?;
163 //! // Get the row id off the blob we just inserted.
164 //! let rowid = db.last_insert_rowid();
165 //! // Open the blob we just inserted for IO.
166 //! let mut blob = db.blob_open(DatabaseName::Main, "test_table", "content", rowid, false)?;
167 //! // Write some data into the blob.
168 //! blob.write_at(b"ABCDEF", 2)?;
169 //!
170 //! // Read the whole blob into a local buffer.
171 //! let mut buf = [0u8; 10];
172 //! blob.read_at_exact(&mut buf, 0)?;
173 //! assert_eq!(&buf, b"\0\0ABCDEF\0\0");
174 //!
175 //! // Insert another blob, this time using a parameter passed in from
176 //! // rust (potentially with a dynamic size).
177 //! db.execute(
178 //!     "INSERT INTO test_table (content) VALUES (?1)",
179 //!     [ZeroBlob(64)],
180 //! )?;
181 //!
182 //! // given a new row ID, we can reopen the blob on that row
183 //! let rowid = db.last_insert_rowid();
184 //! blob.reopen(rowid)?;
185 //! assert_eq!(blob.len(), 64);
186 //! # Ok(())
187 //! # }
188 //! ```
189 use std::cmp::min;
190 use std::io;
191 use std::ptr;
192 
193 use super::ffi;
194 use super::types::{ToSql, ToSqlOutput};
195 use crate::{Connection, DatabaseName, Result};
196 
197 mod pos_io;
198 
199 /// Handle to an open BLOB. See
200 /// [`rusqlite::blob`](crate::blob) documentation for in-depth discussion.
201 pub struct Blob<'conn> {
202     conn: &'conn Connection,
203     blob: *mut ffi::sqlite3_blob,
204     // used by std::io implementations,
205     pos: i32,
206 }
207 
208 impl Connection {
209     /// Open a handle to the BLOB located in `row_id`,
210     /// `column`, `table` in database `db`.
211     ///
212     /// # Failure
213     ///
214     /// Will return `Err` if `db`/`table`/`column` cannot be converted to a
215     /// C-compatible string or if the underlying SQLite BLOB open call
216     /// fails.
217     #[inline]
blob_open<'a>( &'a self, db: DatabaseName<'_>, table: &str, column: &str, row_id: i64, read_only: bool, ) -> Result<Blob<'a>>218     pub fn blob_open<'a>(
219         &'a self,
220         db: DatabaseName<'_>,
221         table: &str,
222         column: &str,
223         row_id: i64,
224         read_only: bool,
225     ) -> Result<Blob<'a>> {
226         let c = self.db.borrow_mut();
227         let mut blob = ptr::null_mut();
228         let db = db.as_cstring()?;
229         let table = super::str_to_cstring(table)?;
230         let column = super::str_to_cstring(column)?;
231         let rc = unsafe {
232             ffi::sqlite3_blob_open(
233                 c.db(),
234                 db.as_ptr(),
235                 table.as_ptr(),
236                 column.as_ptr(),
237                 row_id,
238                 !read_only as std::os::raw::c_int,
239                 &mut blob,
240             )
241         };
242         c.decode_result(rc).map(|_| Blob {
243             conn: self,
244             blob,
245             pos: 0,
246         })
247     }
248 }
249 
250 impl Blob<'_> {
251     /// Move a BLOB handle to a new row.
252     ///
253     /// # Failure
254     ///
255     /// Will return `Err` if the underlying SQLite BLOB reopen call fails.
256     #[inline]
reopen(&mut self, row: i64) -> Result<()>257     pub fn reopen(&mut self, row: i64) -> Result<()> {
258         let rc = unsafe { ffi::sqlite3_blob_reopen(self.blob, row) };
259         if rc != ffi::SQLITE_OK {
260             return self.conn.decode_result(rc);
261         }
262         self.pos = 0;
263         Ok(())
264     }
265 
266     /// Return the size in bytes of the BLOB.
267     #[inline]
268     #[must_use]
size(&self) -> i32269     pub fn size(&self) -> i32 {
270         unsafe { ffi::sqlite3_blob_bytes(self.blob) }
271     }
272 
273     /// Return the current size in bytes of the BLOB.
274     #[inline]
275     #[must_use]
len(&self) -> usize276     pub fn len(&self) -> usize {
277         self.size().try_into().unwrap()
278     }
279 
280     /// Return true if the BLOB is empty.
281     #[inline]
282     #[must_use]
is_empty(&self) -> bool283     pub fn is_empty(&self) -> bool {
284         self.size() == 0
285     }
286 
287     /// Close a BLOB handle.
288     ///
289     /// Calling `close` explicitly is not required (the BLOB will be closed
290     /// when the `Blob` is dropped), but it is available, so you can get any
291     /// errors that occur.
292     ///
293     /// # Failure
294     ///
295     /// Will return `Err` if the underlying SQLite close call fails.
296     #[inline]
close(mut self) -> Result<()>297     pub fn close(mut self) -> Result<()> {
298         self.close_()
299     }
300 
301     #[inline]
close_(&mut self) -> Result<()>302     fn close_(&mut self) -> Result<()> {
303         let rc = unsafe { ffi::sqlite3_blob_close(self.blob) };
304         self.blob = ptr::null_mut();
305         self.conn.decode_result(rc)
306     }
307 }
308 
309 impl io::Read for Blob<'_> {
310     /// Read data from a BLOB incrementally. Will return Ok(0) if the end of
311     /// the blob has been reached.
312     ///
313     /// # Failure
314     ///
315     /// Will return `Err` if the underlying SQLite read call fails.
316     #[inline]
read(&mut self, buf: &mut [u8]) -> io::Result<usize>317     fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
318         let max_allowed_len = (self.size() - self.pos) as usize;
319         let n = min(buf.len(), max_allowed_len) as i32;
320         if n <= 0 {
321             return Ok(0);
322         }
323         let rc = unsafe { ffi::sqlite3_blob_read(self.blob, buf.as_mut_ptr().cast(), n, self.pos) };
324         self.conn
325             .decode_result(rc)
326             .map(|_| {
327                 self.pos += n;
328                 n as usize
329             })
330             .map_err(|err| io::Error::new(io::ErrorKind::Other, err))
331     }
332 }
333 
334 impl io::Write for Blob<'_> {
335     /// Write data into a BLOB incrementally. Will return `Ok(0)` if the end of
336     /// the blob has been reached; consider using `Write::write_all(buf)`
337     /// if you want to get an error if the entirety of the buffer cannot be
338     /// written.
339     ///
340     /// This function may only modify the contents of the BLOB; it is not
341     /// possible to increase the size of a BLOB using this API.
342     ///
343     /// # Failure
344     ///
345     /// Will return `Err` if the underlying SQLite write call fails.
346     #[inline]
write(&mut self, buf: &[u8]) -> io::Result<usize>347     fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
348         let max_allowed_len = (self.size() - self.pos) as usize;
349         let n = min(buf.len(), max_allowed_len) as i32;
350         if n <= 0 {
351             return Ok(0);
352         }
353         let rc = unsafe { ffi::sqlite3_blob_write(self.blob, buf.as_ptr() as *mut _, n, self.pos) };
354         self.conn
355             .decode_result(rc)
356             .map(|_| {
357                 self.pos += n;
358                 n as usize
359             })
360             .map_err(|err| io::Error::new(io::ErrorKind::Other, err))
361     }
362 
363     #[inline]
flush(&mut self) -> io::Result<()>364     fn flush(&mut self) -> io::Result<()> {
365         Ok(())
366     }
367 }
368 
369 impl io::Seek for Blob<'_> {
370     /// Seek to an offset, in bytes, in BLOB.
371     #[inline]
seek(&mut self, pos: io::SeekFrom) -> io::Result<u64>372     fn seek(&mut self, pos: io::SeekFrom) -> io::Result<u64> {
373         let pos = match pos {
374             io::SeekFrom::Start(offset) => offset as i64,
375             io::SeekFrom::Current(offset) => i64::from(self.pos) + offset,
376             io::SeekFrom::End(offset) => i64::from(self.size()) + offset,
377         };
378 
379         if pos < 0 {
380             Err(io::Error::new(
381                 io::ErrorKind::InvalidInput,
382                 "invalid seek to negative position",
383             ))
384         } else if pos > i64::from(self.size()) {
385             Err(io::Error::new(
386                 io::ErrorKind::InvalidInput,
387                 "invalid seek to position past end of blob",
388             ))
389         } else {
390             self.pos = pos as i32;
391             Ok(pos as u64)
392         }
393     }
394 }
395 
396 #[allow(unused_must_use)]
397 impl Drop for Blob<'_> {
398     #[inline]
drop(&mut self)399     fn drop(&mut self) {
400         self.close_();
401     }
402 }
403 
404 /// BLOB of length N that is filled with zeroes.
405 ///
406 /// Zeroblobs are intended to serve as placeholders for BLOBs whose content is
407 /// later written using incremental BLOB I/O routines.
408 ///
409 /// A negative value for the zeroblob results in a zero-length BLOB.
410 #[derive(Copy, Clone)]
411 pub struct ZeroBlob(pub i32);
412 
413 impl ToSql for ZeroBlob {
414     #[inline]
to_sql(&self) -> Result<ToSqlOutput<'_>>415     fn to_sql(&self) -> Result<ToSqlOutput<'_>> {
416         let ZeroBlob(length) = *self;
417         Ok(ToSqlOutput::ZeroBlob(length))
418     }
419 }
420 
421 #[cfg(test)]
422 mod test {
423     use crate::{Connection, DatabaseName, Result};
424     use std::io::{BufRead, BufReader, BufWriter, Read, Seek, SeekFrom, Write};
425 
db_with_test_blob() -> Result<(Connection, i64)>426     fn db_with_test_blob() -> Result<(Connection, i64)> {
427         let db = Connection::open_in_memory()?;
428         let sql = "BEGIN;
429                    CREATE TABLE test (content BLOB);
430                    INSERT INTO test VALUES (ZEROBLOB(10));
431                    END;";
432         db.execute_batch(sql)?;
433         let rowid = db.last_insert_rowid();
434         Ok((db, rowid))
435     }
436 
437     #[test]
test_blob() -> Result<()>438     fn test_blob() -> Result<()> {
439         let (db, rowid) = db_with_test_blob()?;
440 
441         let mut blob = db.blob_open(DatabaseName::Main, "test", "content", rowid, false)?;
442         assert!(!blob.is_empty());
443         assert_eq!(10, blob.len());
444         assert_eq!(4, blob.write(b"Clob").unwrap());
445         assert_eq!(6, blob.write(b"567890xxxxxx").unwrap()); // cannot write past 10
446         assert_eq!(0, blob.write(b"5678").unwrap()); // still cannot write past 10
447         blob.flush().unwrap();
448 
449         blob.reopen(rowid)?;
450         blob.close()?;
451 
452         blob = db.blob_open(DatabaseName::Main, "test", "content", rowid, true)?;
453         let mut bytes = [0u8; 5];
454         assert_eq!(5, blob.read(&mut bytes[..]).unwrap());
455         assert_eq!(&bytes, b"Clob5");
456         assert_eq!(5, blob.read(&mut bytes[..]).unwrap());
457         assert_eq!(&bytes, b"67890");
458         assert_eq!(0, blob.read(&mut bytes[..]).unwrap());
459 
460         blob.seek(SeekFrom::Start(2)).unwrap();
461         assert_eq!(5, blob.read(&mut bytes[..]).unwrap());
462         assert_eq!(&bytes, b"ob567");
463 
464         // only first 4 bytes of `bytes` should be read into
465         blob.seek(SeekFrom::Current(-1)).unwrap();
466         assert_eq!(4, blob.read(&mut bytes[..]).unwrap());
467         assert_eq!(&bytes, b"78907");
468 
469         blob.seek(SeekFrom::End(-6)).unwrap();
470         assert_eq!(5, blob.read(&mut bytes[..]).unwrap());
471         assert_eq!(&bytes, b"56789");
472 
473         blob.reopen(rowid)?;
474         assert_eq!(5, blob.read(&mut bytes[..]).unwrap());
475         assert_eq!(&bytes, b"Clob5");
476 
477         // should not be able to seek negative or past end
478         blob.seek(SeekFrom::Current(-20)).unwrap_err();
479         blob.seek(SeekFrom::End(0)).unwrap();
480         blob.seek(SeekFrom::Current(1)).unwrap_err();
481 
482         // write_all should detect when we return Ok(0) because there is no space left,
483         // and return a write error
484         blob.reopen(rowid)?;
485         blob.write_all(b"0123456789x").unwrap_err();
486         Ok(())
487     }
488 
489     #[test]
test_blob_in_bufreader() -> Result<()>490     fn test_blob_in_bufreader() -> Result<()> {
491         let (db, rowid) = db_with_test_blob()?;
492 
493         let mut blob = db.blob_open(DatabaseName::Main, "test", "content", rowid, false)?;
494         assert_eq!(8, blob.write(b"one\ntwo\n").unwrap());
495 
496         blob.reopen(rowid)?;
497         let mut reader = BufReader::new(blob);
498 
499         let mut line = String::new();
500         assert_eq!(4, reader.read_line(&mut line).unwrap());
501         assert_eq!("one\n", line);
502 
503         line.truncate(0);
504         assert_eq!(4, reader.read_line(&mut line).unwrap());
505         assert_eq!("two\n", line);
506 
507         line.truncate(0);
508         assert_eq!(2, reader.read_line(&mut line).unwrap());
509         assert_eq!("\0\0", line);
510         Ok(())
511     }
512 
513     #[test]
test_blob_in_bufwriter() -> Result<()>514     fn test_blob_in_bufwriter() -> Result<()> {
515         let (db, rowid) = db_with_test_blob()?;
516 
517         {
518             let blob = db.blob_open(DatabaseName::Main, "test", "content", rowid, false)?;
519             let mut writer = BufWriter::new(blob);
520 
521             // trying to write too much and then flush should fail
522             assert_eq!(8, writer.write(b"01234567").unwrap());
523             assert_eq!(8, writer.write(b"01234567").unwrap());
524             writer.flush().unwrap_err();
525         }
526 
527         {
528             // ... but it should've written the first 10 bytes
529             let mut blob = db.blob_open(DatabaseName::Main, "test", "content", rowid, false)?;
530             let mut bytes = [0u8; 10];
531             assert_eq!(10, blob.read(&mut bytes[..]).unwrap());
532             assert_eq!(b"0123456701", &bytes);
533         }
534 
535         {
536             let blob = db.blob_open(DatabaseName::Main, "test", "content", rowid, false)?;
537             let mut writer = BufWriter::new(blob);
538 
539             // trying to write_all too much should fail
540             writer.write_all(b"aaaaaaaaaabbbbb").unwrap();
541             writer.flush().unwrap_err();
542         }
543 
544         {
545             // ... but it should've written the first 10 bytes
546             let mut blob = db.blob_open(DatabaseName::Main, "test", "content", rowid, false)?;
547             let mut bytes = [0u8; 10];
548             assert_eq!(10, blob.read(&mut bytes[..]).unwrap());
549             assert_eq!(b"aaaaaaaaaa", &bytes);
550             Ok(())
551         }
552     }
553 
554     #[test]
zero_blob() -> Result<()>555     fn zero_blob() -> Result<()> {
556         use crate::types::ToSql;
557         let zb = super::ZeroBlob(1);
558         assert!(zb.to_sql().is_ok());
559         Ok(())
560     }
561 }
562