README.md
1IOWOW — `C11` utility library and persistent key/value storage engine
2==================================================================================================================
3
4[](https://github.com/Softmotions/iowow/blob/master/LICENSE)
5
6
7Website https://iowow.softmotions.com
8
9# Key components
10
11* [utils](https://github.com/Softmotions/iowow/tree/master/src/utils) Useful data structures and various commonly used routines.
12* [json](https://github.com/Softmotions/iowow/tree/master/src/json) JSON parsing and manipulation routines incluing JSON Patch/Pointers support.
13* [iwkv.h](https://github.com/Softmotions/iowow/blob/master/src/kv/iwkv.h) Persistent key/value database engine.
14* [iwfsmfile.h](https://github.com/Softmotions/iowow/blob/master/src/fs/iwfsmfile.h) File blocks allocation manager like `malloc()` on files.
15
16## Used by
17
18* EJDB2 — Embeddable JSON database engine. http://ejdb.org
19* [Wirow video conferencing platform](https://github.com/wirow-io/wirow-server/)
20* [Pure C Asynchronous HTTP/IO library with websockets, SSL, routing. ](https://github.com/Softmotions/iwnet)
21
22## IWKV Features
23
24* Support of multiple key-value databases within a single file
25* Online database backups
26* Native support of integer keys
27* [Write Ahead Logging (WAL) support](http://iowow.io/wal)
28* Ultra-fast traversal of database records
29* Compound keys support
30* Good performance comparing its main competitors: `lmdb`, `leveldb`, `kyoto cabinet`
31* Tiny C11 library (200Kb) can be easily embedded into any software
32
33[](https://iowow.softmotions.com/articles/intro/)
34
35## IWKV Limitations
36
37* Maximum iwkv storage file size: `512 GB (0x7fffffff80)`
38* Total size of a single key+value record must be not greater than `255Mb (0xfffffff)`
39
40# Supported platforms
41
42## Linux
43
44#### Building debian packages
45
46```sh
47mkdir build && cd build
48cmake .. -DCMAKE_BUILD_TYPE=Release -DPACKAGE_DEB=ON
49make package
50```
51
52### RPM based Linux distributions
53```sh
54mkdir build && cd build
55cmake .. -DCMAKE_BUILD_TYPE=Release -DPACKAGE_RPM=ON
56make package
57```
58
59## FreeBSD
60
61Successfully tested on FreeBSD 10/11
62
63## OSX
64
65Successfully tested on OSX 10.12/10.13
66
67## Windows
68
69[Cross-compilation for windows](http://iowow.io/iw/win)
70
71
72## MIPS based systems (+big-endian)
73
74Successfully tested on Debian 9.4, MIPS 32, gcc 6.x compiler.
75
76# Examples
77
78[src/kv/examples](https://github.com/Softmotions/iowow/tree/master/src/kv/examples)
79
80## Store and retrieve records
81
82```c
83#include <iowow/iwkv.h>
84#include <string.h>
85#include <stdlib.h>
86
87int main() {
88 IWKV_OPTS opts = {
89 .path = "example1.db",
90 .oflags = IWKV_TRUNC // Cleanup database before open
91 };
92 IWKV iwkv;
93 IWDB mydb;
94 iwrc rc = iwkv_open(&opts, &iwkv);
95 if (rc) {
96 iwlog_ecode_error3(rc);
97 return 1;
98 }
99 // Now open mydb
100 // - Database id: 1
101 rc = iwkv_db(iwkv, 1, 0, &mydb);
102 if (rc) {
103 iwlog_ecode_error2(rc, "Failed to open mydb");
104 return 1;
105 }
106 // Work with db: put/get value
107 IWKV_val key, val;
108 key.data = "foo";
109 key.size = strlen(key.data);
110 val.data = "bar";
111 val.size = strlen(val.data);
112
113 fprintf(stdout, "put: %.*s => %.*s\n",
114 (int) key.size, (char *) key.data,
115 (int) val.size, (char *) val.data);
116
117 rc = iwkv_put(mydb, &key, &val, 0);
118 if (rc) {
119 iwlog_ecode_error3(rc);
120 return rc;
121 }
122 // Retrieve value associated with `foo` key
123 val.data = 0;
124 val.size = 0;
125 rc = iwkv_get(mydb, &key, &val);
126 if (rc) {
127 iwlog_ecode_error3(rc);
128 return rc;
129 }
130
131 fprintf(stdout, "get: %.*s => %.*s\n",
132 (int) key.size, (char *) key.data,
133 (int) val.size, (char *) val.data);
134
135 iwkv_val_dispose(&val);
136 iwkv_close(&iwkv);
137 return 0;
138}
139```
140**Compile and run:**
141
142```sh
143gcc -std=gnu11 -Wall -pedantic -c -o example1.o example1.c
144gcc -o example1 example1.o -liowow
145
146./example1
147 put: foo => bar
148 get: foo => bar
149```
150
151## Cursor iteration example
152
153```c
154///
155/// Fills database with a set of football table records
156/// then traverse records according to club name in ascending and descending orders.
157///
158
159#include "iwkv.h"
160#include <string.h>
161#include <stdlib.h>
162#include <stdint.h>
163
164static struct data_s {
165 const char *club;
166 uint8_t points;
167} _points[] = {
168
169 { "Aston Villa", 25 },
170 { "Manchester City", 57 },
171 { "Arsenal", 40 },
172 { "Everton", 37 },
173 { "West Ham United", 27 },
174 { "Tottenham Hotspur", 41 },
175 { "Wolverhampton Wanderers", 43 },
176 { "Norwich City", 21 },
177 { "Leicester City", 53 },
178 { "Manchester United", 45 },
179 { "Newcastle United", 35 },
180 { "Brighton & Hove Albion", 29 },
181 { "AFC Bournemouth", 27 },
182 { "Crystal Palace", 39 },
183 { "Sheffield United", 43 },
184 { "Burnley", 39 },
185 { "Southampton", 34 },
186 { "Watford", 27 },
187 { "Chelsea", 48 },
188 { "Liverpool", 82 },
189};
190
191static iwrc run(void) {
192 IWKV_OPTS opts = {
193 .path = "cursor1.db",
194 .oflags = IWKV_TRUNC // Cleanup database before open
195 };
196 IWKV iwkv;
197 IWDB db;
198 IWKV_cursor cur = 0;
199 iwrc rc = iwkv_open(&opts, &iwkv);
200 RCRET(rc);
201
202 rc = iwkv_db(iwkv, 1, 0, &db);
203 RCGO(rc, finish);
204
205 for (int i = 0; i < sizeof(_points) / sizeof(_points[0]); ++i) {
206 struct data_s *n = &_points[i];
207 IWKV_val key = { .data = (void *) n->club, .size = strlen(n->club) };
208 IWKV_val val = { .data = &n->points, .size = sizeof(n->points) };
209 RCC(rc, finish, iwkv_put(db, &key, &val, 0));
210 }
211
212 fprintf(stdout, ">>>> Traverse in descending order\n");
213 RCC(rc, finish, iwkv_cursor_open(db, &cur, IWKV_CURSOR_BEFORE_FIRST, 0));
214 while ((rc = iwkv_cursor_to(cur, IWKV_CURSOR_NEXT)) == 0) {
215 IWKV_val key, val;
216 RCC(rc, finish, iwkv_cursor_get(cur, &key, &val));
217 fprintf(stdout, "%.*s: %u\n",
218 (int) key.size, (char *) key.data,
219 *(uint8_t *) val.data);
220 iwkv_kv_dispose(&key, &val);
221 }
222 rc = 0;
223 iwkv_cursor_close(&cur);
224
225 fprintf(stdout, "\n>>>> Traverse in ascending order\n");
226 RCC(rc, finish, iwkv_cursor_open(db, &cur, IWKV_CURSOR_AFTER_LAST, 0));
227 while ((rc = iwkv_cursor_to(cur, IWKV_CURSOR_PREV)) == 0) {
228 IWKV_val key, val;
229 RCC(rc, finish, iwkv_cursor_get(cur, &key, &val));
230 fprintf(stdout, "%.*s: %u\n",
231 (int) key.size, (char *) key.data,
232 *(uint8_t *) val.data);
233 iwkv_kv_dispose(&key, &val);
234 }
235 rc = 0;
236 iwkv_cursor_close(&cur);
237
238 // Select all keys greater or equal than: Manchester United
239 {
240 fprintf(stdout, "\n>>>> Records GE: %s\n", _points[9].club);
241 IWKV_val key = { .data = (void *) _points[9].club, .size = strlen(_points[9].club) }, val;
242 RCC(rc, finish, iwkv_cursor_open(db, &cur, IWKV_CURSOR_GE, &key));
243 do {
244 RCC(rc, finish, iwkv_cursor_get(cur, &key, &val));
245 fprintf(stdout, "%.*s: %u\n",
246 (int) key.size, (char *) key.data,
247 *(uint8_t *) val.data);
248 iwkv_kv_dispose(&key, &val);
249 } while ((rc = iwkv_cursor_to(cur, IWKV_CURSOR_NEXT)) == 0);
250 rc = 0;
251 }
252 iwkv_cursor_close(&cur);
253
254finish:
255 if (cur) {
256 iwkv_cursor_close(&cur);
257 }
258 iwkv_close(&iwkv);
259 return rc;
260}
261
262int main() {
263 iwrc rc = run();
264 if (rc) {
265 iwlog_ecode_error3(rc);
266 return 1;
267 }
268 return 0;
269}
270```
271
272Output:
273```
274>>>> Traverse in descending order
275Wolverhampton Wanderers: 43
276West Ham United: 27
277Watford: 27
278Tottenham Hotspur: 41
279Southampton: 34
280Sheffield United: 43
281Norwich City: 21
282Newcastle United: 35
283Manchester United: 45
284Manchester City: 57
285Liverpool: 82
286Leicester City: 53
287Everton: 37
288Crystal Palace: 39
289Chelsea: 48
290Burnley: 39
291Brighton & Hove Albion: 29
292Aston Villa: 25
293Arsenal: 40
294AFC Bournemouth: 27
295
296>>>> Traverse in ascending order
297AFC Bournemouth: 27
298Arsenal: 40
299Aston Villa: 25
300Brighton & Hove Albion: 29
301Burnley: 39
302Chelsea: 48
303Crystal Palace: 39
304Everton: 37
305Leicester City: 53
306Liverpool: 82
307Manchester City: 57
308Manchester United: 45
309Newcastle United: 35
310Norwich City: 21
311Sheffield United: 43
312Southampton: 34
313Tottenham Hotspur: 41
314Watford: 27
315West Ham United: 27
316Wolverhampton Wanderers: 43
317
318>>>> Records GE: Manchester United
319Manchester United: 45
320Manchester City: 57
321Liverpool: 82
322Leicester City: 53
323Everton: 37
324Crystal Palace: 39
325Chelsea: 48
326Burnley: 39
327Brighton & Hove Albion: 29
328Aston Villa: 25
329Arsenal: 40
330AFC Bournemouth: 27
331```
332
333## IWSTART
334
335IWSTART is an automatic CMake initial project generator for C projects based on iowow / [iwnet](https://github.com/Softmotions/iwnet) / [ejdb2](https://github.com/Softmotions/ejdb) libs.
336
337https://github.com/Softmotions/iwstart
338
339
){kind=link}
){kind=link}
){kind=link}