1# JSONStream 2 3streaming JSON.parse and stringify 4 5 6 7## install 8```npm install JSONStream``` 9 10## example 11 12``` js 13 14var request = require('request') 15 , JSONStream = require('JSONStream') 16 , es = require('event-stream') 17 18request({url: 'http://isaacs.couchone.com/registry/_all_docs'}) 19 .pipe(JSONStream.parse('rows.*')) 20 .pipe(es.mapSync(function (data) { 21 console.error(data) 22 return data 23 })) 24``` 25 26## JSONStream.parse(path) 27 28parse stream of values that match a path 29 30``` js 31 JSONStream.parse('rows.*.doc') 32``` 33 34The `..` operator is the recursive descent operator from [JSONPath](http://goessner.net/articles/JsonPath/), which will match a child at any depth (see examples below). 35 36If your keys have keys that include `.` or `*` etc, use an array instead. 37`['row', true, /^doc/]`. 38 39If you use an array, `RegExp`s, booleans, and/or functions. The `..` operator is also available in array representation, using `{recurse: true}`. 40any object that matches the path will be emitted as 'data' (and `pipe`d down stream) 41 42If `path` is empty or null, no 'data' events are emitted. 43 44If you want to have keys emitted, you can prefix your `*` operator with `$`: `obj.$*` - in this case the data passed to the stream is an object with a `key` holding the key and a `value` property holding the data. 45 46### Examples 47 48query a couchdb view: 49 50``` bash 51curl -sS localhost:5984/tests/_all_docs&include_docs=true 52``` 53you will get something like this: 54 55``` js 56{"total_rows":129,"offset":0,"rows":[ 57 { "id":"change1_0.6995461115147918" 58 , "key":"change1_0.6995461115147918" 59 , "value":{"rev":"1-e240bae28c7bb3667f02760f6398d508"} 60 , "doc":{ 61 "_id": "change1_0.6995461115147918" 62 , "_rev": "1-e240bae28c7bb3667f02760f6398d508","hello":1} 63 }, 64 { "id":"change2_0.6995461115147918" 65 , "key":"change2_0.6995461115147918" 66 , "value":{"rev":"1-13677d36b98c0c075145bb8975105153"} 67 , "doc":{ 68 "_id":"change2_0.6995461115147918" 69 , "_rev":"1-13677d36b98c0c075145bb8975105153" 70 , "hello":2 71 } 72 }, 73]} 74 75``` 76 77we are probably most interested in the `rows.*.doc` 78 79create a `Stream` that parses the documents from the feed like this: 80 81``` js 82var stream = JSONStream.parse(['rows', true, 'doc']) //rows, ANYTHING, doc 83 84stream.on('data', function(data) { 85 console.log('received:', data); 86}); 87//emits anything from _before_ the first match 88stream.on('header', function (data) { 89 console.log('header:', data) // => {"total_rows":129,"offset":0} 90}) 91 92``` 93awesome! 94 95In case you wanted the contents the doc emitted: 96 97``` js 98var stream = JSONStream.parse(['rows', true, 'doc', {emitKey: true}]) //rows, ANYTHING, doc, items in docs with keys 99 100stream.on('data', function(data) { 101 console.log('key:', data.key); 102 console.log('value:', data.value); 103}); 104 105``` 106 107You can also emit the path: 108 109``` js 110var stream = JSONStream.parse(['rows', true, 'doc', {emitPath: true}]) //rows, ANYTHING, doc, items in docs with keys 111 112stream.on('data', function(data) { 113 console.log('path:', data.path); 114 console.log('value:', data.value); 115}); 116 117``` 118 119### recursive patterns (..) 120 121`JSONStream.parse('docs..value')` 122(or `JSONStream.parse(['docs', {recurse: true}, 'value'])` using an array) 123will emit every `value` object that is a child, grand-child, etc. of the 124`docs` object. In this example, it will match exactly 5 times at various depth 125levels, emitting 0, 1, 2, 3 and 4 as results. 126 127```js 128{ 129 "total": 5, 130 "docs": [ 131 { 132 "key": { 133 "value": 0, 134 "some": "property" 135 } 136 }, 137 {"value": 1}, 138 {"value": 2}, 139 {"blbl": [{}, {"a":0, "b":1, "value":3}, 10]}, 140 {"value": 4} 141 ] 142} 143``` 144 145## JSONStream.parse(pattern, map) 146 147provide a function that can be used to map or filter 148the json output. `map` is passed the value at that node of the pattern, 149if `map` return non-nullish (anything but `null` or `undefined`) 150that value will be emitted in the stream. If it returns a nullish value, 151nothing will be emitted. 152 153`JSONStream` also emits `'header'` and `'footer'` events, 154the `'header'` event contains anything in the output that was before 155the first match, and the `'footer'`, is anything after the last match. 156 157## JSONStream.stringify(open, sep, close) 158 159Create a writable stream. 160 161you may pass in custom `open`, `close`, and `seperator` strings. 162But, by default, `JSONStream.stringify()` will create an array, 163(with default options `open='[\n', sep='\n,\n', close='\n]\n'`) 164 165If you call `JSONStream.stringify(false)` 166the elements will only be seperated by a newline. 167 168If you only write one item this will be valid JSON. 169 170If you write many items, 171you can use a `RegExp` to split it into valid chunks. 172 173## JSONStream.stringifyObject(open, sep, close) 174 175Very much like `JSONStream.stringify`, 176but creates a writable stream for objects instead of arrays. 177 178Accordingly, `open='{\n', sep='\n,\n', close='\n}\n'`. 179 180When you `.write()` to the stream you must supply an array with `[ key, data ]` 181as the first argument. 182 183## unix tool 184 185query npm to see all the modules that browserify has ever depended on. 186 187``` bash 188curl https://registry.npmjs.org/browserify | JSONStream 'versions.*.dependencies' 189``` 190 191## numbers 192 193numbers will be emitted as numbers. 194huge numbers that cannot be represented in memory as javascript numbers will be emitted as strings. 195cf https://github.com/creationix/jsonparse/commit/044b268f01c4b8f97fb936fc85d3bcfba179e5bb for details. 196 197## Acknowlegements 198 199this module depends on https://github.com/creationix/jsonparse 200by Tim Caswell 201and also thanks to Florent Jaby for teaching me about parsing with: 202https://github.com/Floby/node-json-streams 203 204## license 205 206Dual-licensed under the MIT License or the Apache License, version 2.0 207 208
){kind=link}