1# LWS System Message Distribution 2 3## Overview 4 5Independent pieces of a system may need to become aware of events and state 6changes in the other pieces quickly, along with the new state if it is small. 7These messages are local to inside a system, although they may be triggered by 8events outside of it. Examples include keypresses, or networking state changes. 9Individual OSes and frameworks typically have their own fragmented apis for 10message-passing, but the lws apis operate the same across any platforms 11including, eg, Windows and RTOS and allow crossplatform code to be written once. 12 13Message payloads are short, less than 384 bytes, below system limits for atomic 14pipe or UDS datagrams and consistent with heap usage on smaller systems, but 15large enough to carry JSON usefully. Messages are typically low duty cycle. 16 17 18 19Messages may be sent by any registered participant, they are allocated on heap 20in a linked-list, and delivered to all other registered participants for that 21message class no sooner than next time around the event loop. This retains the 22ability to handle multiple event queuing in one event loop trip while 23guaranteeing message handling is nonrecursive and so with modest stack usage. 24Messages are passed to all other registered participants before being destroyed. 25 26Messages are delivered to all particpants on the same lws_context by default. 27 28 29 30`lws_smd` apis allow publication and subscription of message objects between 31participants that are in a single process and are informed by callback from lws 32service thread context. 33 34SMD messages can also broadcast between particpants in different lws_contexts in 35different processes, using existing Secure Streams proxying. In this way 36different application processes can intercommunicate and all observe any system 37smd messages they are interested in. 38 39 40 41Registering as a participant and sending messages are threadsafe APIs. 42 43## Message Class 44 45Message class is a bitfield messages use to indicate their general type, eg, 46network status, or UI event like a keypress. Participants set a bitmask to 47filter what kind of messages they care about, classes that are 0 in the peer's 48filter are never delivered to the peer. A message usually indicates it is a 49single class, but it's possible to set multiple class bits and match on any. If 50so, care must be taken the payload can be parsed by readers expecting any of the 51indicated classes, eg, by using JSON. 52 53`lws_smd` tracks a global union mask for all participants' class mask. Requests 54to allocate a message of a class that no participant listens for are rejected, 55not at distribution-time but at message allocation-time, so no heap or cpu is 56wasted on things that are not currently interesting; but such messages start to 57appear as soon as a participant appears that wants them. The message generation 58action should be bypassed without error in the case lws_smd_msg_alloc() 59returns NULL. 60 61Various well-known high level classes are defined but also a bit index 62`LWSSMDCL_USER_BASE_BITNUM`, which can be used by user code to define up to 8 63private classes, with class bit values `(1 << LWSSMDCL_USER_BASE_BITNUM)` thru 64`(1 << (LWSSMDCL_USER_BASE_BITNUM + 7))` 65 66## Messaging guarantees 67 68Sent messages are delivered to all registered participants whose class mask 69indicates they want it, including the sender. The send apis are threadsafe. 70 71Locally-delivered message delivery callbacks occur from lws event loop thread 72context 0 (the only one in the default case `LWS_MAX_SMP` = 1). Clients in 73different processes receive callbacks from the thread context of their UDS 74networking thread. 75 76The message payload may be destroyed immediately when you return from the 77callback, you can't store references to it or expect it to be there later. 78 79Messages are timestamped with a systemwide monotonic timestamp. When 80participants are on the lws event loop, messages are delivered in-order. When 81participants are on different threads, delivery order depends on platform lock 82acquisition. External process participants are connected by the Unix Domain 83Socket capability of Secure Streams, and may be delivered out-of-order; 84receivers that care must consult the message creation timestamps. 85 86## Message Refcounting 87 88To avoid keeping a list of the length of the number of participants for each 89message, a refcount is used in the message, computed at the time the message 90arrived considering the number of active participants that indicated a desire to 91receive messages of that class. 92 93Since peers may detach / close their link asynchronously, the logical peer 94objects at the distributor defer destroying themselves until there is no more 95possibility of messages arriving timestamped with the period they were active. 96A grace period (default 2s) is used to ensure departing peers correctly account 97for message refcounts before being destroyed. 98 99## Message creation 100 101Messages may contain arbitrary text or binary data depending on the class. JSON 102is recommended since lws_smd messages are small and low duty cycle but have 103open-ended content: JSON is maintainable, extensible, debuggable and self- 104documenting and avoids, eg, fragile dependencies on header versions shared 105between teams. To simplify issuing JSON, a threadsafe api to create and send 106messages in one step using format strings is provided: 107 108``` 109int 110lws_smd_msg_printf(struct lws_context *ctx, lws_smd_class_t _class, 111 const char *format, ...); 112``` 113 114## Secure Streams `lws_smd` streamtype 115 116When built with LWS_WITH_SECURE_STREAMS, lws_smd exposes a built-in streamtype 117`_lws_smd` which user Secure Streams may use to interoperate with lws_smd using 118SS payload semantics. 119 120When using `_lws_smd`, the SS info struct member `manual_initial_tx_credit` 121provided by the user when creating the Secure Stream is overloaded to be used as 122the RX class mask for the SMD connection associated with the Secure Stream. 123 124Both RX and TX payloads have a 16-byte binary header before the actual payload. 125For TX, although the header is 16-bytes, only the first 64-bit class bitfield 126needs setting, the timestamp is fetched and added by lws. 127 128 - MSB-first 64-bit class bitfield (currently only 32 least-sig in use) 129 - MSB-First Order 64-bit us-resolution timestamp 130 131A helper `lws_smd_ss_msg_printf()` is provided to format and create and smd 132message from the SS tx() callback in one step, using the same api layout as 133for direct messages via `lws_smd_msg_printf()` 134 135``` 136int 137lws_smd_ss_msg_printf(const char *tag, uint8_t *buf, size_t *len, 138 lws_smd_class_t _class, const char *format, ...); 139``` 140 141## Well-known message schema 142 143Class|Schema 144---|--- 145LWSSMDCL_INTERACTION|lws_button events 146LWSSMDCL_NETWORK|captive portal detection requests and results 147LWSSMDCL_SYSTEM_STATE|lws_system state progression 148 149### User interaction Button events 150 151Class: `LWSSMDCL_INTERACTION` 152 153Produced by lws_button when a user interacts with a defined button. 154 155Click-related events are produced alongside up and down related events, the 156participant can choose which to attend to according to the meaning of the 157interaction. 158 159Both kinds of event go through sophisticated filtering before being issued, see 160`./lib/drivers/button/README.md` for details. 161 162#### SMD Button interaction event 163 164Schema: 165``` 166{ 167 "type": "button", 168 "src": "<controller-name>/<button-name>", 169 "event": "<event-name>" 170} 171``` 172 173For example, `{"type":"button","src":"bc/user","event":"doubleclick"}` 174 175Event name|Meaning 176---|--- 177down|The button passes a filter for being down, useful for duration-based response 178up|The button has come up, useful for duration-based response 179click|The button activity resulted in a classification as a single-click 180longclick|The button activity resulted in a classification as a long-click 181doubleclick|The button activity resulted in a classification as a double-click 182 183### Routing Table Change 184 185Class: `LWSSMDCL_NETWORK` 186 187If able to subscribe to OS routing table changes (eg, by rtnetlink on Linux 188which is supported), lws announces there have been changes using SMD. 189 190If Captive Portal Detect is enabled, and routing tables changes can be seen, 191then a new CPD is requested automatically and the results will be seen over SMD 192when that completes. 193 194Schema: 195 196``` 197 { 198 "rt": "add|del", "add" if being added 199 } 200``` 201 202When the context / pts are created, if linux then lws attempts to get the 203routing table sent, which requires root. This is done before the permissions 204are dropped after protocols init. 205 206Lws maintains a cache of the routing table in each pt. Upon changes, existing 207connections are reassessed to see if their peer can still be routed to, if not 208the connection is closed. 209 210If a gateway route changes, `{"trigger":"cpdcheck","src":"gw-change"}` is 211issued on SMD as well. 212 213### Captive Portal Detection 214 215Class: `LWSSMDCL_NETWORK` 216 217Actively detects if the network can reach the internet or if it is 218intercepted by a captive portal. The detection steps are programmable 219via the Secure Streams Policy for a streamtype `captive_portal_detect`, eg 220 221``` 222 "captive_portal_detect": { 223 "endpoint": "connectivitycheck.android.com", 224 "http_url": "generate_204", 225 "port": 80, 226 "protocol": "h1", 227 "http_method": "GET", 228 "opportunistic": true, 229 "http_expect": 204, 230 "http_fail_redirect": true 231 } 232``` 233 234#### SMD Report Result 235 236Schema: `{"type": "cpd", "result":"<result>"}` 237 238result|meaning 239---|--- 240OK|Internet is reachable 241Captive|Internet is behind a captive portal 242No internet|There is no connectivity 243 244#### SMD Request re-detection 245 246Schema: `{"trigger": "cpdcheck"}` 247 248### lws_system state progression 249 250Class: `LWSSMDCL_SYSTEM_STATE` 251 252Lws system state changes are forwarded to lws_smd messages so participants not 253on the lws event loop directly can be aware of progress. Code registering a 254lws_system notifier callback, on the main lws loop, can synchronously veto state 255changes and hook proposed state changes, lws_smd events are asynchronous 256notifications of state changes after they were decided only... however they are 257available over the whole system. 258 259It's not possible to make validated TLS connections until the system has 260acquired the date as well as acquired an IP on a non-captive portal connection, 261for that reason user code will usually be dependent on the system reaching 262"OPERATIONAL" state if lws is responsible for managing the boot process. 263 264#### System state event 265 266Schema: `{"state":"<state>"}"` 267 268State|Meaning 269---|--- 270CONTEXT_CREATED|We're creating the lws_context 271INITIALIZED|Initial vhosts and protocols initialized 272IFACE_COLDPLUG|Network interfaces discovered 273DHCP|DHCP acquired 274CPD_PRE_TIME|Captive portal detect hook before we have system time 275TIME_VALID|Ntpclient has run 276CPD_POST_TIME|Captive portal detect hook after system time (tls-based check) 277POLICY_VALID|The system policy has been acquired and parsed 278REGISTERED|This device is registered with an authority 279AUTH1|We acquired auth1 from the authority using our registration info 280AUTH2|We acquired auth2 from the authority using our registration info 281OPERATIONAL|We are active and able to make authenticated tls connections 282POLICY_INVALID|The policy is being changed 283
