• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1.. _module-pw_metric:
2
3=========
4pw_metric
5=========
6.. pigweed-module::
7   :name: pw_metric
8
9.. attention::
10   This module is **not yet production ready**; ask us if you are interested in
11   using it out or have ideas about how to improve it.
12
13--------
14Overview
15--------
16Pigweed's metric module is a **lightweight manual instrumentation system** for
17tracking system health metrics like counts or set values. For example,
18``pw_metric`` could help with tracking the number of I2C bus writes, or the
19number of times a buffer was filled before it could drain in time, or safely
20incrementing counters from ISRs.
21
22Key features of ``pw_metric``:
23
24- **Tokenized names** - Names are tokenized using the ``pw_tokenizer`` enabling
25  long metric names that don't bloat your binary.
26
27- **Tree structure** - Metrics can form a tree, enabling grouping of related
28  metrics for clearer organization.
29
30- **Per object collection** - Metrics and groups can live on object instances
31  and be flexibly combined with metrics from other instances.
32
33- **Global registration** - For legacy code bases or just because it's easier,
34  ``pw_metric`` supports automatic aggregation of metrics. This is optional but
35  convenient in many cases.
36
37- **Simple design** - There are only two core data structures: ``Metric`` and
38  ``Group``, which are both simple to understand and use. The only type of
39  metric supported is ``uint32_t`` and ``float``. This module does not support
40  complicated aggregations like running average or min/max.
41
42Example: Instrumenting a single object
43--------------------------------------
44The below example illustrates what instrumenting a class with a metric group
45and metrics might look like. In this case, the object's
46``MySubsystem::metrics()`` member is not globally registered; the user is on
47their own for combining this subsystem's metrics with others.
48
49.. code-block:: cpp
50
51   #include "pw_metric/metric.h"
52
53   class MySubsystem {
54    public:
55     void DoSomething() {
56       attempts_.Increment();
57       if (ActionSucceeds()) {
58         successes_.Increment();
59       }
60     }
61     Group& metrics() { return metrics_; }
62
63    private:
64     PW_METRIC_GROUP(metrics_, "my_subsystem");
65     PW_METRIC(metrics_, attempts_, "attempts", 0u);
66     PW_METRIC(metrics_, successes_, "successes", 0u);
67   };
68
69The metrics subsystem has no canonical output format at this time, but a JSON
70dump might look something like this:
71
72.. code-block:: json
73
74   {
75     "my_subsystem" : {
76       "successes" : 1000,
77       "attempts" : 1200,
78     }
79   }
80
81In this case, every instance of ``MySubsystem`` will have unique counters.
82
83Example: Instrumenting a legacy codebase
84----------------------------------------
85A common situation in embedded development is **debugging legacy code** or code
86which is hard to change; where it is perhaps impossible to plumb metrics
87objects around with dependency injection. The alternative to plumbing metrics
88is to register the metrics through a global mechanism. ``pw_metric`` supports
89this use case. For example:
90
91**Before instrumenting:**
92
93.. code-block:: cpp
94
95   // This code was passed down from generations of developers before; no one
96   // knows what it does or how it works. But it needs to be fixed!
97   void OldCodeThatDoesntWorkButWeDontKnowWhy() {
98     if (some_variable) {
99       DoSomething();
100     } else {
101       DoSomethingElse();
102     }
103   }
104
105**After instrumenting:**
106
107.. code-block:: cpp
108
109   #include "pw_metric/global.h"
110   #include "pw_metric/metric.h"
111
112   PW_METRIC_GLOBAL(legacy_do_something, "legacy_do_something");
113   PW_METRIC_GLOBAL(legacy_do_something_else, "legacy_do_something_else");
114
115   // This code was passed down from generations of developers before; no one
116   // knows what it does or how it works. But it needs to be fixed!
117   void OldCodeThatDoesntWorkButWeDontKnowWhy() {
118     if (some_variable) {
119       legacy_do_something.Increment();
120       DoSomething();
121     } else {
122       legacy_do_something_else.Increment();
123       DoSomethingElse();
124     }
125   }
126
127In this case, the developer merely had to add the metrics header, define some
128metrics, and then start incrementing them. These metrics will be available
129globally through the ``pw::metric::global_metrics`` object defined in
130``pw_metric/global.h``.
131
132Why not just use simple counter variables?
133------------------------------------------
134One might wonder what the point of leveraging a metric library is when it is
135trivial to make some global variables and print them out. There are a few
136reasons:
137
138- **Metrics offload** - To make it easy to get metrics off-device by sharing
139  the infrastructure for offloading.
140
141- **Consistent format** - To get the metrics in a consistent format (e.g.
142  protobuf or JSON) for analysis
143
144- **Uncoordinated collection** - To provide a simple and reliable way for
145  developers on a team to all collect metrics for their subsystems, without
146  having to coordinate to offload. This could extend to code in libraries
147  written by other teams.
148
149- **Pre-boot or interrupt visibility** - Some of the most challenging bugs come
150  from early system boot when not all system facilities are up (e.g. logging or
151  UART). In those cases, metrics provide a low-overhead approach to understand
152  what is happening. During early boot, metrics can be incremented, then after
153  boot dumping the metrics provides insights into what happened. While basic
154  counter variables can work in these contexts too, one still has to deal with
155  the offloading problem; which the library handles.
156
157---------------------
158Metrics API reference
159---------------------
160
161The metrics API consists of just a few components:
162
163- The core data structures ``pw::metric::Metric`` and ``pw::metric::Group``
164- The macros for scoped metrics and groups ``PW_METRIC`` and
165  ``PW_METRIC_GROUP``
166- The macros for globally registered metrics and groups
167  ``PW_METRIC_GLOBAL`` and ``PW_METRIC_GROUP_GLOBAL``
168- The global groups and metrics list: ``pw::metric::global_groups`` and
169  ``pw::metric::global_metrics``.
170
171Metric
172------
173The ``pw::metric::Metric`` provides:
174
175- A 31-bit tokenized name
176- A 1-bit discriminator for int or float
177- A 32-bit payload (int or float)
178- A 32-bit next pointer (intrusive list)
179
180The metric object is 12 bytes on 32-bit platforms.
181
182All of the operations on the pw_metric are atomic, and require the system to
183have a ``std::atomic`` backend implementation defined.
184
185.. cpp:class:: pw::metric::Metric
186
187   .. cpp:function:: Increment(uint32_t amount = 0)
188
189      Increment the metric by the given amount. Results in undefined behaviour if
190      the metric is not of type int.
191
192   .. cpp:function:: Set(uint32_t value)
193
194      Set the metric to the given value. Results in undefined behaviour if the
195      metric is not of type int.
196
197   .. cpp:function:: Set(float value)
198
199      Set the metric to the given value. Results in undefined behaviour if the
200      metric is not of type float.
201
202.. _module-pw_metric-group:
203
204Group
205-----
206The ``pw::metric::Group`` object is simply:
207
208- A name for the group
209- A list of children groups
210- A list of leaf metrics groups
211- A 32-bit next pointer (intrusive list)
212
213The group object is 16 bytes on 32-bit platforms.
214
215.. cpp:class:: pw::metric::Group
216
217   .. cpp:function:: Dump(int indent_level = 0)
218
219      Recursively dump a metrics group to ``pw_log``. Produces output like:
220
221      .. code-block:: none
222
223         "$6doqFw==": {
224           "$05OCZw==": {
225             "$VpPfzg==": 1,
226             "$LGPMBQ==": 1.000000,
227             "$+iJvUg==": 5,
228           }
229           "$9hPNxw==": 65,
230           "$oK7HmA==": 13,
231           "$FCM4qQ==": 0,
232         }
233
234      Note the metric names are tokenized with base64. Decoding requires using
235      the Pigweed detokenizer. With a detokenizing-enabled logger, you could get
236      something like:
237
238      .. code-block:: none
239
240         "i2c_1": {
241           "gyro": {
242             "num_sampleses": 1,
243             "init_time_us": 1.000000,
244             "initialized": 5,
245           }
246           "bus_errors": 65,
247           "transactions": 13,
248           "bytes_sent": 0,
249         }
250
251Macros
252------
253The **macros are the primary mechanism for creating metrics**, and should be
254used instead of directly constructing metrics or groups. The macros handle
255tokenizing the metric and group names.
256
257.. cpp:function:: PW_METRIC(identifier, name, value)
258.. cpp:function:: PW_METRIC(group, identifier, name, value)
259.. cpp:function:: PW_METRIC_STATIC(identifier, name, value)
260.. cpp:function:: PW_METRIC_STATIC(group, identifier, name, value)
261
262   Declare a metric, optionally adding it to a group.
263
264   - **identifier** - An identifier name for the created variable or member.
265     For example: ``i2c_transactions`` might be used as a local or global
266     metric; inside a class, could be named according to members
267     (``i2c_transactions_`` for Google's C++ style).
268   - **name** - The string name for the metric. This will be tokenized. There
269     are no restrictions on the contents of the name; however, consider
270     restricting these to be valid C++ identifiers to ease integration with
271     other systems.
272   - **value** - The initial value for the metric. Must be either a floating
273     point value (e.g. ``3.2f``) or unsigned int (e.g. ``21u``).
274   - **group** - A ``pw::metric::Group`` instance. If provided, the metric is
275     added to the given group.
276
277   The macro declares a variable or member named "name" with type
278   ``pw::metric::Metric``, and works in three contexts: global, local, and
279   member.
280
281   If the ``_STATIC`` variant is used, the macro declares a variable with static
282   storage. These can be used in function scopes, but not in classes.
283
284   1. At global scope:
285
286      .. code-block:: cpp
287
288         PW_METRIC(foo, "foo", 15.5f);
289
290         void MyFunc() {
291           foo.Increment();
292         }
293
294   2. At local function or member function scope:
295
296      .. code-block:: cpp
297
298         void MyFunc() {
299           PW_METRIC(foo, "foo", 15.5f);
300           foo.Increment();
301           // foo goes out of scope here; be careful!
302         }
303
304   3. At member level inside a class or struct:
305
306      .. code-block:: cpp
307
308         struct MyStructy {
309           void DoSomething() {
310             somethings.Increment();
311           }
312           // Every instance of MyStructy will have a separate somethings counter.
313           PW_METRIC(somethings, "somethings", 0u);
314         }
315
316   You can also put a metric into a group with the macro. Metrics can belong to
317   strictly one group, otherwise an assertion will fail. Example:
318
319   .. code-block:: cpp
320
321      PW_METRIC_GROUP(my_group, "my_group");
322      PW_METRIC(my_group, foo, "foo", 0.2f);
323      PW_METRIC(my_group, bar, "bar", 44000u);
324      PW_METRIC(my_group, zap, "zap", 3.14f);
325
326   .. tip::
327      If you want a globally registered metric, see ``pw_metric/global.h``; in
328      that context, metrics are globally registered without the need to
329      centrally register in a single place.
330
331.. cpp:function:: PW_METRIC_GROUP(identifier, name)
332.. cpp:function:: PW_METRIC_GROUP(parent_group, identifier, name)
333.. cpp:function:: PW_METRIC_GROUP_STATIC(identifier, name)
334.. cpp:function:: PW_METRIC_GROUP_STATIC(parent_group, identifier, name)
335
336   Declares a ``pw::metric::Group`` with name name; the name is tokenized.
337   Works similar to ``PW_METRIC`` and can be used in the same contexts (global,
338   local, and member). Optionally, the group can be added to a parent group.
339
340   If the ``_STATIC`` variant is used, the macro declares a variable with static
341   storage. These can be used in function scopes, but not in classes.
342
343   Example:
344
345   .. code-block:: cpp
346
347      PW_METRIC_GROUP(my_group, "my_group");
348      PW_METRIC(my_group, foo, "foo", 0.2f);
349      PW_METRIC(my_group, bar, "bar", 44000u);
350      PW_METRIC(my_group, zap, "zap", 3.14f);
351
352.. cpp:function:: PW_METRIC_GLOBAL(identifier, name, value)
353
354   Declare a ``pw::metric::Metric`` with name name, and register it in the
355   global metrics list ``pw::metric::global_metrics``.
356
357   Example:
358
359   .. code-block:: cpp
360
361      #include "pw_metric/metric.h"
362      #include "pw_metric/global.h"
363
364      // No need to coordinate collection of foo and bar; they're autoregistered.
365      PW_METRIC_GLOBAL(foo, "foo", 0.2f);
366      PW_METRIC_GLOBAL(bar, "bar", 44000u);
367
368   Note that metrics defined with ``PW_METRIC_GLOBAL`` should never be added to
369   groups defined with ``PW_METRIC_GROUP_GLOBAL``. Each metric can only belong
370   to one group, and metrics defined with ``PW_METRIC_GLOBAL`` are
371   pre-registered with the global metrics list.
372
373   .. attention::
374      Do not create ``PW_METRIC_GLOBAL`` instances anywhere other than global
375      scope. Putting these on an instance (member context) would lead to dangling
376      pointers and misery. Metrics are never deleted or unregistered!
377
378.. cpp:function:: PW_METRIC_GROUP_GLOBAL(identifier, name)
379
380   Declare a ``pw::metric::Group`` with name name, and register it in the
381   global metric groups list ``pw::metric::global_groups``.
382
383   Note that metrics created with ``PW_METRIC_GLOBAL`` should never be added to
384   groups! Instead, just create a freestanding metric and register it into the
385   global group (like in the example below).
386
387   Example:
388
389   .. code-block:: cpp
390
391      #include "pw_metric/metric.h"
392      #include "pw_metric/global.h"
393
394      // No need to coordinate collection of this group; it's globally registered.
395      PW_METRIC_GROUP_GLOBAL(legacy_system, "legacy_system");
396      PW_METRIC(legacy_system, foo, "foo", 0.2f);
397      PW_METRIC(legacy_system, bar, "bar", 44000u);
398
399   .. attention::
400      Do not create ``PW_METRIC_GROUP_GLOBAL`` instances anywhere other than
401      global scope. Putting these on an instance (member context) would lead to
402      dangling pointers and misery. Metrics are never deleted or unregistered!
403
404.. cpp:function:: PW_METRIC_TOKEN(name)
405
406   Declare a ``pw::metric::Token`` (``pw::tokenizer::Token``) for a metric with
407   name ``name``. This token matches the ``.name()`` of a metric with the same
408   name.
409
410   This is a wrapper around ``PW_TOKENIZE_STRING_MASK`` and carries the same
411   semantics.
412
413   This is unlikely to be used by most pw_metric consumers.
414
415----------------------
416Usage & Best Practices
417----------------------
418This library makes several tradeoffs to enable low memory use per-metric, and
419one of those tradeoffs results in requiring care in constructing the metric
420trees.
421
422Use the Init() pattern for static objects with metrics
423------------------------------------------------------
424A common pattern in embedded systems is to allocate many objects globally, and
425reduce reliance on dynamic allocation (or eschew malloc entirely). This leads
426to a pattern where rich/large objects are statically constructed at global
427scope, then interacted with via tasks or threads. For example, consider a
428hypothetical global ``Uart`` object:
429
430.. code-block:: cpp
431
432   class Uart {
433    public:
434     Uart(span<std::byte> rx_buffer, span<std::byte> tx_buffer)
435       : rx_buffer_(rx_buffer), tx_buffer_(tx_buffer) {}
436
437     // Send/receive here...
438
439    private:
440     pw::span<std::byte> rx_buffer;
441     pw::span<std::byte> tx_buffer;
442   };
443
444   std::array<std::byte, 512> uart_rx_buffer;
445   std::array<std::byte, 512> uart_tx_buffer;
446   Uart uart1(uart_rx_buffer, uart_tx_buffer);
447
448Through the course of building a product, the team may want to add metrics to
449the UART to for example gain insight into which operations are triggering lots
450of data transfer. When adding metrics to the above imaginary UART object, one
451might consider the following approach:
452
453.. code-block:: cpp
454
455   class Uart {
456    public:
457     Uart(span<std::byte> rx_buffer,
458          span<std::byte> tx_buffer,
459          Group& parent_metrics)
460       : rx_buffer_(rx_buffer),
461         tx_buffer_(tx_buffer) {
462         // PROBLEM! parent_metrics may not be constructed if it's a reference
463         // to a static global.
464         parent_metrics.Add(tx_bytes_);
465         parent_metrics.Add(rx_bytes_);
466      }
467
468     // Send/receive here which increment tx/rx_bytes.
469
470    private:
471     pw::span<std::byte> rx_buffer;
472     pw::span<std::byte> tx_buffer;
473
474     PW_METRIC(tx_bytes_, "tx_bytes", 0);
475     PW_METRIC(rx_bytes_, "rx_bytes", 0);
476   };
477
478   PW_METRIC_GROUP(global_metrics, "/");
479   PW_METRIC_GROUP(global_metrics, uart1_metrics, "uart1");
480
481   std::array<std::byte, 512> uart_rx_buffer;
482   std::array<std::byte, 512> uart_tx_buffer;
483   Uart uart1(uart_rx_buffer,
484              uart_tx_buffer,
485              uart1_metrics);
486
487However, this **is incorrect**, since the ``parent_metrics`` (pointing to
488``uart1_metrics`` in this case) may not be constructed at the point of
489``uart1`` getting constructed. Thankfully in the case of ``pw_metric`` this
490will result in an assertion failure (or it will work correctly if the
491constructors are called in a favorable order), so the problem will not go
492unnoticed.  Instead, consider using the ``Init()`` pattern for static objects,
493where references to dependencies may only be stored during construction, but no
494methods on the dependencies are called.
495
496Instead, the ``Init()`` approach separates global object construction into two
497phases: The constructor where references are stored, and a ``Init()`` function
498which is called after all static constructors have run. This approach works
499correctly, even when the objects are allocated globally:
500
501.. code-block:: cpp
502
503   class Uart {
504    public:
505     // Note that metrics is not passed in here at all.
506     Uart(span<std::byte> rx_buffer,
507          span<std::byte> tx_buffer)
508       : rx_buffer_(rx_buffer),
509         tx_buffer_(tx_buffer) {}
510
511      // Precondition: parent_metrics is already constructed.
512      void Init(Group& parent_metrics) {
513         parent_metrics.Add(tx_bytes_);
514         parent_metrics.Add(rx_bytes_);
515      }
516
517     // Send/receive here which increment tx/rx_bytes.
518
519    private:
520     pw::span<std::byte> rx_buffer;
521     pw::span<std::byte> tx_buffer;
522
523     PW_METRIC(tx_bytes_, "tx_bytes", 0);
524     PW_METRIC(rx_bytes_, "rx_bytes", 0);
525   };
526
527   PW_METRIC_GROUP(root_metrics, "/");
528   PW_METRIC_GROUP(root_metrics, uart1_metrics, "uart1");
529
530   std::array<std::byte, 512> uart_rx_buffer;
531   std::array<std::byte, 512> uart_tx_buffer;
532   Uart uart1(uart_rx_buffer,
533              uart_tx_buffer);
534
535   void main() {
536     // uart1_metrics is guaranteed to be initialized by this point, so it is
537     safe to pass it to Init().
538     uart1.Init(uart1_metrics);
539   }
540
541.. attention::
542   Be extra careful about **static global metric registration**. Consider using
543   the ``Init()`` pattern.
544
545Metric member order matters in objects
546--------------------------------------
547The order of declaring in-class groups and metrics matters if the metrics are
548within a group declared inside the class. For example, the following class will
549work fine:
550
551.. code-block:: cpp
552
553   #include "pw_metric/metric.h"
554
555   class PowerSubsystem {
556    public:
557      Group& metrics() { return metrics_; }
558      const Group& metrics() const { return metrics_; }
559
560    private:
561     PW_METRIC_GROUP(metrics_, "power");  // Note metrics_ declared first.
562     PW_METRIC(metrics_, foo, "foo", 0.2f);
563     PW_METRIC(metrics_, bar, "bar", 44000u);
564   };
565
566but the following one will not since the group is constructed after the metrics
567(and will result in a compile error):
568
569.. code-block:: cpp
570
571   #include "pw_metric/metric.h"
572
573   class PowerSubsystem {
574    public:
575      Group& metrics() { return metrics_; }
576      const Group& metrics() const { return metrics_; }
577
578    private:
579     PW_METRIC(metrics_, foo, "foo", 0.2f);
580     PW_METRIC(metrics_, bar, "bar", 44000u);
581     PW_METRIC_GROUP(metrics_, "power");  // Error: metrics_ must be first.
582   };
583
584.. attention::
585
586   Put **groups before metrics** when declaring metrics members inside classes.
587
588Thread safety
589-------------
590``pw_metric`` has **no built-in synchronization for manipulating the tree**
591structure. Users are expected to either rely on shared global mutex when
592constructing the metric tree, or do the metric construction in a single thread
593(e.g. a boot/init thread). The same applies for destruction, though we do not
594advise destructing metrics or groups.
595
596Individual metrics have atomic ``Increment()``, ``Set()``, and the value
597accessors ``as_float()`` and ``as_int()`` which don't require separate
598synchronization, and can be used from ISRs.
599
600.. attention::
601
602   **You must synchronize access to metrics**. ``pw_metrics`` does not
603   internally synchronize access during construction. Metric Set/Increment are
604   safe.
605
606Lifecycle
607---------
608Metric objects are not designed to be destructed, and are expected to live for
609the lifetime of the program or application. If you need dynamic
610creation/destruction of metrics, ``pw_metric`` does not attempt to cover that
611use case. Instead, ``pw_metric`` covers the case of products with two execution
612phases:
613
6141. A boot phase where the metric tree is created.
6152. A run phase where metrics are collected. The tree structure is fixed.
616
617Technically, it is possible to destruct metrics provided care is taken to
618remove the given metric (or group) from the list it's contained in. However,
619there are no helper functions for this, so be careful.
620
621Below is an example that **is incorrect**. Don't do what follows!
622
623.. code-block:: cpp
624
625   #include "pw_metric/metric.h"
626
627   void main() {
628     PW_METRIC_GROUP(root, "/");
629     {
630       // BAD! The metrics have a different lifetime than the group.
631       PW_METRIC(root, temperature, "temperature_f", 72.3f);
632       PW_METRIC(root, humidity, "humidity_relative_percent", 33.2f);
633     }
634     // OOPS! root now has a linked list that points to the destructed
635     // "humidity" object.
636   }
637
638.. attention::
639   **Don't destruct metrics**. Metrics are designed to be registered /
640   structured upfront, then manipulated during a device's active phase. They do
641   not support destruction.
642
643.. _module-pw_metric-exporting:
644
645-----------------
646Exporting metrics
647-----------------
648Collecting metrics on a device is not useful without a mechanism to export
649those metrics for analysis and debugging. ``pw_metric`` offers optional RPC
650service libraries (``:metric_service_nanopb`` based on nanopb, and
651``:metric_service_pwpb`` based on pw_protobuf) that enable exporting a
652user-supplied set of on-device metrics via RPC. This facility is intended to
653function from the early stages of device bringup through production in the
654field.
655
656The metrics are fetched by calling the ``MetricService.Get`` RPC method, which
657streams all registered metrics to the caller in batches (server streaming RPC).
658Batching the returned metrics avoids requiring a large buffer or large RPC MTU.
659
660The returned metric objects have flattened paths to the root. For example, the
661returned metrics (post detokenization and jsonified) might look something like:
662
663.. code-block:: json
664
665   {
666     "/i2c1/failed_txns": 17,
667     "/i2c1/total_txns": 2013,
668     "/i2c1/gyro/resets": 24,
669     "/i2c1/gyro/hangs": 1,
670     "/spi1/thermocouple/reads": 242,
671     "/spi1/thermocouple/temp_celsius": 34.52,
672   }
673
674Note that there is no nesting of the groups; the nesting is implied from the
675path.
676
677RPC service setup
678-----------------
679To expose a ``MetricService`` in your application, do the following:
680
6811. Define metrics around the system, and put them in a group or list of
682   metrics. Easy choices include for example the ``global_groups`` and
683   ``global_metrics`` variables; or creat your own.
6842. Create an instance of ``pw::metric::MetricService``.
6853. Register the service with your RPC server.
686
687For example:
688
689.. code-block:: cpp
690
691   #include "pw_rpc/server.h"
692   #include "pw_metric/metric.h"
693   #include "pw_metric/global.h"
694   #include "pw_metric/metric_service_nanopb.h"
695
696   // Note: You must customize the RPC server setup; see pw_rpc.
697   Channel channels[] = {
698    Channel::Create<1>(&uart_output),
699   };
700   Server server(channels);
701
702   // Metric service instance, pointing to the global metric objects.
703   // This could also point to custom per-product or application objects.
704   pw::metric::MetricService metric_service(
705       pw::metric::global_metrics,
706       pw::metric::global_groups);
707
708   void RegisterServices() {
709     server.RegisterService(metric_service);
710     // Register other services here.
711   }
712
713   void main() {
714     // ... system initialization ...
715
716     RegisterServices();
717
718     // ... start your applcation ...
719   }
720
721.. attention::
722   Take care when exporting metrics. Ensure **appropriate access control** is in
723   place. In some cases it may make sense to entirely disable metrics export for
724   production builds. Although reading metrics via RPC won't influence the
725   device, in some cases the metrics could expose sensitive information if
726   product owners are not careful.
727
728.. attention::
729   **MetricService::Get is a synchronous RPC method**
730
731   Calls to is ``MetricService::Get`` are blocking and will send all metrics
732   immediately, even though it is a server-streaming RPC. This will work fine if
733   the device doesn't have too many metrics, or doesn't have concurrent RPCs
734   like logging, but could be a problem in some cases.
735
736   We plan to offer an async version where the application is responsible for
737   pumping the metrics into the streaming response. This gives flow control to
738   the application.
739
740-----------
741Size report
742-----------
743The below size report shows the cost in code and memory for a few examples of
744metrics. This does not include the RPC service.
745
746.. TODO: b/388905812 - Re-enable the size report.
747.. .. include:: metric_size_report
748.. include:: ../size_report_notice
749
750.. attention::
751   At time of writing, **the above sizes show an unexpectedly large flash
752   impact**. We are investigating why GCC is inserting large global static
753   constructors per group, when all the logic should be reused across objects.
754
755-------------
756Metric Parser
757-------------
758The metric_parser Python Module requests the system metrics via RPC, then parses the
759response while detokenizing the group and metrics names, and returns the metrics
760in a dictionary organized by group and value.
761
762----------------
763Design tradeoffs
764----------------
765There are many possible approaches to metrics collection and aggregation. We've
766chosen some points on the tradeoff curve:
767
768- **Atomic-sized metrics** - Using simple metric objects with just uint32/float
769  enables atomic operations. While it might be nice to support larger types, it
770  is more useful to have safe metrics increment from interrupt subroutines.
771
772- **No aggregate metrics (yet)** - Aggregate metrics (e.g. average, max, min,
773  histograms) are not supported, and must be built on top of the simple base
774  metrics. By taking this route, we can considerably simplify the core metrics
775  system and have aggregation logic in separate modules. Those modules can then
776  feed into the metrics system - for example by creating multiple metrics for a
777  single underlying metric. For example: "foo", "foo_max", "foo_min" and so on.
778
779  The other problem with automatic aggregation is that what period the
780  aggregation happens over is often important, and it can be hard to design
781  this cleanly into the API. Instead, this responsibility is pushed to the user
782  who must take more care.
783
784  Note that we will add helpers for aggregated metrics.
785
786- **No virtual metrics** - An alternate approach to the concrete Metric class
787  in the current module is to have a virtual interface for metrics, and then
788  allow those metrics to have their own storage. This is attractive but can
789  lead to many vtables and excess memory use in simple one-metric use cases.
790
791- **Linked list registration** - Using linked lists for registration is a
792  tradeoff, accepting some memory overhead in exchange for flexibility. Other
793  alternatives include a global table of metrics, which has the disadvantage of
794  requiring centralizing the metrics -- an impossibility for middleware like
795  Pigweed.
796
797- **Synchronization** - The only synchronization guarantee provided by
798  pw_metric is that increment and set are atomic. Other than that, users are on
799  their own to synchonize metric collection and updating.
800
801- **No fast metric lookup** - The current design does not make it fast to
802  lookup a metric at runtime; instead, one must run a linear search of the tree
803  to find the matching metric. In most non-dynamic use cases, this is fine in
804  practice, and saves having a more involved hash table. Metric updates will be
805  through direct member or variable accesses.
806
807- **Relying on C++ static initialization** - In short, the convenience
808  outweighs the cost and risk. Without static initializers, it would be
809  impossible to automatically collect the metrics without post-processing the
810  C++ code to find the metrics; a huge and debatably worthwhile approach. We
811  have carefully analyzed the static initializer behaviour of Pigweed's
812  IntrusiveList and are confident it is correct.
813
814- **Both local & global support** - Potentially just one approach (the local or
815  global one) could be offered, making the module less complex. However, we
816  feel the additional complexity is worthwhile since there are legimitate use
817  cases for both e.g. ``PW_METRIC`` and ``PW_METRIC_GLOBAL``. We'd prefer to
818  have a well-tested upstream solution for these use cases rather than have
819  customers re-implement one of these.
820
821----------------
822Roadmap & Status
823----------------
824- **String metric names** - ``pw_metric`` stores metric names as tokens. On one
825  hand, this is great for production where having a compact binary is often a
826  requirement to fit the application in the given part. However, in early
827  development before flash is a constraint, string names are more convenient to
828  work with since there is no need for host-side detokenization. We plan to add
829  optional support for using supporting strings.
830
831- **Aggregate metrics** - We plan to add support for aggregate metrics on top
832  of the simple metric mechanism, either as another module or as additional
833  functionality inside this one. Likely examples include min/max,
834
835- **Selectively enable or disable metrics** - Currently the metrics are always
836  enabled once included. In practice this is not ideal since many times only a
837  few metrics are wanted in production, but having to strip all the metrics
838  code is error prone. Instead, we will add support for controlling what
839  metrics are enabled or disabled at compile time. This may rely on of C++20's
840  support for zero-sized members to fully remove the cost.
841
842- **Async RPC** - The current RPC service exports the metrics by streaming
843  them to the client in batches. However, the current solution streams all the
844  metrics to completion; this may block the RPC thread. In the future we will
845  have an async solution where the user is in control of flow priority.
846
847- **Timer integration** - We would like to add a stopwatch type mechanism to
848  time multiple in-flight events.
849
850- **C support** - In practice it's often useful or necessary to instrument
851  C-only code. While it will be impossible to support the global registration
852  system that the C++ version supports, we will figure out a solution to make
853  instrumenting C code relatively smooth.
854
855- **Global counter** - We may add a global metric counter to help detect cases
856  where post-initialization metrics manipulations are done.
857
858- **Proto structure** - It may be possible to directly map metrics to a custom
859  proto structure, where instead of a name or token field, a tag field is
860  provided. This could result in elegant export to an easily machine parsable
861  and compact representation on the host. We may investigate this in the
862  future.
863
864- **Safer data structures** - At a cost of 4B per metric and 4B per group, it
865  may be possible to make metric structure instantiation safe even in static
866  constructors, and also make it safe to remove metrics dynamically. We will
867  consider whether this tradeoff is the right one, since a 4B cost per metric
868  is substantial on projects with many metrics.
869