1.. _module-pw_bloat: 2 3-------- 4pw_bloat 5-------- 6The bloat module provides tools and helpers around using 7`Bloaty McBloatface <https://github.com/google/bloaty>`_ including generating 8size report cards for output binaries through Pigweed's GN build 9system. 10 11Bloat report cards allow tracking the memory usage of a system over time as code 12changes are made and provide a breakdown of which parts of the code have the 13largest size impact. 14 15``pw bloat`` CLI command 16======================== 17``pw_bloat`` includes a plugin for the Pigweed command line capable of running 18size reports on ELF binaries. 19 20.. note:: 21 22 The bloat CLI plugin is still experimental and only supports a small subset 23 of ``pw_bloat``'s capabilities. Notably, it currently only runs on binaries 24 which define memory region symbols; refer to the 25 :ref:`memoryregions documentation <module-pw_bloat-memoryregions>` 26 for details. 27 28Basic usage 29^^^^^^^^^^^ 30 31**Running a size report on a single executable** 32 33.. code-block:: sh 34 35 $ pw bloat out/docs/obj/pw_result/size_report/bin/ladder_and_then.elf 36 37 ▒█████▄ █▓ ▄███▒ ▒█ ▒█ ░▓████▒ ░▓████▒ ▒▓████▄ 38 ▒█░ █░ ░█▒ ██▒ ▀█▒ ▒█░ █ ▒█ ▒█ ▀ ▒█ ▀ ▒█ ▀█▌ 39 ▒█▄▄▄█░ ░█▒ █▓░ ▄▄░ ▒█░ █ ▒█ ▒███ ▒███ ░█ █▌ 40 ▒█▀ ░█░ ▓█ █▓ ░█░ █ ▒█ ▒█ ▄ ▒█ ▄ ░█ ▄█▌ 41 ▒█ ░█░ ░▓███▀ ▒█▓▀▓█░ ░▓████▒ ░▓████▒ ▒▓████▀ 42 43 +----------------------+---------+ 44 | memoryregions | sizes | 45 +======================+=========+ 46 |FLASH |1,048,064| 47 |RAM | 196,608| 48 |VECTOR_TABLE | 512| 49 +======================+=========+ 50 |Total |1,245,184| 51 +----------------------+---------+ 52 53**Running a size report diff** 54 55.. code-block:: sh 56 57 58 $ pw bloat out/docs/obj/pw_metric/size_report/bin/one_metric.elf \ 59 --diff out/docs/obj/pw_metric/size_report/bin/base.elf \ 60 -d symbols 61 62 ▒█████▄ █▓ ▄███▒ ▒█ ▒█ ░▓████▒ ░▓████▒ ▒▓████▄ 63 ▒█░ █░ ░█▒ ██▒ ▀█▒ ▒█░ █ ▒█ ▒█ ▀ ▒█ ▀ ▒█ ▀█▌ 64 ▒█▄▄▄█░ ░█▒ █▓░ ▄▄░ ▒█░ █ ▒█ ▒███ ▒███ ░█ █▌ 65 ▒█▀ ░█░ ▓█ █▓ ░█░ █ ▒█ ▒█ ▄ ▒█ ▄ ░█ ▄█▌ 66 ▒█ ░█░ ░▓███▀ ▒█▓▀▓█░ ░▓████▒ ░▓████▒ ▒▓████▀ 67 68 +-----------------------------------------------------------------------------------+ 69 | | 70 +-----------------------------------------------------------------------------------+ 71 | diff| memoryregions | symbols | sizes| 72 +=====+======================+===============================================+======+ 73 | |FLASH | | -4| 74 | | |[section .FLASH.unused_space] | -408| 75 | | |main | +60| 76 | | |__sf_fake_stdout | +4| 77 | | |pw_boot_PreStaticMemoryInit | -2| 78 | | |_isatty | -2| 79 | NEW| |_GLOBAL__sub_I_group_foo | +84| 80 | NEW| |pw::metric::Group::~Group() | +34| 81 | NEW| |pw::intrusive_list_impl::List::insert_after() | +32| 82 | NEW| |pw::metric::Metric::Increment() | +32| 83 | NEW| |__cxa_atexit | +28| 84 | NEW| |pw::metric::Metric::Metric() | +28| 85 | NEW| |pw::metric::Metric::as_int() | +28| 86 | NEW| |pw::intrusive_list_impl::List::Item::unlist() | +20| 87 | NEW| |pw::metric::Group::Group() | +18| 88 | NEW| |pw::intrusive_list_impl::List::Item::previous()| +14| 89 | NEW| |pw::metric::TypedMetric<>::~TypedMetric() | +14| 90 | NEW| |__aeabi_atexit | +12| 91 +-----+----------------------+-----------------------------------------------+------+ 92 | |RAM | | 0| 93 | | |[section .stack] | -32| 94 | NEW| |group_foo | +16| 95 | NEW| |metric_x | +12| 96 | NEW| |[section .static_init_ram] | +4| 97 +=====+======================+===============================================+======+ 98 |Total| | | -4| 99 +-----+----------------------+-----------------------------------------------+------+ 100 101 102.. _bloat-howto: 103 104Defining size reports in GN 105=========================== 106 107Diff Size Reports 108^^^^^^^^^^^^^^^^^ 109Size reports can be defined using the GN template ``pw_size_diff``. The template 110requires at least two executable targets on which to perform a size diff. The 111base for the size diff can be specified either globally through the top-level 112``base`` argument, or individually per-binary within the ``binaries`` list. 113 114**Arguments** 115 116* ``base``: Optional default base target for all listed binaries. 117* ``source_filter``: Optional global regex to filter labels in the diff output. 118* ``data_sources``: Optional global list of datasources from bloaty config file 119* ``binaries``: List of binaries to size diff. Each binary specifies a target, 120 a label for the diff, and optionally a base target, source filter, and data 121 sources that override the global ones (if specified). 122 123 124.. code:: 125 126 import("$dir_pw_bloat/bloat.gni") 127 128 executable("empty_base") { 129 sources = [ "empty_main.cc" ] 130 } 131 132 executable("hello_world_printf") { 133 sources = [ "hello_printf.cc" ] 134 } 135 136 executable("hello_world_iostream") { 137 sources = [ "hello_iostream.cc" ] 138 } 139 140 pw_size_diff("my_size_report") { 141 base = ":empty_base" 142 data_sources = "symbols,segments" 143 binaries = [ 144 { 145 target = ":hello_world_printf" 146 label = "Hello world using printf" 147 }, 148 { 149 target = ":hello_world_iostream" 150 label = "Hello world using iostream" 151 data_sources = "symbols" 152 }, 153 ] 154 } 155 156A sample ``pw_size_diff`` ReST size report table can be found within module 157docs. For example, see the :ref:`pw_checksum-size-report` section of the 158``pw_checksum`` module for more detail. 159 160 161Single Binary Size Reports 162^^^^^^^^^^^^^^^^^^^^^^^^^^^ 163Size reports can also be defined using ``pw_size_report``, which provides 164a size report for a single binary. The template requires a target binary. 165 166**Arguments** 167 168* ``target``: Binary target to run size report on. 169* ``data_sources``: Optional list of data sources to organize outputs. 170* ``source_filter``: Optional regex to filter labels in the output. 171 172.. code:: 173 174 import("$dir_pw_bloat/bloat.gni") 175 176 executable("hello_world_iostream") { 177 sources = [ "hello_iostream.cc" ] 178 } 179 180 pw_size_report("hello_world_iostream_size_report") { 181 target = ":hello_iostream" 182 data_sources = "segments,symbols" 183 source_filter = "pw::hello" 184 } 185 186Sample Single Binary ASCII Table Generated 187 188.. code-block:: 189 190 ┌─────────────┬──────────────────────────────────────────────────┬──────┐ 191 │segment_names│ symbols │ sizes│ 192 ├═════════════┼══════════════════════════════════════════════════┼══════┤ 193 │FLASH │ │12,072│ 194 │ │pw::kvs::KeyValueStore::InitializeMetadata() │ 684│ 195 │ │pw::kvs::KeyValueStore::Init() │ 456│ 196 │ │pw::kvs::internal::EntryCache::Find() │ 444│ 197 │ │pw::kvs::FakeFlashMemory::Write() │ 240│ 198 │ │pw::kvs::internal::Entry::VerifyChecksumInFlash() │ 228│ 199 │ │pw::kvs::KeyValueStore::GarbageCollectSector() │ 220│ 200 │ │pw::kvs::KeyValueStore::RemoveDeletedKeyEntries() │ 220│ 201 │ │pw::kvs::KeyValueStore::AppendEntry() │ 204│ 202 │ │pw::kvs::KeyValueStore::Get() │ 194│ 203 │ │pw::kvs::internal::Entry::Read() │ 188│ 204 │ │pw::kvs::ChecksumAlgorithm::Finish() │ 26│ 205 │ │pw::kvs::internal::Entry::ReadKey() │ 26│ 206 │ │pw::kvs::internal::Sectors::BaseAddress() │ 24│ 207 │ │pw::kvs::ChecksumAlgorithm::Update() │ 20│ 208 │ │pw::kvs::FlashTestPartition() │ 8│ 209 │ │pw::kvs::FakeFlashMemory::Disable() │ 6│ 210 │ │pw::kvs::FakeFlashMemory::Enable() │ 6│ 211 │ │pw::kvs::FlashMemory::SelfTest() │ 6│ 212 │ │pw::kvs::FlashPartition::Init() │ 6│ 213 │ │pw::kvs::FlashPartition::sector_size_bytes() │ 6│ 214 │ │pw::kvs::FakeFlashMemory::IsEnabled() │ 4│ 215 ├─────────────┼──────────────────────────────────────────────────┼──────┤ 216 │RAM │ │ 1,424│ 217 │ │test_kvs │ 992│ 218 │ │pw::kvs::(anonymous namespace)::test_flash │ 384│ 219 │ │pw::kvs::(anonymous namespace)::test_partition │ 24│ 220 │ │pw::kvs::FakeFlashMemory::no_errors_ │ 12│ 221 │ │borrowable_kvs │ 8│ 222 │ │kvs_entry_count │ 4│ 223 ├═════════════┼══════════════════════════════════════════════════┼══════┤ 224 │Total │ │13,496│ 225 └─────────────┴──────────────────────────────────────────────────┴──────┘ 226 227 228Size reports are typically included in ReST documentation, as described in 229`Documentation integration`_. Size reports may also be printed in the build 230output if desired. To enable this in the GN build 231(``pigweed/pw_bloat/bloat.gni``), set the ``pw_bloat_SHOW_SIZE_REPORTS`` 232build arg to ``true``. 233 234Collecting size report data 235^^^^^^^^^^^^^^^^^^^^^^^^^^^ 236Each ``pw_size_report`` target outputs a JSON file containing the sizes of all 237top-level labels in the binary. (By default, this represents "segments", i.e. 238ELF program headers.) If a build produces multiple images, it may be useful to 239collect all of their sizes into a single file to provide a snapshot of sizes at 240some point in time --- for example, to display per-commit size deltas through 241CI. 242 243The ``pw_size_report_aggregation`` template is provided to collect multiple size 244reports' data into a single JSON file. 245 246**Arguments** 247 248* ``deps``: List of ``pw_size_report`` targets whose data to collect. 249* ``output``: Path to the output JSON file. 250 251.. code:: 252 253 import("$dir_pw_bloat/bloat.gni") 254 255 pw_size_report_aggregation("image_sizes") { 256 deps = [ 257 ":app_image_size_report", 258 ":bootloader_image_size_report", 259 ] 260 output = "$root_gen_dir/artifacts/image_sizes.json" 261 } 262 263Documentation integration 264========================= 265Bloat reports are easy to add to documentation files. All ``pw_size_diff`` 266and ``pw_size_report`` targets output a file containing a tabular report card. 267This file can be imported directly into a ReST documentation file using the 268``include`` directive. 269 270For example, the ``simple_bloat_loop`` and ``simple_bloat_function`` size 271reports under ``//pw_bloat/examples`` are imported into this file as follows: 272 273.. code:: rst 274 275 Simple bloat loop example 276 ^^^^^^^^^^^^^^^^^^^^^^^^^ 277 .. include:: examples/simple_bloat_loop 278 279 Simple bloat function example 280 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 281 .. include:: examples/simple_bloat_function 282 283Resulting in this output: 284 285Simple bloat loop example 286^^^^^^^^^^^^^^^^^^^^^^^^^ 287.. include:: examples/simple_bloat_loop 288 289Simple bloat function example 290^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 291.. include:: examples/simple_bloat_function 292 293Additional Bloaty data sources 294============================== 295`Bloaty McBloatface <https://github.com/google/bloaty>`_ by itself cannot help 296answer some questions which embedded developers frequently face such as 297understanding how much space is left. To address this, Pigweed provides Python 298tooling (``pw_bloat.bloaty_config``) to generate bloaty configuration files 299based on the final ELF files through small tweaks in the linker scripts to 300expose extra information. 301 302See the sections below on how to enable the additional data sections through 303modifications in your linker script(s). 304 305As an example to generate the helper configuration which enables additional data 306sources for ``example.elf`` if you've updated your linker script(s) accordingly, 307simply run 308``python -m pw_bloaty.bloaty_config example.elf > example.bloaty``. The 309``example.bloaty`` can then be used with bloaty using the ``-c`` flag, for 310example 311``bloaty -c example.bloaty example.elf --domain vm -d memoryregions,utilization`` 312which may return something like: 313 314.. code-block:: 315 316 84.2% 1023Ki FLASH 317 94.2% 963Ki Free space 318 5.8% 59.6Ki Used space 319 15.8% 192Ki RAM 320 100.0% 192Ki Used space 321 0.0% 512 VECTOR_TABLE 322 96.9% 496 Free space 323 3.1% 16 Used space 324 0.0% 0 Not resident in memory 325 NAN% 0 Used space 326 327 328``utilization`` data source 329^^^^^^^^^^^^^^^^^^^^^^^^^^^ 330The most common question many embedded developers face when using ``bloaty`` is 331how much space you are using and how much space is left. To correctly answer 332this, section sizes must be used in order to correctly account for section 333alignment requirements. 334 335The generated ``utilization`` data source will work with any ELF file, where 336``Used Space`` is reported for the sum of virtual memory size of all sections. 337 338In order for ``Free Space`` to be reported, your linker scripts must include 339properly aligned sections which span the unused remaining space for the relevant 340memory region with the ``unused_space`` string anywhere in their name. This 341typically means creating a trailing section which is pinned to span to the end 342of the memory region. 343 344For example imagine this partial example GNU LD linker script: 345 346.. code-block:: 347 348 MEMORY 349 { 350 FLASH(rx) : \ 351 ORIGIN = PW_BOOT_FLASH_BEGIN, \ 352 LENGTH = PW_BOOT_FLASH_SIZE 353 RAM(rwx) : \ 354 ORIGIN = PW_BOOT_RAM_BEGIN, \ 355 LENGTH = PW_BOOT_RAM_SIZE 356 } 357 358 SECTIONS 359 { 360 /* Main executable code. */ 361 .code : ALIGN(4) 362 { 363 /* Application code. */ 364 *(.text) 365 *(.text*) 366 KEEP(*(.init)) 367 KEEP(*(.fini)) 368 369 . = ALIGN(4); 370 /* Constants.*/ 371 *(.rodata) 372 *(.rodata*) 373 } >FLASH 374 375 /* Explicitly initialized global and static data. (.data)*/ 376 .static_init_ram : ALIGN(4) 377 { 378 *(.data) 379 *(.data*) 380 . = ALIGN(4); 381 } >RAM AT> FLASH 382 383 /* Zero initialized global/static data. (.bss) */ 384 .zero_init_ram (NOLOAD) : ALIGN(4) 385 { 386 *(.bss) 387 *(.bss*) 388 *(COMMON) 389 . = ALIGN(4); 390 } >RAM 391 } 392 393Could be modified as follows enable ``Free Space`` reporting: 394 395.. code-block:: 396 397 MEMORY 398 { 399 FLASH(rx) : ORIGIN = PW_BOOT_FLASH_BEGIN, LENGTH = PW_BOOT_FLASH_SIZE 400 RAM(rwx) : ORIGIN = PW_BOOT_RAM_BEGIN, LENGTH = PW_BOOT_RAM_SIZE 401 402 /* Each memory region above has an associated .*.unused_space section that 403 * overlays the unused space at the end of the memory segment. These 404 * segments are used by pw_bloat.bloaty_config to create the utilization 405 * data source for bloaty size reports. 406 * 407 * These sections MUST be located immediately after the last section that is 408 * placed in the respective memory region or lld will issue a warning like: 409 * 410 * warning: ignoring memory region assignment for non-allocatable section 411 * '.VECTOR_TABLE.unused_space' 412 * 413 * If this warning occurs, it's also likely that LLD will have created quite 414 * large padded regions in the ELF file due to bad cursor operations. This 415 * can cause ELF files to balloon from hundreds of kilobytes to hundreds of 416 * megabytes. 417 * 418 * Attempting to add sections to the memory region AFTER the unused_space 419 * section will cause the region to overflow. 420 */ 421 } 422 423 SECTIONS 424 { 425 /* Main executable code. */ 426 .code : ALIGN(4) 427 { 428 /* Application code. */ 429 *(.text) 430 *(.text*) 431 KEEP(*(.init)) 432 KEEP(*(.fini)) 433 434 . = ALIGN(4); 435 /* Constants.*/ 436 *(.rodata) 437 *(.rodata*) 438 } >FLASH 439 440 /* Explicitly initialized global and static data. (.data)*/ 441 .static_init_ram : ALIGN(4) 442 { 443 *(.data) 444 *(.data*) 445 . = ALIGN(4); 446 } >RAM AT> FLASH 447 448 /* Defines a section representing the unused space in the FLASH segment. 449 * This MUST be the last section assigned to the FLASH region. 450 */ 451 PW_BLOAT_UNUSED_SPACE(FLASH) 452 453 /* Zero initialized global/static data. (.bss). */ 454 .zero_init_ram (NOLOAD) : ALIGN(4) 455 { 456 *(.bss) 457 *(.bss*) 458 *(COMMON) 459 . = ALIGN(4); 460 } >RAM 461 462 /* Defines a section representing the unused space in the RAM segment. This 463 * MUST be the last section assigned to the RAM region. 464 */ 465 PW_BLOAT_UNUSED_SPACE(RAM) 466 } 467 468The preprocessor macro ``PW_BLOAT_UNUSED_SPACE`` is defined in 469``pw_bloat/bloat_macros.ld``. To use these macros include this file in your 470``pw_linker_script`` as follows: 471 472.. code-block:: 473 474 pw_linker_script("my_linker_script") { 475 includes = [ "$dir_pw_bloat/bloat_macros.ld" ] 476 linker_script = "my_project_linker_script.ld" 477 } 478 479Note that linker scripts are not natively supported by GN and can't be provided 480through ``deps``, the ``bloat_macros.ld`` must be passed in the ``includes`` 481list. 482 483.. _module-pw_bloat-memoryregions: 484 485``memoryregions`` data source 486^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 487Understanding how symbols, sections, and other data sources can be attributed 488back to the memory regions defined in your linker script is another common 489problem area. Unfortunately the ELF format does not include the original memory 490regions, meaning ``bloaty`` can not do this today by itself. In addition, it's 491relatively common that there are multiple memory regions which alias to the same 492memory but through different buses which could make attribution difficult. 493 494Instead of taking the less portable and brittle approach to parse ``*.map`` 495files, ``pw_bloat.bloaty_config`` consumes symbols which are defined in the 496linker script with a special format to extract this information from the ELF 497file: ``pw_bloat_config_memory_region_NAME_{start,end}{_N,}``. 498 499These symbols are defined by the preprocessor macros ``PW_BLOAT_MEMORY_REGION`` 500and ``PW_BLOAT_MEMORY_REGION_MAP`` with the right address and size for the 501regions. To use these macros include the ``pw_bloat/bloat_macros.ld`` in your 502``pw_linker_script`` as follows: 503 504.. code-block:: 505 506 pw_linker_script("my_linker_script") { 507 includes = [ "$dir_pw_bloat/bloat_macros.ld" ] 508 linker_script = "my_project_linker_script.ld" 509 } 510 511These symbols are then used to determine how to map segments to these memory 512regions. Note that segments must be used in order to account for inter-section 513padding which are not attributed against any sections. 514 515As an example, if you have a single view in the single memory region named 516``FLASH``, then you should include the following macro in your linker script to 517generate the symbols needed for the that region: 518 519.. code-block:: 520 521 PW_BLOAT_MEMORY_REGION(FLASH) 522 523As another example, if you have two aliased memory regions (``DCTM`` and 524``ITCM``) into the same effective memory named you'd like to call ``RAM``, then 525you should produce the following four symbols in your linker script: 526 527.. code-block:: 528 529 PW_BLOAT_MEMORY_REGION_MAP(RAM, ITCM) 530 PW_BLOAT_MEMORY_REGION_MAP(RAM, DTCM) 531