1.. _docs-module-structure: 2 3---------------- 4Module Structure 5---------------- 6The Pigweed module structure is designed to keep as much code as possible for a 7particular slice of functionality in one place. That means including the code 8from multiple languages, as well as all the related documentation and tests. 9 10Additionally, the structure is designed to limit the number of places a file 11could go, so that when reading callsites it is obvious where a header is from. 12That is where the duplicated ``<module>`` occurrences in file paths comes from. 13 14Example module structure 15------------------------ 16.. code-block:: python 17 18 pw_foo/... 19 20 docs.rst # If there is just 1 docs file, call it docs.rst 21 README.md # All modules must have a short README for gittiles 22 23 BUILD.gn # GN build required 24 BUILD # Bazel build required 25 26 # C++ public headers; the repeated module name is required 27 public/pw_foo/foo.h 28 public/pw_foo/baz.h 29 30 # Exposed private headers go under internal/ 31 public/pw_foo/internal/bar.h 32 public/pw_foo/internal/qux.h 33 34 # Public override headers must go in 'public_overrides' 35 public_overrides/gtest/gtest.h 36 public_overrides/string.h 37 38 # Private headers go into <module>_*/... 39 pw_foo_internal/zap.h 40 pw_foo_private/zip.h 41 pw_foo_secret/alxx.h 42 43 # C++ implementations go in the root 44 foo_impl.cc 45 foo.cc 46 baz.cc 47 bar.cc 48 zap.cc 49 zip.cc 50 alxx.cc 51 52 # C++ tests also go in the root 53 foo_test.cc 54 bar_test.cc 55 zip_test.cc 56 57 # Python files go into 'py/<module>/...' 58 py/BUILD.gn # Python packages are declared in GN using pw_python_package 59 py/setup.py # Python files are structured as standard Python packages 60 py/foo_test.py # Tests go in py/ but outside of the Python package 61 py/bar_test.py 62 py/pw_foo/__init__.py 63 py/pw_foo/__main__.py 64 py/pw_foo/bar.py 65 py/pw_foo/py.typed # Indicates that this package has type annotations 66 67 # Go files go into 'go/...' 68 go/... 69 70 # Examples go in examples/, mixing different languages 71 examples/demo.py 72 examples/demo.cc 73 examples/demo.go 74 examples/BUILD.gn 75 examples/BUILD 76 77 # Size reports go under size_report/ 78 size_report/BUILD.gn 79 size_report/base.cc 80 size_report/use_case_a.cc 81 size_report/use_case_b.cc 82 83 # Protobuf definition files go into <module>_protos/... 84 pw_foo_protos/foo.proto 85 pw_foo_protos/internal/zap.proto 86 87 # Other directories are fine, but should be private. 88 data/... 89 graphics/... 90 collection_of_tests/... 91 code_relating_to_subfeature/... 92 93Module name 94----------- 95Pigweed upstream modules are always named with a prefix ``pw_`` to enforce 96namespacing. Projects using Pigweed that wish to make their own modules can use 97whatever name they like, but we suggest picking a short prefix to namespace 98your product (e.g. for an Internet of Toast project, perhaps the prefix could 99be ``it_``). 100 101C++ module structure 102-------------------- 103 104C++ public headers 105~~~~~~~~~~~~~~~~~~ 106Located ``{pw_module_dir}/public/<module>``. These are headers that must be 107exposed due to C++ limitations (i.e. are included from the public interface, 108but are not intended for public use). 109 110**Public headers** should take the form: 111 112``{pw_module_dir}/public/<module>/*.h`` 113 114**Exposed private headers** should take the form: 115 116``{pw_module_dir}/public/<module>/internal/*.h`` 117 118Examples: 119 120.. code-block:: 121 122 pw_foo/... 123 public/pw_foo/foo.h 124 public/pw_foo/a_header.h 125 public/pw_foo/baz.h 126 127For headers that must be exposed due to C++ limitations (i.e. are included from 128the public interface, but are not intended for use), place the headers in a 129``internal`` subfolder under the public headers directory; as 130``{pw_module_dir}/public/<module>/internal/*.h``. For example: 131 132.. code-block:: 133 134 pw_foo/... 135 public/pw_foo/internal/secret.h 136 public/pw_foo/internal/business.h 137 138.. note:: 139 140 These headers must not override headers from other modules. For 141 that, there is the ``public_overrides/`` directory. 142 143C++ public override headers 144~~~~~~~~~~~~~~~~~~~~~~~~~~~ 145Located ``{pw_module_dir}/public_overrides/<module>``. In general, the Pigweed 146philosophy is to avoid having "things hiding under rocks", and having header 147files with the same name that can override each other is considered a rock 148where surprising things can hide. Additionally, a design goal of the Pigweed 149module structure is to make it so there is ideally exactly one obvious place 150to find a header based on an ``#include``. 151 152However, in some cases header overrides are necessary to enable flexibly 153combining modules. To make this as explicit as possible, headers which override 154other headers must go in 155 156``{pw_module_dir}/public_overrides/...``` 157 158For example, the ``pw_unit_test`` module provides a header override for 159``gtest/gtest.h``. The structure of the module is (omitting some files): 160 161.. code-block:: 162 163 pw_unit_test/... 164 165 public_overrides/gtest 166 public_overrides/gtest/gtest.h 167 168 public/pw_unit_test 169 public/pw_unit_test/framework.h 170 public/pw_unit_test/simple_printing_event_handler.h 171 public/pw_unit_test/event_handler.h 172 173Note that the overrides are in a separate directory ``public_overrides``. 174 175C++ implementation files 176~~~~~~~~~~~~~~~~~~~~~~~~ 177Located ``{pw_module_dir}/``. C++ implementation files go at the top level of 178the module. Implementation files must always use "" style includes. 179 180Example: 181 182.. code-block:: 183 184 pw_unit_test/... 185 main.cc 186 framework.cc 187 test.gni 188 BUILD.gn 189 README.md 190 191.. _module-structure-compile-time-configuration: 192 193Compile-time configuration 194~~~~~~~~~~~~~~~~~~~~~~~~~~ 195Pigweed modules are intended to be used in a wide variety of environments. 196In support of this, some modules expose compile-time configuration options. 197Pigweed has an established pattern for declaring and overriding module 198configuration. 199 200.. tip:: 201 202 Compile-time configuration provides flexibility, but also imposes 203 restrictions. A module can only have one configuration in a given build. 204 This makes testing modules with compile-time configuration more difficult. 205 Where appropriate, consider alternatives such as C++ templates or runtime 206 configuration. 207 208Declaring configuration 209^^^^^^^^^^^^^^^^^^^^^^^ 210Configuration options are declared in a header file as macros. If the macro is 211not already defined, a default definition is provided. Otherwise, nothing is 212done. Configuration headers may include ``static_assert`` statements to validate 213configuration values. 214 215.. code-block:: c++ 216 217 // Example configuration header 218 219 #ifndef PW_FOO_INPUT_BUFFER_SIZE_BYTES 220 #define PW_FOO_INPUT_BUFFER_SIZE_BYTES 128 221 #endif // PW_FOO_INPUT_BUFFER_SIZE_BYTES 222 223 static_assert(PW_FOO_INPUT_BUFFER_SIZE_BYTES >= 64); 224 225The configuration header may go in one of three places in the module, depending 226on whether the header should be exposed by the module or not. 227 228.. code-block:: 229 230 pw_foo/... 231 232 # Publicly accessible configuration header 233 public/pw_foo/config.h 234 235 # Internal configuration header that is included by other module headers 236 public/pw_foo/internal/config.h 237 238 # Internal configuration header 239 pw_foo_private/config.h 240 241The configuration header is provided by a build system library. This library 242acts as a :ref:`facade<docs-module-structure-facades>`. The facade uses a 243variable such as ``pw_foo_CONFIG``. In upstream Pigweed, all config facades 244default to the ``pw_build_DEFAULT_MODULE_CONFIG`` backend. In the GN build 245system, the config facade is declared as follows: 246 247.. code-block:: 248 249 declare_args() { 250 # The build target that overrides the default configuration options for this 251 # module. This should point to a source set that provides defines through a 252 # public config (which may -include a file or add defines directly). 253 pw_foo_CONFIG = pw_build_DEFAULT_MODULE_CONFIG 254 } 255 256 # An example source set for each potential config header location follows. 257 258 # Publicly accessible configuration header (most common) 259 pw_source_set("config") { 260 public = [ "public/pw_foo/config.h" ] 261 public_configs = [ ":public_include_path" ] 262 public_deps = [ pw_foo_CONFIG ] 263 } 264 265 # Internal configuration header that is included by other module headers 266 pw_source_set("config") { 267 sources = [ "public/pw_foo/internal/config.h" ] 268 public_configs = [ ":public_include_path" ] 269 public_deps = [ pw_foo_CONFIG ] 270 visibility = [":*"] # Only allow this module to depend on ":config" 271 friend = [":*"] # Allow this module to access the config.h header. 272 } 273 274 # Internal configuration header 275 pw_source_set("config") { 276 public = [ "pw_foo_private/config.h" ] 277 public_deps = [ pw_foo_CONFIG ] 278 visibility = [":*"] # Only allow this module to depend on ":config" 279 } 280 281Overriding configuration 282^^^^^^^^^^^^^^^^^^^^^^^^ 283As noted above, all module configuration facades default to the same backend 284(``pw_build_DEFAULT_MODULE_CONFIG``). This allows projects to override 285configuration values for multiple modules from a single configuration backend, 286if desired. The configuration values may also be overridden individually by 287setting backends for the individual module configurations (e.g. in GN, 288``pw_foo_CONFIG = "//configuration:my_foo_config"``). 289 290Configurations options are overridden by setting macros in the config backend. 291These macro definitions can be provided through compilation options, such as 292``-DPW_FOO_INPUT_BUFFER_SIZE_BYTES=256``. Configuration macro definitions may 293also be set in a header file. The header file is included using the ``-include`` 294compilation option. 295 296This example shows two ways to configure a module in the GN build system. 297 298.. code-block:: 299 300 # In the toolchain, set either pw_build_DEFAULT_MODULE_CONFIG or pw_foo_CONFIG 301 pw_build_DEFAULT_MODULE_CONFIG = get_path_info(":define_overrides", "abspath") 302 303 # This configuration sets PW_FOO_INPUT_BUFFER_SIZE_BYTES using the -D flag. 304 pw_source_set("define_overrides") { 305 public_configs = [ ":define_options" ] 306 } 307 308 config("define_options") { 309 defines = [ "PW_FOO_INPUT_BUFFER_SIZE_BYTES=256" ] 310 } 311 312 # This configuration sets PW_FOO_INPUT_BUFFER_SIZE_BYTES in a header file. 313 pw_source_set("include_overrides") { 314 public_configs = [ ":set_options_in_header_file" ] 315 316 # Header file with #define PW_FOO_INPUT_BUFFER_SIZE_BYTES 256 317 sources = [ "my_config_overrides.h" ] 318 } 319 320 config("set_options_in_header_file") { 321 cflags = [ 322 "-include", 323 rebase_path("my_config_overrides.h", root_build_dir), 324 ] 325 } 326 327.. admonition:: Why this config pattern is preferred 328 329 Alternate patterns for configuring a module include overriding the module's 330 config header or having that header optionally include a header at a known 331 path (e.g. ``pw_foo/config_overrides.h``). There are a few downsides to these 332 approaches: 333 334 * The module needs its own config header that defines, provides defaults for, 335 and validates the configuration options. Replacing this header with a 336 user-defined header would require defining all options in the user's header, 337 which is cumbersome and brittle, and would bypass validation in the module's 338 header. 339 * Including a config override header at a particular path would prevent 340 multiple modules from sharing the same configuration file. Multiple headers 341 could redirect to the same configuration file, but this would still require 342 creating a separate header and setting the config backend variable for each 343 module. 344 * Optionally including a config override header requires boilerplate code that 345 would have to be duplicated in every configurable module. 346 * An optional config override header file would silently be excluded if the 347 file path were accidentally misspelled. 348 349Python module structure 350----------------------- 351Python code is structured as described in the :ref:`docs-python-build-structure` 352section of :ref:`docs-python-build`. 353 354.. _docs-module-structure-facades: 355 356Facades 357------- 358In Pigweed, facades represent a dependency that can be swapped at compile time. 359Facades are similar in concept to a virtual interface, but the implementation is 360set by the build system. Runtime polymorphism with facades is not 361possible, and each facade may only have one implementation (backend) per 362toolchain compilation. 363 364In the simplest sense, a facade is just a dependency represented by a variable. 365For example, the ``pw_log`` facade is represented by the ``pw_log_BACKEND`` 366build variable. Facades typically are bundled with a build system library that 367depends on the backend. 368 369Facades are essential in some circumstances: 370 371* Low-level, platform-specific features (:ref:`module-pw_cpu_exception`). 372* Features that require a macro or non-virtual function interface 373 (:ref:`module-pw_log`, :ref:`module-pw_assert`). 374* Highly leveraged code where a virtual interface or callback is too costly or 375 cumbersome (:ref:`module-pw_tokenizer`). 376 377.. caution:: 378 379 Modules should only use facades when necessary. Facades are permanently locked 380 to a particular implementation at compile time. Multiple backends cannot be 381 used in one build, and runtime dependency injection is not possible, which 382 makes testing difficult. Where appropriate, modules should use other 383 mechanisms, such as virtual interfaces, callbacks, or templates, in place of 384 facades. 385 386The GN build system provides the 387:ref:`pw_facade template<module-pw_build-facade>` as a convenient way to declare 388facades. 389 390Documentation 391------------- 392Documentation should go in the root module folder, typically in the 393``docs.rst`` file. There must be a docgen entry for the documentation in the 394``BUILD.gn`` file with the target name ``docs``; so the full target for the 395docs would be ``<module>:docs``. 396 397.. code-block:: 398 399 pw_example_module/... 400 401 docs.rst 402 403For modules with more involved documentation, create a separate directory 404called ``docs/`` under the module root, and put the ``.rst`` files and other 405related files (like images and diagrams) there. 406 407.. code-block:: 408 409 pw_example_module/... 410 411 docs/docs.rst 412 docs/bar.rst 413 docs/foo.rst 414 docs/image/screenshot.png 415 docs/image/diagram.svg 416 417Creating a new Pigweed module 418----------------------------- 419To create a new Pigweed module, follow the below steps. 420 421.. tip:: 422 423 Connect with the Pigweed community (by `mailing the Pigweed list 424 <https://groups.google.com/forum/#!forum/pigweed>`_ or `raising your idea 425 in the Pigweed chat <https://discord.gg/M9NSeTA>`_) to discuss your module 426 idea before getting too far into the implementation. This can prevent 427 accidentally duplicating work, or avoiding writing code that won't get 428 accepted. 429 4301. Create module folder following `Module name`_ guidelines. 4312. Add `C++ public headers`_ files in 432 ``{pw_module_dir}/public/{pw_module_name}/`` 4333. Add `C++ implementation files`_ files in ``{pw_module_dir}/`` 4344. Add module documentation 435 436 - Add ``{pw_module_dir}/README.md`` that has a module summary 437 - Add ``{pw_module_dir}/docs.rst`` that contains the main module 438 documentation 439 4405. Add GN build support in ``{pw_module_dir}/BUILD.gn`` 441 442 - Declare tests in ``pw_test_group("tests")`` 443 - Declare docs in ``pw_docs_group("docs")`` 444 4456. Add Bazel build support in ``{pw_module_dir}/BUILD.bazel`` 446 4477. Add CMake build support in ``{pw_module_dir}/CMakeLists.txt`` 448 4498. Add the new module to the ``/PIGWEED_MODULES`` list 450 451 Modules must be listed one per line with no extra spaces or comments. This 452 automatically adds the new module, its tests, and its docs, to the GN build. 453 4549. Update the generated Pigweed modules lists file 455 456 .. code-block:: bash 457 458 ninja -C out update_modules 459 46010. Add the new module to CMake build 461 462 - In ``/CMakeLists.txt`` add ``add_subdirectory(pw_new)`` 463 46411. Run :ref:`module-pw_module-module-check` 465 466 - ``$ pw module-check {pw_module_dir}`` 467 46812. Contribute your module to upstream Pigweed (optional but encouraged!) 469
