1# Diagnostic tooling support tiers 2 3Diagnostic tooling is important to the consumers of Node.js. It is used both 4in development and in production in order to investigate problems. The failure 5of one of these tools may be as big a problem for an end user as a bug within 6the runtime itself. 7 8The Node.js project has assessed the tools and the APIs which support those 9tools. Each of the tools and APIs has been put into one of 10the following tiers. 11 12* Tier 1 - Must always be working (CI tests passing) for all 13 Current and LTS Node.js releases. A release will not be shipped if the test 14 suite for the tool/API is not green. To be considered for inclusion 15 in this tier it must have a good test suite and that test suite and a job 16 must exist in the Node.js CI so that it can be run as part of the release 17 process. Tests on `main` will be run nightly when possible to provide 18 early warning of potential issues. No commit to the current and LTS 19 release branches should break this tool/API if the next major release 20 is within 1 month. In addition: 21 * The maintainers of the tool must remain responsive when there 22 are problems; 23 * The tool must be actively used by the ecosystem; 24 * The tool must be heavily depended on; 25 * The tool must have a guide or other documentation in the Node.js GitHub 26 organization or website; 27 * The tool must be working on all supported platforms; 28 * The tool must only be using APIs exposed by Node.js as opposed to 29 its dependencies; and 30 * The tool must be open source. 31 32* Tier 2 - Must be working (CI tests passing) for all 33 LTS releases. An LTS release will not be shipped if the test 34 suite for the tool/API is not green. To be considered for inclusion 35 in this tier it must have a good test suite and that test suite and a job 36 must exist in the Node.js CI so that it can be run as part of the release 37 process. In addition: 38 * The maintainers of the tool must remain responsive when 39 there are problems; 40 * The tool must be actively used by the ecosystem; 41 * The tool must be heavily depended on; 42 * The tool must have a guide or other documentation in the Node.js GitHub 43 organization or website; 44 * The tool must be open source. 45 46* Tier 3 - If possible its test suite will be run at least nightly 47 in the Node.js CI or in CITGM, and issues opened for failures. 48 Does not block shipping a release. 49 50* Tier 4 - Does not block shipping a release. 51 52* Unclassified - tool/API is new or does not have the required testing in the 53 Node.js CI in order to qualify for a higher tier. 54 55The choice of which tier a particular tool will be assigned to, will be a 56collaborative decision between Diagnostics WG and Release WG. Some of the 57criteria considered might be: 58 59* If the tool fits into a key category as listed below. 60* Whether the tool is actively used by the ecosystem. 61* The availability of alternatives. 62* Impact to the overall ecosystem if the tool is not working. 63* The availability of reliable test suite that can be integrated into our CI. 64* The availability of maintainer or community collaborator who will help 65 resolve issues when there are CI failures. 66* If the tool is maintained by the Node.js Foundation GitHub organization. 67 68The current categories of tools/APIs that fall under these Tiers are: 69 70* FFDC (F) - First failure data capture, easy to consume initial diagnostic 71 information. 72* Tracing (T) - use of logging to provide information about execution flow. 73* Memory (M) - tools that provide additional information about memory 74 used in the Heap or by native code. 75* Profiling (P) - tools that provide additional information about where 76 CPU cycles are being spent. 77* AsyncFlow (A) - tools that provide additional insight into asynchronous 78 execution flow. 79 80## Adding a tool to this list 81 82Any tool that might be used to investigate issues when running Node.js can 83be added to the list. If there is a new tool that should be added to the 84list, it should start by being added to the "Not yet classified" or 85"Tier 4" lists. Once it has been added to the list "promotion" to Tier 3 86through Tier 1 requires that the requirements listed above be met AND 87have agreement from Diagnostics WG and Release WG based on the criteria 88listed above. 89 90## Tiers 91 92The tools are currently assigned to Tiers as follows: 93 94## Tier 1 95 96| Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier | 97| --------- | --------------------- | ----------------------------- | ----------------------- | ----------- | 98| FFDC | [diagnostic report][] | Yes | Yes | 1 | 99| | | | | | 100 101## Tier 2 102 103| Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier | 104| --------- | ------------- | ----------------------------- | ----------------------- | ----------- | 105| | | | | | 106 107## Tier 3 108 109| Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier | 110| --------- | ------------------------------------ | ----------------------------- | ----------------------- | ----------- | 111| Profiling | V8 CPU profiler | Partial (V8 Tests) | Yes | 1 | 112| Profiling | --prof/--prof-process flags | Yes | Yes | 1 | 113| Profiling | V8 CodeEventHandler API | Partial (V8 Tests) | Yes | 2 | 114| Profiling | V8 --interpreted-frames-native-stack | Yes | Yes | 2 | 115| Profiling | [Linux perf][] | Yes | Partial | 2 | 116| Profiling | [node-clinic][] | No | No | 3 | 117| Debugger | [Chrome Dev tools][] | No | No | 3 | 118 119## Tier 4 120 121| Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier | 122| --------- | ------------- | ----------------------------- | ----------------------- | ----------- | 123| Profiling | [0x][] | No | No | 3 | 124 125## Not yet classified 126 127| Tool Type | Tool/API Name | Regular Testing in Node.js CI | Integrated with Node.js | Target Tier | 128| --------- | ----------------------------------------- | ----------------------------- | ----------------------- | ----------- | 129| Memory | V8 heap profiler | No | Yes | 1 | 130| Memory | V8 sampling heap profiler | No | Yes | 1 | 131| AsyncFlow | [Async Hooks (API)][] | ? | Yes | 1 | 132| Debugger | V8 Debug protocol (API) | No | Yes | 1 | 133| Debugger | [Command line Debug Client][] | ? | Yes | 1 | 134| Tracing | [trace\_events (API)][trace_events (API)] | No | Yes | 1 | 135| Tracing | trace\_gc | No | Yes | 1 | 136 137[0x]: https://github.com/davidmarkclements/0x 138[Async Hooks (API)]: https://nodejs.org/api/async_hooks.html 139[Chrome Dev Tools]: https://developer.chrome.com/docs/devtools/ 140[Command line Debug Client]: https://nodejs.org/api/inspector.html 141[Linux perf]: https://perf.wiki.kernel.org/index.php/Main_Page 142[diagnostic report]: https://nodejs.org/api/report.html 143[node-clinic]: https://github.com/clinicjs/node-clinic/ 144[trace_events (API)]: https://nodejs.org/api/tracing.html 145