1# PyTorch ONNX Exporter (POE) Rules are based on sarif ReportingDescriptor format. 2 3## Rules for PyTorch (TorchScript based) ONNX Exporter (POE) 4 5- id: POE0001 6 name: node-missing-onnx-shape-inference 7 short_description: 8 text: Node is missing ONNX shape inference. 9 full_description: 10 text: "Node is missing ONNX shape inference. 11 This usually happens when the node is not valid under standard ONNX operator spec." 12 markdown: | 13 Node is missing ONNX shape inference. 14 This usually happens when the node is not valid under standard ONNX operator spec. 15 message_strings: 16 default: 17 text: "The shape inference of {op_name} type is missing, so it may result in wrong shape inference for the exported graph. 18 Please consider adding it in symbolic function." 19 help_uri: 20 properties: 21 deprecated: false 22 tags: [] 23 24- id: POE0002 25 name: missing-custom-symbolic-function 26 short_description: 27 text: Missing symbolic function for custom PyTorch operator, cannot translate node to ONNX. 28 full_description: 29 text: Missing symbolic function for custom PyTorch operator, cannot translate node to ONNX. 30 markdown: | 31 Missing symbolic function for custom PyTorch operator, cannot translate node to ONNX. 32 message_strings: 33 default: 34 text: "ONNX export failed on an operator with unrecognized namespace {op_name}. 35 If you are trying to export a custom operator, make sure you registered 36 it with the right domain and version." 37 help_uri: 38 properties: 39 deprecated: false 40 tags: [] 41 42- id: POE0003 43 name: missing-standard-symbolic-function 44 short_description: 45 text: Missing symbolic function for standard PyTorch operator, cannot translate node to ONNX. 46 full_description: 47 text: Missing symbolic function for standard PyTorch operator, cannot translate node to ONNX. 48 markdown: | 49 Missing symbolic function for standard PyTorch operator, cannot translate node to ONNX. 50 message_strings: 51 default: 52 text: "Exporting the operator '{op_name}' to ONNX opset version {opset_version} is not supported. 53 Please feel free to request support or submit a pull request on PyTorch GitHub: {issue_url}." 54 help_uri: 55 properties: 56 deprecated: false 57 tags: [] 58 59 60- id: POE0004 61 name: operator-supported-in-newer-opset-version 62 short_description: 63 text: Operator is supported in newer opset version. 64 full_description: 65 text: Operator is supported in newer opset version. 66 markdown: | 67 Operator is supported in newer opset version. 68 69 Example: 70 ```python 71 torch.onnx.export(model, args, ..., opset_version=9) 72 ``` 73 message_strings: 74 default: 75 text: "Exporting the operator '{op_name}' to ONNX opset version {opset_version} is not supported. 76 Support for this operator was added in version {supported_opset_version}, try exporting with this version." 77 help_uri: 78 properties: 79 deprecated: false 80 tags: [] 81 82## Rules for PyTorch (dynamo fx based) ONNX Exporter (FXE) 83 84- id: FXE0007 85 name: fx-graph-to-onnx 86 short_description: 87 text: Transforms graph from FX IR to ONNX IR. 88 full_description: 89 text: "Transforms graph from FX IR to ONNX IR." 90 markdown: | 91 This diagnostic tracks the transformation process from an FX Graph (in FX IR) to an ONNX Graph (in ONNX IR). 92 93 ## Key Representations: 94 95 - **FX Graph**: The graph in FX IR produced by dynamo or symbolic tracing. 96 - **ONNX Graph**: The graph in ONNX IR and [operators](https://onnx.ai/onnx/operators/). 97 98 ## Additional Notes: 99 100 - Prior to this transformation step, the FX graph undergoes preprocessing through multiple FX passes. 101 To gain insight into these transformations, refer to diagnostic `FXE0010`. 102 - To enable a detailed view of the graph transformation in progress within this diagnostic, switch to the DEBUG mode. 103 104 - Set DiagnosticOptions.verbosity_level to logging.DEBUG. 105 - Activate the environment variable TORCH_LOGS='onnx_diagnostics'. 106 107 - For specific information related to node-level FX to ONNX transformations, explore the diagnostic `FXE0008`. 108 message_strings: 109 default: 110 text: "Transforming FX graph {graph_name} to ONNX graph." 111 help_uri: 112 properties: 113 deprecated: false 114 tags: [] 115 116- id: FXE0008 117 name: fx-node-to-onnx 118 short_description: 119 text: Transforms an FX node to an ONNX node. 120 full_description: 121 text: "Transforms an FX node to an ONNX node." 122 markdown: | 123 This diagnostic tracks the transformation process from an FX Node to ONNX [Operators](https://onnx.ai/onnx/operators/). 124 125 The process of converting FX Node to ONNX Node involves dealing with six distinct node types: 126 1. `placeholder`: Represents a module input, maps to an ONNX graph input. 127 2. `call_module`: Symbolizes a call to a submodule, maps to an ONNX 128 3. `call_method`: Symbolizes a method call. Not yet implemented. 129 4. `call_function`: Symbolizes a function call. [Core ATen](https://pytorch.org/docs/stable/ir.html#core-aten-ir) is expected 130 as the function call target. The mapping from ATen to ONNX is implemented by [ONNXScript torchlib](https://github.com/microsoft/onnxscript/tree/main/onnxscript/function_libs/torch_lib/ops). 131 This [guide](https://pytorch.org/docs/stable/onnx.html#onnx-script-functions) shows how to write and register a custom symbolic function for call_function FX node. 132 5. `get_attr`: Indicates an attribute access within the current module. Maps to an ONNX graph initializer. 133 6. `output`: Represents the module's output. Maps to an ONNX graph output. 134 135 For a granular understanding of how each node type is transformed, refer to the implementation details in `FxOnnxInterpreter`. 136 message_strings: 137 default: 138 text: "Transforming FX node {node_repr} to ONNX node." 139 help_uri: 140 properties: 141 deprecated: false 142 tags: [] 143 144- id: FXE0010 145 name: fx-pass 146 short_description: 147 text: FX graph transformation during ONNX export before converting from FX IR to ONNX IR. 148 full_description: 149 text: "FX graph transformation during ONNX export before converting from FX IR to ONNX IR." 150 markdown: | 151 This diagnostic tracks the FX passes executed during the ONNX export process prior 152 to converting from FX IR (Intermediate Representation) to ONNX IR. 153 154 Under the scope of ONNX export, an FX pass refers to a specific transformation applied to the FX GraphModule. 155 The primary aim of these passes is to streamline the graph into a format that aligns more with the ONNX IR. 156 Moreover, these passes work to substitute unsupported FX IR features with those recognized and endorsed by 157 ONNX IR. Common transformations include, but aren't limited to, decomposition, functionalization and 158 type promotion. 159 160 For those who are interested in a comprehensive log detailing the modifications made during these passes, 161 there are a couple of options: 162 163 - Set DiagnosticOptions.verbosity_level to logging.DEBUG. 164 - Activate the environment variable TORCH_LOGS='onnx_diagnostics'. 165 166 However, it's noteworthy that by default, such detailed logging is turned off. The primary reason being 167 its considerable impact on performance. 168 169 For an in-depth understanding of each specific pass, please refer to the directory: torch/onnx/_internal/fx/passes. 170 171 message_strings: 172 default: 173 text: "Running {pass_name} pass." 174 help_uri: 175 properties: 176 deprecated: false 177 tags: [] 178 179- id: FXE0011 180 name: no-symbolic-function-for-call-function 181 short_description: 182 text: Cannot find symbolic function to convert the "call_function" FX node to ONNX. 183 full_description: 184 text: "Cannot find symbolic function to convert the \"call_function\" FX node to ONNX. 185 " 186 markdown: | 187 This error occurs when the ONNX converter is unable to find a corresponding symbolic function 188 to convert a "call_function" node in the input graph to its equivalence in ONNX. The "call_function" 189 node represents a normalized function call in PyTorch, such as "torch.aten.ops.add". 190 191 To resolve this error, you can try one of the following: 192 193 - If exists, apply the auto-fix suggested by the diagnostic. TODO: this part is not available yet. 194 - Rewrite the model using only supported PyTorch operators or functions. 195 - Follow this [guide](https://pytorch.org/tutorials/beginner/onnx/onnx_registry_tutorial.html#overview) to write and 196 register a custom symbolic function for the unsupported call_function FX node. 197 198 message_strings: 199 default: 200 text: "No symbolic function to convert the \"call_function\" node {target} to ONNX. " 201 help_uri: 202 properties: 203 deprecated: false 204 tags: [] 205 206- id: FXE0012 207 name: unsupported-fx-node-analysis 208 short_description: 209 text: Result from FX graph analysis to reveal unsupported FX nodes. 210 full_description: 211 text: "Result from FX graph analysis to reveal unsupported FX nodes." 212 markdown: | 213 This error indicates that an FX graph contains one or more unsupported nodes. The error message 214 is typically accompanied by a list of the unsupported nodes found during analysis. 215 216 To resolve this error, you can try resolving each individual unsupported node error by following 217 the suggestions by its diagnostic. Typically, options include: 218 219 - If exists, apply the auto-fix suggested by the diagnostic. TODO: this part is not available yet. 220 - Rewrite the model using only supported PyTorch operators or functions. 221 - Follow this [guide](https://pytorch.org/docs/stable/onnx.html#onnx-script-functions) to write and 222 register a custom symbolic function for the unsupported call_function FX node. 223 message_strings: 224 default: 225 text: "Unsupported FX nodes: {node_op_to_target_mapping}. " 226 help_uri: 227 properties: 228 deprecated: false 229 tags: [] 230 231- id: FXE0013 232 name: op-level-debugging 233 short_description: 234 text: Report any op level validation failure in warnings. 235 full_description: 236 text: "Report any op level validation failure in warnings." 237 markdown: | 238 This warning message indicates that during op level debugging, certain symbolic functions 239 have failed to match the results of torch ops when using real tensors generated from fake 240 tensors. It is important to note that the symbolic functions may not necessarily be 241 incorrect, as the validation process is non-deterministic and should only be used as a 242 reference. 243 244 There are two categories of warnings that can be triggered: 245 246 1. Non-validated operators: 247 If the warnings are caused by the following errors, they can be disregarded by users, 248 as these errors occur due to the non-deterministic nature of the validation. However, 249 it is important to be aware that the operators have not been validated. 250 251 - IndexError: Unsupported input arguments of randomized dimensions/indices(INT64). 252 - RuntimeError: Unsupported input arguments for torch ops are generated. 253 - ValueError: Arguments/keyword arguments do not match the signature of the symbolic function. 254 255 2. Potentially wrong torchlib operators: 256 If the warnings are triggered by the following error, users should be aware that the symbolic functions 257 may be incorrect in dispatching or implementation. In such cases, it is recommended to report 258 the issue to the PyTorch-ONNX team, or create/register a custom symbolic function to replace the default one. 259 260 - AssertionError: The symbolic function is potentially wrong as the results do not match the results of torch ops. 261 - TypeError: The symbolic function is potentially wrong as the opschema doesn't match inputs. 262 263 message_strings: 264 default: 265 text: "FX node: {node} and its onnx function: {symbolic_fn} fails on op level validation." 266 help_uri: 267 properties: 268 deprecated: false 269 tags: [] 270 271- id: FXE0014 272 name: find-opschema-matched-symbolic-function 273 short_description: 274 text: Find the OnnxFunction that matches the input/attribute dtypes by comparing them with their opschemas. 275 full_description: 276 text: "Find the OnnxFunction that matches the input dtypes by comparing them with their opschemas. 277 A warning will be issued if the matched OnnxFunction is not an exact match." 278 markdown: | 279 When an ATen/Custom operator is registered and needs to be dispatched to an OnnxFunction, the input/attribute 280 dtypes of the ATen/Custom operator are compared with the input/attribute dtypes of the OnnxFunction opschemas 281 to find a match. However, if a perfect/exact match is not found, the dispatcher will attempt to find 282 the nearest match with the highest number of input/attribute dtypes matching the OnnxFunction opschemas, while 283 issuing a warning. 284 285 There are two types of level that can be triggered in this rule: 286 287 1. NOTE: A perfect match is found, and no warning is issued. 288 2. WARNING: The matched OnnxFunction is not a perfect/exact match. 289 290 Here are some suggestions based on the WARNING situation: 291 292 1. If there are NO errors or mismatches in the results, it is safe to disregard this warning, 293 as the definition of OnnxFunction schema is usually more stringent. 294 2. If there are errors or mismatches in the results, it is recommended to: 295 (a) Enable op_level_debugging to determine if the OnnxFunction might be incorrect. 296 (b) Report the issue to the PyTorch-ONNX team. 297 (c) Create/register a custom symbolic function to replace the default one. 298 299 message_strings: 300 default: 301 text: "The OnnxFunction: {symbolic_fn} is the nearest match of the node {node}." 302 help_uri: 303 properties: 304 deprecated: false 305 tags: [] 306 307- id: FXE0015 308 name: fx-node-insert-type-promotion 309 short_description: 310 text: Determine if type promotion is required for the FX node. Insert cast nodes if needed. 311 full_description: 312 text: "Determine if type promotion is required for the FX node. Insert cast nodes if needed." 313 markdown: | 314 This diagnostic monitors the node-level type promotion insertion process. In PyTorch, there is an automatic process called implicit type promotion, 315 where the input types of an operator are promoted to a common type. The determination of the common type is based on the type promotion rule specific to each operator. 316 To learn more about PyTorch's type promotion rules, refer to the [elementwise_dtypes doc](https://github.com/pytorch/pytorch/blob/f044613f78df713fb57f70c608483c9f10ad332e/torch/_prims_common/__init__.py#L1252-L1335) 317 and [torch._refs ops](https://github.com/pytorch/pytorch/blob/a475ea4542dfe961c9d097e33ab5041f61c8c17f/torch/_refs/__init__.py#L484). 318 319 However, implicit type promotion is not supported in ONNX. Therefore, to replicate the PyTorch behavior, we need to explicitly insert cast nodes. 320 This diagnostic tracks the process of node-level type promotion insertion. 321 322 The type promotion rules used by this process can be found in `torch/onnx/_internal/fx/passes/type_promotion.py.` 323 To update or add new type promotion rules, please refer to the [Note: Update type promotion rule] section. 324 message_strings: 325 default: 326 text: "Performing explicit type promotion for node {target}. " 327 help_uri: 328 properties: 329 deprecated: false 330 tags: [] 331 332- id: FXE0016 333 name: find-operator-overloads-in-onnx-registry 334 short_description: 335 text: Find the list of OnnxFunction of the PyTorch operator in onnx registry. 336 full_description: 337 text: "This rule involves finding the list of OnnxFunction for the PyTorch operator overload in the ONNX registry. 338 If the operator overload is not supported but its default overload is, a warning will be issued. 339 If both the operator overload and its default overload are not supported, an error will be issued." 340 markdown: | 341 The operator overload name serves the purpose of verifying whether a PyTorch operator is registered in the ONNX registry. 342 If it's not found, the dispatcher takes a fallback approach and tries to locate the default overload of the PyTorch 343 operator in the registry. If even the default overload is absent, it signifies that the operator is officially unsupported. 344 345 There are three types of level that can be triggered in this rule: 346 347 1. NOTE: The op overload is supported. 348 2. WARNING: The op overload is not supported, but it's default overload is supported. 349 3. ERROR: The op overload is not supported, and it's default overload is also not supported. 350 351 Here are some suggestions based on the WARNING situation: 352 353 1. If there are NO errors or mismatches in the results, it is safe to disregard this warning. 354 2. If there are errors or mismatches in the results, it is recommended to: 355 (a) Enable op_level_debugging to determine if the OnnxFunction might be incorrect. 356 (b) Report the unsupported overload to the PyTorch-ONNX team. 357 (c) Create/register a custom symbolic function to replace the default one. 358 359 Here are some suggestions based on the ERROR situation: 360 361 1. Report the unsupported operator to the PyTorch-ONNX team. 362 2. Create/register a custom symbolic function to replace the default one. 363 364 message_strings: 365 default: 366 text: "Checking if the FX node: {node} is supported in onnx registry." 367 help_uri: 368 properties: 369 deprecated: false 370 tags: [] 371