1# Troubleshooting gRPC 2 3This guide is for troubleshooting gRPC implementations based on C core library (sources for most of them are living in the `grpc/grpc` repository). 4 5## Enabling extra logging and tracing 6 7Extra logging can be very useful for diagnosing problems. It can be used to increase the amount of information 8that gets printed to stderr. 9 10## Setting Logging Severity and Verbosity 11 12[gRPC uses absl logging](https://abseil.io/docs/cpp/guides/logging). 13Verbosity can be set using absl flags such as 14`--minloglevel`, `--v` and `--vmodule`. 15 16These can also be programmatically set using 17[these absl APIs.](https://github.com/abseil/abseil-cpp/blob/master/absl/log/globals.h) 18 19Example 20``` 21# Disable all logs other than FATAL for the entire application 22./helloworld_application_using_grpc --v=-1 --minloglevel=3 23``` 24 25## GRPC_VERBOSITY 26 27This is not recommended for gRPC C++. For gRPC C++ it is strongly recommended that you use absl to control gRPC verbosity. For Python, ObjC, PHP, and Ruby, GRPC_VERBOSITY will be supported. Users need to note that this will change settings for all libraries/binaries that use absl in the application. GRPC_VERBOSITY will alter the global absl settings, not just settings for gRPC. 28 29To learn how to set GRPC_VERBOSITY refer [Environment Variables Overview](doc/environment_variables.md) 30 31## GRPC_TRACE 32 33`GRPC_TRACE` can be used to enable extra logging for specific internal gRPC components. Enabling the right traces can be invaluable 34for diagnosing for what is going wrong when things aren't working as intended. Possible values for `GRPC_TRACE` are [listed here](doc/trace_flags.md). 35Multiple traces can be enabled at once (use comma as separator). 36 37``` 38# Enable debug logs for the entire application 39./helloworld_application_using_grpc --v=2 --minloglevel=0 40``` 41 42``` 43# Print information about invocations of low-level C core API. 44# Note that trace logs that use `VLOG` won't be displayed. 45# Many tracers user log level INFO. 46# So unless absl settings are correct, no traces will be printed. 47GRPC_TRACE=api ./helloworld_application_using_grpc --v=-1 --minloglevel=0 48``` 49 50``` 51# Print info from 3 different tracers, including tracing logs 52GRPC_TRACE=tcp,http,api ./helloworld_application_using_grpc --v=2 --minloglevel=0 53``` 54 55Known limitations: `GPRC_TRACE=tcp` is currently not implemented for Windows (you won't see any tcp traces). 56 57Please note that the `GRPC_TRACE` environment variable has nothing to do with gRPC's "tracing" feature (= tracing RPCs in 58microservice environment to gain insight about how requests are processed by deployment), it is merely used to enable printing 59of extra logs. 60 61## Preventing gRPC Log Noise 62 63Log noise could consume a lot of resources. We recommend tuning settings for production systems very carefully. 64* Avoid using GRPC_VERBOSITY flag. This has been deprecated. If this value of this flag is anything other than "ERROR" or "NONE" it will cause log noise. 65* Always avoid setting --v and --vmodule to anything other than -1 for production systems. 66* Avoid setting --minloglevel=0 for production systems. Anything greater than 0 should be fine. 67* If setting this does not eliminate your log noise, look for instances of functions `--v`, `--vmodule`, `absl::SetVLogLevel` and `absl::SetMinLogLevel` in your entire codebase and any libraries/components/configs that you may be using. 68