1 //! Service load measurement 2 //! 3 //! This module provides the [`Load`] trait, which allows measuring how loaded a service is. 4 //! It also provides several wrapper types that measure load in different ways: 5 //! 6 //! - [`Constant`] — Always returns the same constant load value for a service. 7 //! - [`PendingRequests`] — Measures load by tracking the number of in-flight requests. 8 //! - [`PeakEwma`] — Measures load using a moving average of the peak latency for the service. 9 //! 10 //! In general, you will want to use one of these when using the types in [`tower::balance`] which 11 //! balance services depending on their load. Which load metric to use depends on your exact 12 //! use-case, but the ones above should get you quite far! 13 //! 14 //! When the `discover` feature is enabled, wrapper types for [`Discover`] that 15 //! wrap the discovered services with the given load estimator are also provided. 16 //! 17 //! # When does a request complete? 18 //! 19 //! For many applications, the request life-cycle is relatively simple: when a service responds to 20 //! a request, that request is done, and the system can forget about it. However, for some 21 //! applications, the service may respond to the initial request while other parts of the system 22 //! are still acting on that request. In such an application, the system load must take these 23 //! requests into account as well, or risk the system underestimating its own load. 24 //! 25 //! To support these use-cases, the load estimators in this module are parameterized by the 26 //! [`TrackCompletion`] trait, with [`CompleteOnResponse`] as the default type. The behavior of 27 //! [`CompleteOnResponse`] is what you would normally expect for a request-response cycle: when the 28 //! response is produced, the request is considered "finished", and load goes down. This can be 29 //! overriden by your own user-defined type to track more complex request completion semantics. See 30 //! the documentation for [`completion`] for more details. 31 //! 32 //! # Examples 33 //! 34 //! ```rust 35 //! # #[cfg(feature = "util")] 36 //! use tower::util::ServiceExt; 37 //! # #[cfg(feature = "util")] 38 //! use tower::{load::Load, Service}; 39 //! # #[cfg(feature = "util")] 40 //! async fn simple_balance<S1, S2, R>( 41 //! svc1: &mut S1, 42 //! svc2: &mut S2, 43 //! request: R 44 //! ) -> Result<S1::Response, S1::Error> 45 //! where 46 //! S1: Load + Service<R>, 47 //! S2: Load<Metric = S1::Metric> + Service<R, Response = S1::Response, Error = S1::Error> 48 //! { 49 //! if svc1.load() < svc2.load() { 50 //! svc1.ready().await?.call(request).await 51 //! } else { 52 //! svc2.ready().await?.call(request).await 53 //! } 54 //! } 55 //! ``` 56 //! 57 //! [`tower::balance`]: crate::balance 58 //! [`Discover`]: crate::discover::Discover 59 //! [`CompleteOnResponse`]: crate::load::completion::CompleteOnResponse 60 // TODO: a custom completion example would be good here 61 62 pub mod completion; 63 mod constant; 64 pub mod peak_ewma; 65 pub mod pending_requests; 66 67 pub use self::{ 68 completion::{CompleteOnResponse, TrackCompletion}, 69 constant::Constant, 70 peak_ewma::PeakEwma, 71 pending_requests::PendingRequests, 72 }; 73 74 #[cfg(feature = "discover")] 75 pub use self::{peak_ewma::PeakEwmaDiscover, pending_requests::PendingRequestsDiscover}; 76 77 /// Types that implement this trait can give an estimate of how loaded they are. 78 /// 79 /// See the module documentation for more details. 80 pub trait Load { 81 /// A comparable load metric. 82 /// 83 /// Lesser values indicate that the service is less loaded, and should be preferred for new 84 /// requests over another service with a higher value. 85 type Metric: PartialOrd; 86 87 /// Estimate the service's current load. load(&self) -> Self::Metric88 fn load(&self) -> Self::Metric; 89 } 90