Lines Matching +full:loss +full:- +full:of +full:- +full:signal
1 .. SPDX-License-Identifier: GPL-2.0
10 This readme tries to provide some background on the hows and whys of RDS,
14 http://oss.oracle.com/pipermail/rds-devel/2007-November/000228.html
22 cluster - so in a cluster with N processes you need N sockets, in contrast
23 to N*N if you use a connection-oriented socket transport like TCP.
25 RDS is not Infiniband-specific; it was designed to support different
29 The high-level semantics of RDS from the application's point of view are
34 the end point of a connection. All socket operations that involve
39 transport has to be IP-based. In fact, RDS over IB uses a
43 The port space is entirely independent of UDP, TCP or any other
59 a active-active HA scenario), but only as long as the address
64 RDS supports a number of sysctls in /proc/sys/net/rds
72 to create RDS sockets. SOL_RDS is the socket-level to be used
87 This is a soft limit rather than a hard limit - RDS will
105 return with -EMSGSIZE
108 of queued bytes over the SO_SNDSIZE threshold will return
124 the msg.msg_control buffer of struct msghdr. The format of the
133 we signal POLLIN.
136 to any destination, RDS will always signal POLLOUT as long as
137 there's room on the send queue (ie the number of bytes queued
141 a destination marked congested - in this case you will loop
144 this - by using congestion notifications, and by checking for
162 one of RDS_TRANS_TCP or RDS_TRANS_IB. When retrieving the
168 implicitly (via bind(2)) will return an error of EOPNOTSUPP.
175 see rds-rdma(7) manpage (available in rds-tools)
194 per-packet sequence number
196 piggybacked acknowledgment of last packet received
198 length of data, not including header
213 indicate to other end of connection that
222 passing RDMA-related information.
229 into memory. This creates a potential message loss if the HCA is
232 if another HCA is available for fail-over.
237 packets. Ack-only packets are reduced by only allowing one to be
244 RDS's IB transport uses a credit-based mechanism to verify that
253 number of bytes queued equals or exceeds rcvbuf then the socket
255 should return block or return -EWOULDBLOCK.
259 "back-pressure" is considered a bug.
264 terminate in the local address of the bitmap which changed.
270 is much easier to implement than some finer-grained
271 communication of per-port congestion. The sender does a very
279 As mentioned above, RDS is not IB-specific. Its code is divided
285 The transport layer handles the details of the transport. The IB
303 code to handle the actual copy-to-user.
306 per-socket information
309 per-connection information
312 pointers to transport-specific functions
315 non-transport-specific statistics
328 then maintained forever -- if there are transport errors, the
329 connection will be dropped and re-established.
332 partially-sent datagrams to be retransmitted when the connection is
333 re-established.
340 - struct rds_message built from incoming data
341 - CMSGs parsed (e.g. RDMA ops)
342 - transport connection alloced and connected if not already
343 - rds_message placed on send queue
344 - send worker awoken
347 - calls rds_send_xmit() until queue is empty
350 - transmits congestion map if one is pending
351 - may set ACK_REQUIRED
352 - calls transport to send either non-RDMA or RDMA message
356 - allocs work requests from send ring
357 - adds any new send credits available to peer (h_credits)
358 - maps the rds_message's sg list
359 - piggybacks ack
360 - populates work requests
361 - post send to connection's queue pair
367 - looks at write completions
368 - unmaps recv buffer from device
369 - no errors, call rds_ib_process_recv()
370 - refill recv ring
373 - validate header checksum
374 - copy header to rds_ib_incoming struct if start of a new datagram
375 - add to ibinc's fraglist
376 - if competed datagram:
377 - update cong map if datagram was cong update
378 - call rds_recv_incoming() otherwise
379 - note if ack is required
382 - drop duplicate packets
383 - respond to pings
384 - find the sock associated with this datagram
385 - add to sock queue
386 - wake up sock
387 - do some congestion calculations
389 - copy data into user iovec
390 - handle CMSGs
391 - return to application
395 Mprds is multipathed-RDS, primarily intended for RDS-over-TCP
397 implementation of RDS-over-TCP is implemented by demultiplexing multiple
402 (a) upper-bounded to the single-flow bandwidth,
403 (b) suffers from head-of-line blocking for all the RDS sockets.
409 (e.g., of local address and RDS port number) and packets for that RDS
414 a common (to all paths) part, and a per-path struct rds_conn_path. All
422 transport is multipath-capable, rds_sendmsg() hashes outgoing traffic
428 a different number of paths. As a result, the peering nodes need
429 to agree on the number of paths to be used for the connection.
436 port 0) with the ping packet having a rds extension header option of
438 number of paths supported by the sender. The "probe" ping packet will
440 The receiver of a ping from RDS_FLAG_PROBE_PORT will thus immediately
442 sent in response to a probe-ping should contain the rcvr's npaths
443 when the rcvr is mprds-capable.
445 If the rcvr is not mprds-capable, the exthdr in the ping will be
447 of the probe-ping can default to single-path mprds.