1# gRPC Completion Queue 2 3_Author: Sree Kuchibhotla (@sreecha) - Sep 2018_ 4 5Code: [completion_queue.cc](https://github.com/grpc/grpc/blob/v1.15.1/src/core/lib/surface/completion_queue.cc) 6 7This document gives an overview of completion queue architecture and focuses mainly on the interaction between completion queue and the Polling engine layer. 8 9## Completion queue attributes 10Completion queue has two attributes 11 12 - Completion_type: 13 - GRPC_CQ_NEXT: grpc_completion_queue_next() can be called (but not grpc_completion_queue_pluck()) 14 - GRPC_CQ_PLUCK: grpc_completion_queue_pluck() can be called (but not grpc_completion_queue_next()) 15 - GRPC_CQ_CALLBACK: The tags in the queue are function pointers to callbacks. Also, neither next() nor pluck() can be called on this 16 17 - Polling_type: 18 - GRPC_CQ_NON_POLLING: Threads calling completion_queue_next/pluck do not do any polling 19 - GRPC_CQ_DEFAULT_POLLING: Threads calling completion_queue_next/pluck do polling 20 - GRPC_CQ_NON_LISTENING: Functionally similar to default polling except for a boolean attribute that states that the cq is non-listening. This is used by the grpc-server code to not associate any listening sockets with this completion-queue’s pollset 21 22 23## Details 24 25 26 27 28### **grpc\_completion\_queue\_next()** & **grpc_completion_queue_pluck()** APIS 29 30 31``` C++ 32grpc_completion_queue_next(cq, deadline)/pluck(cq, deadline, tag) { 33 while(true) { 34 \\ 1. If an event is queued in the completion queue, dequeue and return 35 \\ (in case of pluck() dequeue only if the tag is the one we are interested in) 36 37 \\ 2. If completion queue shutdown return 38 39 \\ 3. In case of pluck, add (tag, worker) pair to the tag<->worker map on the cq 40 41 \\ 4. Call grpc_pollset_work(cq’s-pollset, deadline) to do polling 42 \\ Note that if this function found some fds to be readable/writable/error, 43 \\ it would have scheduled those closures (which may queue completion events 44 \\ on SOME completion queue - not necessarily this one) 45 } 46} 47``` 48 49### Queuing a completion event (i.e., "tag") 50 51``` C++ 52grpc_cq_end_op(cq, tag) { 53 \\ 1. Queue the tag in the event queue 54 55 \\ 2. Find the pollset corresponding to the completion queue 56 \\ (i) If the cq is of type GRPC_CQ_NEXT, then KICK ANY worker 57 \\ i.e., call grpc_pollset_kick(pollset, nullptr) 58 \\ (ii) If the cq is of type GRPC_CQ_PLUCK, then search the tag<->worker 59 \\ map on the completion queue to find the worker. Then specifically 60 \\ kick that worker i.e call grpc_pollset_kick(pollset, worker) 61} 62 63``` 64 65
