• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1Interoperability Test Case Descriptions
2=======================================
3
4Client and server use
5[test.proto](../src/proto/grpc/testing/test.proto)
6and the [gRPC over HTTP/2 v2 protocol](./PROTOCOL-HTTP2.md).
7
8Client
9------
10
11Clients implement test cases that test certain functionally. Each client is
12provided the test case it is expected to run as a command-line parameter. Names
13should be lowercase and without spaces.
14
15Clients should accept these arguments:
16* --server_host=HOSTNAME
17    * The server host to connect to. For example, "localhost" or "127.0.0.1"
18* --server_host_override=HOSTNAME
19    * The server host to claim to be connecting to, for use in TLS and HTTP/2
20      :authority header. If unspecified, the value of --server_host will be
21      used
22* --server_port=PORT
23    * The server port to connect to. For example, "8080"
24* --test_case=TESTCASE
25    * The name of the test case to execute. For example, "empty_unary"
26* --use_tls=BOOLEAN
27    * Whether to use a plaintext or encrypted connection
28* --use_test_ca=BOOLEAN
29    * Whether to replace platform root CAs with
30      [ca.pem](https://github.com/grpc/grpc/blob/master/src/core/tsi/test_creds/ca.pem)
31      as the CA root
32* --default_service_account=ACCOUNT_EMAIL
33    * Email of the GCE default service account.
34* --oauth_scope=SCOPE
35    * OAuth scope. For example, "https://www.googleapis.com/auth/xapi.zoo"
36* --service_account_key_file=PATH
37    * The path to the service account JSON key file generated from GCE developer
38    console.
39
40Clients must support TLS with ALPN. Clients must not disable certificate
41checking.
42
43### empty_unary
44
45This test verifies that implementations support zero-size messages. Ideally,
46client implementations would verify that the request and response were zero
47bytes serialized, but this is generally prohibitive to perform, so is not
48required.
49
50Server features:
51* [EmptyCall][]
52
53Procedure:
54 1. Client calls EmptyCall with the default Empty message
55
56Client asserts:
57* call was successful
58* response is non-null
59
60*It may be possible to use UnaryCall instead of EmptyCall, but it is harder to
61ensure that the proto serialized to zero bytes.*
62
63### cacheable_unary
64
65This test verifies that gRPC requests marked as cacheable use GET verb instead
66of POST, and that server sets appropriate cache control headers for the response
67to be cached by a proxy. This test requires that the server is behind
68a caching proxy. Use of current timestamp in the request prevents accidental
69cache matches left over from previous tests.
70
71Server features:
72* [CacheableUnaryCall][]
73
74Procedure:
75 1. Client calls CacheableUnaryCall with `SimpleRequest` request with payload
76    set to current timestamp. Timestamp format is irrelevant, and resolution is
77    in nanoseconds.
78    Client adds a `x-user-ip` header with value `1.2.3.4` to the request.
79    This is done since some proxys such as GFE will not cache requests from
80    localhost.
81    Client marks the request as cacheable by setting the cacheable flag in the
82    request context. Longer term this should be driven by the method option
83    specified in the proto file itself.
84 2. Client calls CacheableUnaryCall again immediately with the same request and
85    configuration as the previous call.
86
87Client asserts:
88* Both calls were successful
89* The payload body of both responses is the same.
90
91### large_unary
92
93This test verifies unary calls succeed in sending messages, and touches on flow
94control (even if compression is enabled on the channel).
95
96Server features:
97* [UnaryCall][]
98
99Procedure:
100 1. Client calls UnaryCall with:
101
102    ```
103    {
104      response_size: 314159
105      payload:{
106        body: 271828 bytes of zeros
107      }
108    }
109    ```
110
111Client asserts:
112* call was successful
113* response payload body is 314159 bytes in size
114* clients are free to assert that the response payload body contents are zero
115  and comparing the entire response message against a golden response
116
117### client_compressed_unary
118
119This test verifies the client can compress unary messages by sending two unary
120calls, for compressed and uncompressed payloads. It also sends an initial
121probing request to verify whether the server supports the [CompressedRequest][]
122feature by checking if the probing call fails with an `INVALID_ARGUMENT` status.
123
124Server features:
125* [UnaryCall][]
126* [CompressedRequest][]
127
128Procedure:
129 1. Client calls UnaryCall with the feature probe, an *uncompressed* message:
130    ```
131    {
132      expect_compressed:{
133        value: true
134      }
135      response_size: 314159
136      payload:{
137        body: 271828 bytes of zeros
138      }
139    }
140    ```
141
142 1. Client calls UnaryCall with the *compressed* message:
143
144    ```
145    {
146      expect_compressed:{
147        value: true
148      }
149      response_size: 314159
150      payload:{
151        body: 271828 bytes of zeros
152      }
153    }
154    ```
155
156 1. Client calls UnaryCall with the *uncompressed* message:
157
158    ```
159    {
160      expect_compressed:{
161        value: false
162      }
163      response_size: 314159
164      payload:{
165        body: 271828 bytes of zeros
166      }
167    }
168    ```
169
170    Client asserts:
171    * First call failed with `INVALID_ARGUMENT` status.
172    * Subsequent calls were successful.
173    * Response payload body is 314159 bytes in size.
174    * Clients are free to assert that the response payload body contents are
175      zeros and comparing the entire response message against a golden response.
176
177
178### server_compressed_unary
179
180This test verifies the server can compress unary messages. It sends two unary
181requests, expecting the server's response to be compressed or not according to
182the `response_compressed` boolean.
183
184Whether compression was actually performed is determined by the compression bit
185in the response's message flags. *Note that some languages may not have access
186to the message flags, in which case the client will be unable to verify that
187the `response_compressed` boolean is obeyed by the server*.
188
189
190Server features:
191* [UnaryCall][]
192* [CompressedResponse][]
193
194Procedure:
195 1. Client calls UnaryCall with `SimpleRequest`:
196
197    ```
198    {
199      response_compressed:{
200        value: true
201      }
202      response_size: 314159
203      payload:{
204        body: 271828 bytes of zeros
205      }
206    }
207    ```
208
209    ```
210    {
211      response_compressed:{
212        value: false
213      }
214      response_size: 314159
215      payload:{
216        body: 271828 bytes of zeros
217      }
218    }
219    ```
220    Client asserts:
221    * call was successful
222    * if supported by the implementation, when `response_compressed` is true,
223      the response MUST have the compressed message flag set.
224    * if supported by the implementation, when `response_compressed` is false,
225      the response MUST NOT have the compressed message flag set.
226    * response payload body is 314159 bytes in size in both cases.
227    * clients are free to assert that the response payload body contents are
228      zero and comparing the entire response message against a golden response
229
230
231### client_streaming
232
233This test verifies that client-only streaming succeeds.
234
235Server features:
236* [StreamingInputCall][]
237
238Procedure:
239 1. Client calls StreamingInputCall
240 2. Client sends:
241
242    ```
243    {
244      payload:{
245        body: 27182 bytes of zeros
246      }
247    }
248    ```
249
250 3. Client then sends:
251
252    ```
253    {
254      payload:{
255        body: 8 bytes of zeros
256      }
257    }
258    ```
259
260 4. Client then sends:
261
262    ```
263    {
264      payload:{
265        body: 1828 bytes of zeros
266      }
267    }
268    ```
269
270 5. Client then sends:
271
272    ```
273    {
274      payload:{
275        body: 45904 bytes of zeros
276      }
277    }
278    ```
279
280 6. Client half-closes
281
282Client asserts:
283* call was successful
284* response aggregated_payload_size is 74922
285
286
287### client_compressed_streaming
288
289This test verifies the client can compress requests on per-message basis by
290performing a two-request streaming call. It also sends an initial probing
291request to verify whether the server supports the [CompressedRequest][] feature
292by checking if the probing call fails with an `INVALID_ARGUMENT` status.
293
294Procedure:
295 1. Client calls `StreamingInputCall` and sends the following feature-probing
296    *uncompressed* `StreamingInputCallRequest` message
297
298    ```
299    {
300      expect_compressed:{
301        value: true
302      }
303      payload:{
304        body: 27182 bytes of zeros
305      }
306    }
307    ```
308    If the call does not fail with `INVALID_ARGUMENT`, the test fails.
309    Otherwise, we continue.
310
311 1. Client calls `StreamingInputCall` again, sending the *compressed* message
312
313    ```
314    {
315      expect_compressed:{
316        value: true
317      }
318      payload:{
319        body: 27182 bytes of zeros
320      }
321    }
322    ```
323
324 1. And finally, the *uncompressed* message
325    ```
326    {
327      expect_compressed:{
328        value: false
329      }
330      payload:{
331        body: 45904 bytes of zeros
332      }
333    }
334    ```
335
336 1. Client half-closes
337
338Client asserts:
339* First call fails with `INVALID_ARGUMENT`.
340* Next calls succeeds.
341* Response aggregated payload size is 73086.
342
343
344### server_streaming
345
346This test verifies that server-only streaming succeeds.
347
348Server features:
349* [StreamingOutputCall][]
350
351Procedure:
352 1. Client calls StreamingOutputCall with `StreamingOutputCallRequest`:
353
354    ```
355    {
356      response_parameters:{
357        size: 31415
358      }
359      response_parameters:{
360        size: 9
361      }
362      response_parameters:{
363        size: 2653
364      }
365      response_parameters:{
366        size: 58979
367      }
368    }
369    ```
370
371Client asserts:
372* call was successful
373* exactly four responses
374* response payload bodies are sized (in order): 31415, 9, 2653, 58979
375* clients are free to assert that the response payload body contents are zero
376  and comparing the entire response messages against golden responses
377
378### server_compressed_streaming
379
380This test verifies that the server can compress streaming messages and disable
381compression on individual messages, expecting the server's response to be
382compressed or not according to the `response_compressed` boolean.
383
384Whether compression was actually performed is determined by the compression bit
385in the response's message flags. *Note that some languages may not have access
386to the message flags, in which case the client will be unable to verify that the
387`response_compressed` boolean is obeyed by the server*.
388
389Server features:
390* [StreamingOutputCall][]
391* [CompressedResponse][]
392
393
394Procedure:
395 1. Client calls StreamingOutputCall with `StreamingOutputCallRequest`:
396
397    ```
398    {
399      response_parameters:{
400        compressed: {
401          value: true
402        }
403        size: 31415
404      }
405      response_parameters:{
406        compressed: {
407          value: false
408        }
409        size: 92653
410      }
411    }
412    ```
413
414    Client asserts:
415    * call was successful
416    * exactly two responses
417    * if supported by the implementation, when `response_compressed` is false,
418      the response's messages MUST NOT have the compressed message flag set.
419    * if supported by the implementation, when `response_compressed` is true,
420      the response's messages MUST have the compressed message flag set.
421    * response payload bodies are sized (in order): 31415, 92653
422    * clients are free to assert that the response payload body contents are
423      zero and comparing the entire response messages against golden responses
424
425### ping_pong
426
427This test verifies that full duplex bidi is supported.
428
429Server features:
430* [FullDuplexCall][]
431
432Procedure:
433 1. Client calls FullDuplexCall with:
434
435    ```
436    {
437      response_parameters:{
438        size: 31415
439      }
440      payload:{
441        body: 27182 bytes of zeros
442      }
443    }
444    ```
445
446 2. After getting a reply, it sends:
447
448    ```
449    {
450      response_parameters:{
451        size: 9
452      }
453      payload:{
454        body: 8 bytes of zeros
455      }
456    }
457    ```
458
459 3. After getting a reply, it sends:
460
461    ```
462    {
463      response_parameters:{
464        size: 2653
465      }
466      payload:{
467        body: 1828 bytes of zeros
468      }
469    }
470    ```
471
472 4. After getting a reply, it sends:
473
474    ```
475    {
476      response_parameters:{
477        size: 58979
478      }
479      payload:{
480        body: 45904 bytes of zeros
481      }
482    }
483    ```
484
485 5. After getting a reply, client half-closes
486
487Client asserts:
488* call was successful
489* exactly four responses
490* response payload bodies are sized (in order): 31415, 9, 2653, 58979
491* clients are free to assert that the response payload body contents are zero
492  and comparing the entire response messages against golden responses
493
494### empty_stream
495
496This test verifies that streams support having zero-messages in both
497directions.
498
499Server features:
500* [FullDuplexCall][]
501
502Procedure:
503 1. Client calls FullDuplexCall and then half-closes
504
505Client asserts:
506* call was successful
507* exactly zero responses
508
509### compute_engine_creds
510
511This test is only for cloud-to-prod path.
512
513This test verifies unary calls succeed in sending messages while using Service
514Credentials from GCE metadata server. The client instance needs to be created
515with desired oauth scope.
516
517The test uses `--default_service_account` with GCE service account email and
518`--oauth_scope` with the OAuth scope to use. For testing against
519grpc-test.sandbox.googleapis.com, "https://www.googleapis.com/auth/xapi.zoo"
520should
521be passed in as `--oauth_scope`.
522
523Server features:
524* [UnaryCall][]
525* [Echo Authenticated Username][]
526* [Echo OAuth Scope][]
527
528Procedure:
529 1. Client configures channel to use GCECredentials
530 2. Client calls UnaryCall on the channel with:
531
532    ```
533    {
534      response_size: 314159
535      payload:{
536        body: 271828 bytes of zeros
537      }
538      fill_username: true
539      fill_oauth_scope: true
540    }
541    ```
542
543Client asserts:
544* call was successful
545* received SimpleResponse.username equals the value of
546  `--default_service_account` flag
547* received SimpleResponse.oauth_scope is in `--oauth_scope`
548* response payload body is 314159 bytes in size
549* clients are free to assert that the response payload body contents are zero
550  and comparing the entire response message against a golden response
551
552### jwt_token_creds
553
554This test is only for cloud-to-prod path.
555
556This test verifies unary calls succeed in sending messages while using JWT
557token (created by the project's key file)
558
559Test caller should set flag `--service_account_key_file` with the
560path to json key file downloaded from
561https://console.developers.google.com. Alternately, if using a
562usable auth implementation, she may specify the file location in the environment
563variable GOOGLE_APPLICATION_CREDENTIALS.
564
565Server features:
566* [UnaryCall][]
567* [Echo Authenticated Username][]
568* [Echo OAuth Scope][]
569
570Procedure:
571 1. Client configures the channel to use JWTTokenCredentials
572 2. Client calls UnaryCall with:
573
574    ```
575    {
576      response_size: 314159
577      payload:{
578        body: 271828 bytes of zeros
579      }
580      fill_username: true
581    }
582    ```
583
584Client asserts:
585* call was successful
586* received SimpleResponse.username is not empty and is in the json key file used
587by the auth library. The client can optionally check the username matches the
588email address in the key file or equals the value of `--default_service_account`
589flag.
590* response payload body is 314159 bytes in size
591* clients are free to assert that the response payload body contents are zero
592  and comparing the entire response message against a golden response
593
594### oauth2_auth_token
595
596This test is only for cloud-to-prod path and some implementations may run
597in GCE only.
598
599This test verifies unary calls succeed in sending messages using an OAuth2 token
600that is obtained out of band. For the purpose of the test, the OAuth2 token is
601actually obtained from a service account credentials or GCE credentials via the
602language-specific authorization library.
603
604The difference between this test and the other auth tests is that it
605first uses the authorization library to obtain an authorization token.
606
607The test
608- uses the flag `--service_account_key_file` with the path to a json key file
609downloaded from https://console.developers.google.com. Alternately, if using a
610usable auth implementation, it may specify the file location in the environment
611variable GOOGLE_APPLICATION_CREDENTIALS, *OR* if GCE credentials is used to
612fetch the token, `--default_service_account` can be used to pass in GCE service
613account email.
614- uses the flag `--oauth_scope` for the oauth scope.  For testing against
615grpc-test.sandbox.googleapis.com, "https://www.googleapis.com/auth/xapi.zoo"
616should be passed as the `--oauth_scope`.
617
618Server features:
619* [UnaryCall][]
620* [Echo Authenticated Username][]
621* [Echo OAuth Scope][]
622
623Procedure:
624 1. Client uses the auth library to obtain an authorization token
625 2. Client configures the channel to use AccessTokenCredentials with the access
626    token obtained in step 1
627 3. Client calls UnaryCall with the following message
628
629    ```
630    {
631      fill_username: true
632      fill_oauth_scope: true
633    }
634    ```
635
636Client asserts:
637* call was successful
638* received SimpleResponse.username is valid. Depending on whether a service
639account key file or GCE credentials was used, client should check against the
640json key file or GCE default service account email.
641* received SimpleResponse.oauth_scope is in `--oauth_scope`
642
643### per_rpc_creds
644
645Similar to the other auth tests, this test is only for cloud-to-prod path.
646
647This test verifies unary calls succeed in sending messages using a JWT or a
648service account credentials set on the RPC.
649
650The test
651- uses the flag `--service_account_key_file` with the path to a json key file
652downloaded from https://console.developers.google.com. Alternately, if using a
653usable auth implementation, it may specify the file location in the environment
654variable GOOGLE_APPLICATION_CREDENTIALS
655- optionally uses the flag `--oauth_scope` for the oauth scope if implementator
656wishes to use service account credential instead of JWT credential. For testing
657against grpc-test.sandbox.googleapis.com, oauth scope
658"https://www.googleapis.com/auth/xapi.zoo" should be used.
659
660Server features:
661* [UnaryCall][]
662* [Echo Authenticated Username][]
663* [Echo OAuth Scope][]
664
665Procedure:
666 1. Client configures the channel with just SSL credentials
667 2. Client calls UnaryCall, setting per-call credentials to
668    JWTTokenCredentials. The request is the following message
669
670    ```
671    {
672      fill_username: true
673    }
674    ```
675
676Client asserts:
677* call was successful
678* received SimpleResponse.username is not empty and is in the json key file used
679by the auth library. The client can optionally check the username matches the
680email address in the key file.
681
682
683### custom_metadata
684
685This test verifies that custom metadata in either binary or ascii format can be
686sent as initial-metadata by the client and as both initial- and trailing-metadata
687by the server.
688
689Server features:
690* [UnaryCall][]
691* [FullDuplexCall][]
692* [Echo Metadata][]
693
694Procedure:
695 1. The client attaches custom metadata with the following keys and values:
696
697    ```
698    key: "x-grpc-test-echo-initial", value: "test_initial_metadata_value"
699    key: "x-grpc-test-echo-trailing-bin", value: 0xababab
700    ```
701
702    to a UnaryCall with request:
703
704    ```
705    {
706      response_size: 314159
707      payload:{
708        body: 271828 bytes of zeros
709      }
710    }
711    ```
712
713 2. The client attaches custom metadata with the following keys and values:
714
715    ```
716    key: "x-grpc-test-echo-initial", value: "test_initial_metadata_value"
717    key: "x-grpc-test-echo-trailing-bin", value: 0xababab
718    ```
719
720    to a FullDuplexCall with request:
721
722    ```
723    {
724      response_parameters:{
725        size: 314159
726      }
727      payload:{
728        body: 271828 bytes of zeros
729      }
730    }
731    ```
732
733    and then half-closes
734
735Client asserts:
736* call was successful
737* metadata with key `"x-grpc-test-echo-initial"` and value
738  `"test_initial_metadata_value"`is received in the initial metadata for calls
739  in Procedure steps 1 and 2.
740* metadata with key `"x-grpc-test-echo-trailing-bin"` and value `0xababab` is
741  received in the trailing metadata for calls in Procedure steps 1 and 2.
742
743
744
745### status_code_and_message
746
747This test verifies unary calls succeed in sending messages, and propagate back
748status code and message sent along with the messages.
749
750Server features:
751* [UnaryCall][]
752* [FullDuplexCall][]
753* [Echo Status][]
754
755Procedure:
756 1. Client calls UnaryCall with:
757
758    ```
759    {
760      response_status:{
761        code: 2
762        message: "test status message"
763      }
764    }
765    ```
766
767 2. Client calls FullDuplexCall with:
768
769    ```
770    {
771      response_status:{
772        code: 2
773        message: "test status message"
774      }
775    }
776    ```
777
778    and then half-closes
779
780
781Client asserts:
782* received status code is the same as the sent code for both Procedure steps 1
783  and 2
784* received status message is the same as the sent message for both Procedure
785  steps 1 and 2
786
787### special_status_message
788
789This test verifies Unicode and whitespace is correctly processed in status
790message. "\t" is horizontal tab. "\r" is carriage return.  "\n" is line feed.
791
792Server features:
793* [UnaryCall][]
794* [Echo Status][]
795
796Procedure:
797 1. Client calls UnaryCall with:
798
799    ```
800    {
801      response_status:{
802        code: 2
803        message: "\t\ntest with whitespace\r\nand Unicode BMP ☺ and non-BMP ��\t\n"
804      }
805    }
806    ```
807
808Client asserts:
809* received status code is the same as the sent code for Procedure step 1
810* received status message is the same as the sent message for Procedure step 1,
811  including all whitespace characters
812
813### unimplemented_method
814
815This test verifies that calling an unimplemented RPC method returns the
816UNIMPLEMENTED status code.
817
818Server features:
819N/A
820
821Procedure:
822* Client calls `grpc.testing.TestService/UnimplementedCall` with an empty
823  request (defined as `grpc.testing.Empty`):
824
825    ```
826    {
827    }
828    ```
829
830Client asserts:
831* received status code is 12 (UNIMPLEMENTED)
832
833### unimplemented_service
834
835This test verifies calling an unimplemented server returns the UNIMPLEMENTED
836status code.
837
838Server features:
839N/A
840
841Procedure:
842* Client calls `grpc.testing.UnimplementedService/UnimplementedCall` with an
843  empty request (defined as `grpc.testing.Empty`)
844
845Client asserts:
846* received status code is 12 (UNIMPLEMENTED)
847
848### cancel_after_begin
849
850This test verifies that a request can be cancelled after metadata has been sent
851but before payloads are sent.
852
853Server features:
854* [StreamingInputCall][]
855
856Procedure:
857 1. Client starts StreamingInputCall
858 2. Client immediately cancels request
859
860Client asserts:
861* Call completed with status CANCELLED
862
863### cancel_after_first_response
864
865This test verifies that a request can be cancelled after receiving a message
866from the server.
867
868Server features:
869* [FullDuplexCall][]
870
871Procedure:
872 1. Client starts FullDuplexCall with
873
874    ```
875    {
876      response_parameters:{
877        size: 31415
878      }
879      payload:{
880        body: 27182 bytes of zeros
881      }
882    }
883    ```
884
885 2. After receiving a response, client cancels request
886
887Client asserts:
888* Call completed with status CANCELLED
889
890### timeout_on_sleeping_server
891
892This test verifies that an RPC request whose lifetime exceeds its configured
893timeout value will end with the DeadlineExceeded status.
894
895Server features:
896* [FullDuplexCall][]
897
898Procedure:
899 1. Client calls FullDuplexCall with the following request and sets its timeout
900    to 1ms
901
902    ```
903    {
904      payload:{
905        body: 27182 bytes of zeros
906      }
907    }
908    ```
909
910 2. Client waits
911
912Client asserts:
913* Call completed with status DEADLINE_EXCEEDED.
914
915### concurrent_large_unary
916
917Status: TODO
918
919Client performs 1000 large_unary tests in parallel on the same channel.
920
921### Flow control. Pushback at client for large messages (TODO: fix name)
922
923Status: TODO
924
925This test verifies that a client sending faster than a server can drain sees
926pushback (i.e., attempts to send succeed only after appropriate delays).
927
928### Experimental Tests
929
930These tests are not yet standardized, and are not yet implemented in all
931languages. Therefore they are not part of our interop matrix.
932
933#### rpc_soak
934
935The client performs many large_unary RPCs in sequence over the same channel.
936The number of RPCs is configured by the experimental flag, `soak_iterations`.
937
938#### channel_soak
939
940The client performs many large_unary RPCs in sequence. Before each RPC, it
941tears down and rebuilds the channel. The number of RPCs is configured by
942the experimental flag, `soak_iterations`.
943
944This tests puts stress on several gRPC components; the resolver, the load
945balancer, and the RPC hotpath.
946
947#### long_lived_channel
948
949The client performs a number of large_unary RPCs over a single long-lived
950channel with a fixed but configurable interval between each RPC.
951
952### TODO Tests
953
954#### High priority:
955
956Propagation of status code and message (yangg)
957
958Multiple thousand simultaneous calls on same Channel (ctiller)
959
960Metadata: client headers, server headers + trailers, binary+ascii
961
962#### Normal priority:
963
964Cancel before start (ctiller)
965
966Cancel after sent first message (ctiller)
967
968Cancel after received headers (ctiller)
969
970Timeout but completed before expire (zhaoq)
971
972Multiple thousand simultaneous calls timeout on same Channel (ctiller)
973
974#### Lower priority:
975
976Flow control. Pushback at client for large messages (abhishek)
977
978Flow control. Pushback at server for large messages (abhishek)
979
980Going over max concurrent streams doesn't fail (client controls itself)
981(abhishek)
982
983RPC method not implemented (yangg)
984
985Multiple thousand simultaneous calls on different Channels (ctiller)
986
987Failed TLS hostname verification (ejona?)
988
989Large amount of headers to cause CONTINUATIONs; 63K of 'X's, all in one header.
990
991#### To priorize:
992
993Start streaming RPC but don't send any requests, server responds
994
995### Postponed Tests
996
997Resilience to buggy servers: These tests would verify that a client application
998isn't affected negatively by the responses put on the wire by a buggy server
999(e.g. the client library won't make the application crash).
1000
1001Reconnect after transport failure
1002
1003Reconnect backoff
1004
1005Fuzz testing
1006
1007
1008Server
1009------
1010
1011Servers implement various named features for clients to test with. Server
1012features are orthogonal. If a server implements a feature, it is always
1013available for clients. Names are simple descriptions for developer
1014communication and tracking.
1015
1016Servers should accept these arguments:
1017
1018* --port=PORT
1019
1020    * The port to listen on. For example, "8080"
1021
1022* --use_tls=BOOLEAN
1023
1024    * Whether to use a plaintext or encrypted connection
1025
1026Servers must support TLS with ALPN. They should use
1027[server1.pem](https://github.com/grpc/grpc/blob/master/src/core/tsi/test_creds/server1.pem)
1028for their certificate.
1029
1030### EmptyCall
1031[EmptyCall]: #emptycall
1032
1033Server implements EmptyCall which immediately returns the empty message.
1034
1035### UnaryCall
1036[UnaryCall]: #unarycall
1037
1038Server implements UnaryCall which immediately returns a SimpleResponse with a
1039payload body of size `SimpleRequest.response_size` bytes and type as appropriate
1040for the `SimpleRequest.response_type`. If the server does not support the
1041`response_type`, then it should fail the RPC with `INVALID_ARGUMENT`.
1042
1043### CacheableUnaryCall
1044[CacheableUnaryCall]: #cacheableunarycall
1045
1046Server gets the default SimpleRequest proto as the request. The content of the
1047request is ignored. It returns the SimpleResponse proto with the payload set
1048to current timestamp.  The timestamp is an integer representing current time
1049with nanosecond resolution. This integer is formated as ASCII decimal in the
1050response. The format is not really important as long as the response payload
1051is different for each request. In addition it adds
1052  1. cache control headers such that the response can be cached by proxies in
1053     the response path. Server should be behind a caching proxy for this test
1054     to pass. Currently we set the max-age to 60 seconds.
1055
1056### CompressedResponse
1057[CompressedResponse]: #compressedresponse
1058
1059When the client sets `response_compressed` to true, the server's response is
1060sent back compressed. Note that `response_compressed` is present on both
1061`SimpleRequest` (unary) and `StreamingOutputCallRequest` (streaming).
1062
1063### CompressedRequest
1064[CompressedRequest]: #compressedrequest
1065
1066When the client sets `expect_compressed` to true, the server expects the client
1067request to be compressed. If it's not, it fails the RPC with `INVALID_ARGUMENT`.
1068Note that `response_compressed` is present on both `SimpleRequest` (unary) and
1069`StreamingOutputCallRequest` (streaming).
1070
1071### StreamingInputCall
1072[StreamingInputCall]: #streaminginputcall
1073
1074Server implements StreamingInputCall which upon half close immediately returns
1075a StreamingInputCallResponse where aggregated_payload_size is the sum of all
1076request payload bodies received.
1077
1078### StreamingOutputCall
1079[StreamingOutputCall]: #streamingoutputcall
1080
1081Server implements StreamingOutputCall by replying, in order, with one
1082StreamingOutputCallResponse for each ResponseParameters in
1083StreamingOutputCallRequest. Each StreamingOutputCallResponse should have a
1084payload body of size ResponseParameters.size bytes, as specified by its
1085respective ResponseParameters. After sending all responses, it closes with OK.
1086
1087### FullDuplexCall
1088[FullDuplexCall]: #fullduplexcall
1089
1090Server implements FullDuplexCall by replying, in order, with one
1091StreamingOutputCallResponse for each ResponseParameters in each
1092StreamingOutputCallRequest. Each StreamingOutputCallResponse should have a
1093payload body of size ResponseParameters.size bytes, as specified by its
1094respective ResponseParameters. After receiving half close and sending all
1095responses, it closes with OK.
1096
1097### Echo Status
1098[Echo Status]: #echo-status
1099When the client sends a response_status in the request payload, the server closes
1100the stream with the status code and messsage contained within said response_status.
1101The server will not process any further messages on the stream sent by the client.
1102This can be used by clients to verify correct handling of different status codes and
1103associated status messages end-to-end.
1104
1105### Echo Metadata
1106[Echo Metadata]: #echo-metadata
1107When the client sends metadata with the key `"x-grpc-test-echo-initial"` with its
1108request, the server sends back exactly this key and the corresponding value back to
1109the client as part of initial metadata. When the client sends metadata with the key
1110`"x-grpc-test-echo-trailing-bin"` with its request, the server sends back exactly this
1111key and the corresponding value back to the client as trailing metadata.
1112
1113### Observe ResponseParameters.interval_us
1114[Observe ResponseParameters.interval_us]: #observe-responseparametersinterval_us
1115
1116In StreamingOutputCall and FullDuplexCall, server delays sending a
1117StreamingOutputCallResponse by the ResponseParameters's `interval_us` for that
1118particular response, relative to the last response sent. That is, `interval_us`
1119acts like a sleep *before* sending the response and accumulates from one
1120response to the next.
1121
1122Interaction with flow control is unspecified.
1123
1124### Echo Auth Information
1125
1126Status: Pending
1127
1128#### Echo Authenticated Username
1129[Echo Authenticated Username]: #echo-authenticated-username
1130
1131If a SimpleRequest has fill_username=true and that request was successfully
1132authenticated, then the SimpleResponse should have username filled with the
1133canonical form of the authenticated source. The canonical form is dependent on
1134the authentication method, but is likely to be a base 10 integer identifier or
1135an email address.
1136
1137#### Echo OAuth scope
1138[Echo OAuth Scope]: #echo-oauth-scope
1139
1140If a SimpleRequest has `fill_oauth_scope=true` and that request was successfully
1141authenticated via OAuth, then the SimpleResponse should have oauth_scope filled
1142with the scope of the method being invoked.
1143
1144Although a general server-side feature, most test servers won't implement this
1145feature. The TLS server `grpc-test.sandbox.googleapis.com:443` supports this
1146feature. It requires at least the OAuth scope
1147`https://www.googleapis.com/auth/xapi.zoo` for authentication to succeed.
1148
1149Discussion:
1150
1151Ideally, this would be communicated via metadata and not in the
1152request/response, but we want to use this test in code paths that don't yet
1153fully communicate metadata.
1154