1.. program:: nghttpx 2 3nghttpx - HTTP/2 proxy - HOW-TO 4=============================== 5 6:doc:`nghttpx.1` is a proxy translating protocols between HTTP/2 and 7other protocols (e.g., HTTP/1). It operates in several modes and each 8mode may require additional programs to work with. This article 9describes each operation mode and explains the intended use-cases. It 10also covers some useful options later. 11 12Default mode 13------------ 14 15If nghttpx is invoked without :option:`--http2-proxy`, it operates in 16default mode. In this mode, it works as reverse proxy (gateway) for 17HTTP/3, HTTP/2 and HTTP/1 clients to backend servers. This is also 18known as "HTTP/2 router". 19 20By default, frontend connection is encrypted using SSL/TLS. So 21server's private key and certificate must be supplied to the command 22line (or through configuration file). In this case, the frontend 23protocol selection will be done via ALPN or NPN. 24 25To turn off encryption on frontend connection, use ``no-tls`` keyword 26in :option:`--frontend` option. HTTP/2 and HTTP/1 are available on 27the frontend, and an HTTP/1 connection can be upgraded to HTTP/2 using 28HTTP Upgrade. Starting HTTP/2 connection by sending HTTP/2 connection 29preface is also supported. 30 31In order to receive HTTP/3 traffic, use ``quic`` parameter in 32:option:`--frontend` option (.e.g, ``--frontend='*,443;quic'``) 33 34nghttpx can listen on multiple frontend addresses. This is achieved 35by using multiple :option:`--frontend` options. For each frontend 36address, TLS can be enabled or disabled. 37 38By default, backend connections are not encrypted. To enable TLS 39encryption on backend connections, use ``tls`` keyword in 40:option:`--backend` option. Using patterns and ``proto`` keyword in 41:option:`--backend` option, backend application protocol can be 42specified per host/request path pattern. It means that you can use 43both HTTP/2 and HTTP/1 in backend connections at the same time. Note 44that default backend protocol is HTTP/1.1. To use HTTP/2 in backend, 45you have to specify ``h2`` in ``proto`` keyword in :option:`--backend` 46explicitly. 47 48The backend is supposed to be a Web server. For example, to make 49nghttpx listen to encrypted HTTP/2 requests at port 8443, and a 50backend Web server is configured to listen to HTTP requests at port 518080 on the same host, run nghttpx command-line like this: 52 53.. code-block:: text 54 55 $ nghttpx -f0.0.0.0,8443 -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt 56 57Then an HTTP/2 enabled client can access the nghttpx server using HTTP/2. For 58example, you can send a GET request using nghttp: 59 60.. code-block:: text 61 62 $ nghttp -nv https://localhost:8443/ 63 64HTTP/2 proxy mode 65----------------- 66 67If nghttpx is invoked with :option:`--http2-proxy` (or its shorthand 68:option:`-s`) option, it operates in HTTP/2 proxy mode. The supported 69protocols in frontend and backend connections are the same as in `default 70mode`_. The difference is that this mode acts like a forward proxy and 71assumes the backend is an HTTP proxy server (e.g., Squid, Apache Traffic 72Server). HTTP/1 requests must include an absolute URI in request line. 73 74By default, the frontend connection is encrypted. So this mode is 75also called secure proxy. 76 77To turn off encryption on the frontend connection, use ``no-tls`` keyword 78in :option:`--frontend` option. 79 80The backend must be an HTTP proxy server. nghttpx supports multiple 81backend server addresses. It translates incoming requests to HTTP 82request to backend server. The backend server performs real proxy 83work for each request, for example, dispatching requests to the origin 84server and caching contents. 85 86The backend connection is not encrypted by default. To enable 87encryption, use ``tls`` keyword in :option:`--backend` option. The 88default backend protocol is HTTP/1.1. To use HTTP/2 in backend 89connection, use :option:`--backend` option, and specify ``h2`` in 90``proto`` keyword explicitly. 91 92For example, to make nghttpx listen to encrypted HTTP/2 requests at 93port 8443, and a backend HTTP proxy server is configured to listen to 94HTTP/1 requests at port 8080 on the same host, run nghttpx command-line 95like this: 96 97.. code-block:: text 98 99 $ nghttpx -s -f'*,8443' -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt 100 101At the time of this writing, Firefox 41 and Chromium v46 can use 102nghttpx as HTTP/2 proxy. 103 104To make Firefox or Chromium use nghttpx as HTTP/2 proxy, user has to 105create proxy.pac script file like this: 106 107.. code-block:: javascript 108 109 function FindProxyForURL(url, host) { 110 return "HTTPS SERVERADDR:PORT"; 111 } 112 113``SERVERADDR`` and ``PORT`` is the hostname/address and port of the 114machine nghttpx is running. Please note that both Firefox and 115Chromium require valid certificate for secure proxy. 116 117For Firefox, open Preference window and select Advanced then click 118Network tab. Clicking Connection Settings button will show the 119dialog. Select "Automatic proxy configuration URL" and enter the path 120to proxy.pac file, something like this: 121 122.. code-block:: text 123 124 file:///path/to/proxy.pac 125 126For Chromium, use following command-line: 127 128.. code-block:: text 129 130 $ google-chrome --proxy-pac-url=file:///path/to/proxy.pac --use-npn 131 132As HTTP/1 proxy server, Squid may work as out-of-box. Traffic server 133requires to be configured as forward proxy. Here is the minimum 134configuration items to edit: 135 136.. code-block:: text 137 138 CONFIG proxy.config.reverse_proxy.enabled INT 0 139 CONFIG proxy.config.url_remap.remap_required INT 0 140 141Consult Traffic server `documentation 142<http://trafficserver.readthedocs.org/en/latest/admin-guide/configuration/transparent-forward-proxying.en.html>`_ 143to know how to configure traffic server as forward proxy and its 144security implications. 145 146ALPN support 147------------ 148 149ALPN support requires OpenSSL >= 1.0.2. 150 151Disable frontend SSL/TLS 152------------------------ 153 154The frontend connections are encrypted with SSL/TLS by default. To 155turn off SSL/TLS, use ``no-tls`` keyword in :option:`--frontend` 156option. If this option is used, the private key and certificate are 157not required to run nghttpx. 158 159Enable backend SSL/TLS 160---------------------- 161 162The backend connections are not encrypted by default. To enable 163SSL/TLS encryption, use ``tls`` keyword in :option:`--backend` option. 164 165Enable SSL/TLS on memcached connection 166-------------------------------------- 167 168By default, memcached connection is not encrypted. To enable 169encryption, use ``tls`` keyword in 170:option:`--tls-ticket-key-memcached` for TLS ticket key, and 171:option:`--tls-session-cache-memcached` for TLS session cache. 172 173Specifying additional server certificates 174----------------------------------------- 175 176nghttpx accepts additional server private key and certificate pairs 177using :option:`--subcert` option. It can be used multiple times. 178 179Specifying additional CA certificate 180------------------------------------ 181 182By default, nghttpx tries to read CA certificate from system. But 183depending on the system you use, this may fail or is not supported. 184To specify CA certificate manually, use :option:`--cacert` option. 185The specified file must be PEM format and can contain multiple 186certificates. 187 188By default, nghttpx validates server's certificate. If you want to 189turn off this validation, knowing this is really insecure and what you 190are doing, you can use :option:`--insecure` option to disable 191certificate validation. 192 193Read/write rate limit 194--------------------- 195 196nghttpx supports transfer rate limiting on frontend connections. You 197can do rate limit per frontend connection for reading and writing 198individually. 199 200To perform rate limit for reading, use :option:`--read-rate` and 201:option:`--read-burst` options. For writing, use 202:option:`--write-rate` and :option:`--write-burst`. 203 204Please note that rate limit is performed on top of TCP and nothing to 205do with HTTP/2 flow control. 206 207Rewriting location header field 208------------------------------- 209 210nghttpx automatically rewrites location response header field if the 211following all conditions satisfy: 212 213* In the default mode (:option:`--http2-proxy` is not used) 214* :option:`--no-location-rewrite` is not used 215* URI in location header field is an absolute URI 216* URI in location header field includes non empty host component. 217* host (without port) in URI in location header field must match the 218 host appearing in ``:authority`` or ``host`` header field. 219 220When rewrite happens, URI scheme is replaced with the ones used in 221frontend, and authority is replaced with which appears in 222``:authority``, or ``host`` request header field. ``:authority`` 223header field has precedence over ``host``. 224 225Hot swapping 226------------ 227 228nghttpx supports hot swapping using signals. The hot swapping in 229nghttpx is multi step process. First send USR2 signal to nghttpx 230process. It will do fork and execute new executable, using same 231command-line arguments and environment variables. 232 233As of nghttpx version 1.20.0, that is all you have to do. The new 234main process sends QUIT signal to the original process, when it is 235ready to serve requests, to shut it down gracefully. 236 237For earlier versions of nghttpx, you have to do one more thing. At 238this point, both current and new processes can accept requests. To 239gracefully shutdown current process, send QUIT signal to current 240nghttpx process. When all existing frontend connections are done, the 241current process will exit. At this point, only new nghttpx process 242exists and serves incoming requests. 243 244If you want to just reload configuration file without executing new 245binary, send SIGHUP to nghttpx main process. 246 247Re-opening log files 248-------------------- 249 250When rotating log files, it is desirable to re-open log files after 251log rotation daemon renamed existing log files. To tell nghttpx to 252re-open log files, send USR1 signal to nghttpx process. It will 253re-open files specified by :option:`--accesslog-file` and 254:option:`--errorlog-file` options. 255 256Multiple frontend addresses 257--------------------------- 258 259nghttpx can listen on multiple frontend addresses. To specify them, 260just use :option:`--frontend` (or its shorthand :option:`-f`) option 261repeatedly. TLS can be enabled or disabled per frontend address 262basis. For example, to listen on port 443 with TLS enabled, and on 263port 80 without TLS: 264 265.. code-block:: text 266 267 frontend=*,443 268 frontend=*,80;no-tls 269 270 271Multiple backend addresses 272-------------------------- 273 274nghttpx supports multiple backend addresses. To specify them, just 275use :option:`--backend` (or its shorthand :option:`-b`) option 276repeatedly. For example, to use ``192.168.0.10:8080`` and 277``192.168.0.11:8080``, use command-line like this: 278``-b192.168.0.10,8080 -b192.168.0.11,8080``. In configuration file, 279this looks like: 280 281.. code-block:: text 282 283 backend=192.168.0.10,8080 284 backend=192.168.0.11,8008 285 286nghttpx can route request to different backend according to request 287host and path. For example, to route request destined to host 288``doc.example.com`` to backend server ``docserv:3000``, you can write 289like so: 290 291.. code-block:: text 292 293 backend=docserv,3000;doc.example.com/ 294 295When you write this option in command-line, you should enclose 296argument with single or double quotes, since the character ``;`` has a 297special meaning in shell. 298 299To route, request to request path ``/foo`` to backend server 300``[::1]:8080``, you can write like so: 301 302.. code-block:: text 303 304 backend=::1,8080;/foo 305 306If the last character of path pattern is ``/``, all request paths 307which start with that pattern match: 308 309.. code-block:: text 310 311 backend=::1,8080;/bar/ 312 313The request path ``/bar/buzz`` matches the ``/bar/``. 314 315You can use ``*`` at the end of the path pattern to make it wildcard 316pattern. ``*`` must match at least one character: 317 318.. code-block:: text 319 320 backend=::1,8080;/sample* 321 322The request path ``/sample1/foo`` matches the ``/sample*`` pattern. 323 324Of course, you can specify both host and request path at the same 325time: 326 327.. code-block:: text 328 329 backend=192.168.0.10,8080;example.com/foo 330 331We can use ``*`` in the left most position of host to achieve wildcard 332suffix match. If ``*`` is the left most character, then the remaining 333string should match the request host suffix. ``*`` must match at 334least one character. For example, ``*.example.com`` matches 335``www.example.com`` and ``dev.example.com``, and does not match 336``example.com`` and ``nghttp2.org``. The exact match (without ``*``) 337always takes precedence over wildcard match. 338 339One important thing you have to remember is that we have to specify 340default routing pattern for so called "catch all" pattern. To write 341"catch all" pattern, just specify backend server address, without 342pattern. 343 344Usually, host is the value of ``Host`` header field. In HTTP/2, the 345value of ``:authority`` pseudo header field is used. 346 347When you write multiple backend addresses sharing the same routing 348pattern, they are used as load balancing. For example, to use 2 349servers ``serv1:3000`` and ``serv2:3000`` for request host 350``example.com`` and path ``/myservice``, you can write like so: 351 352.. code-block:: text 353 354 backend=serv1,3000;example.com/myservice 355 backend=serv2,3000;example.com/myservice 356 357You can also specify backend application protocol in 358:option:`--backend` option using ``proto`` keyword after pattern. 359Utilizing this allows ngttpx to route certain request to HTTP/2, other 360requests to HTTP/1. For example, to route requests to ``/ws/`` in 361backend HTTP/1.1 connection, and use backend HTTP/2 for other 362requests, do this: 363 364.. code-block:: text 365 366 backend=serv1,3000;/;proto=h2 367 backend=serv1,3000;/ws/;proto=http/1.1 368 369The default backend protocol is HTTP/1.1. 370 371TLS can be enabled per pattern basis: 372 373.. code-block:: text 374 375 backend=serv1,8443;/;proto=h2;tls 376 backend=serv2,8080;/ws/;proto=http/1.1 377 378In the above case, connection to serv1 will be encrypted by TLS. On 379the other hand, connection to serv2 will not be encrypted by TLS. 380 381Dynamic hostname lookup 382----------------------- 383 384By default, nghttpx performs backend hostname lookup at start up, or 385configuration reload, and keeps using them in its entire session. To 386make nghttpx perform hostname lookup dynamically, use ``dns`` 387parameter in :option:`--backend` option, like so: 388 389.. code-block:: text 390 391 backend=foo.example.com,80;;dns 392 393nghttpx will cache resolved addresses for certain period of time. To 394change this cache period, use :option:`--dns-cache-timeout`. 395 396Enable PROXY protocol 397--------------------- 398 399PROXY protocol can be enabled per frontend. In order to enable PROXY 400protocol, use ``proxyproto`` parameter in :option:`--frontend` option, 401like so: 402 403.. code-block:: text 404 405 frontend=*,443;proxyproto 406 407nghttpx supports both PROXY protocol v1 and v2. AF_UNIX in PROXY 408protocol version 2 is ignored. 409 410Session affinity 411---------------- 412 413Two kinds of session affinity are available: client IP, and HTTP 414Cookie. 415 416To enable client IP based affinity, specify ``affinity=ip`` parameter 417in :option:`--backend` option. If PROXY protocol is enabled, then an 418address obtained from PROXY protocol is taken into consideration. 419 420To enable HTTP Cookie based affinity, specify ``affinity=cookie`` 421parameter, and specify a name of cookie in ``affinity-cookie-name`` 422parameter. Optionally, a Path attribute can be specified in 423``affinity-cookie-path`` parameter: 424 425.. code-block:: text 426 427 backend=127.0.0.1,3000;;affinity=cookie;affinity-cookie-name=nghttpxlb;affinity-cookie-path=/ 428 429Secure attribute of cookie is set if client connection is protected by 430TLS. ``affinity-cookie-stickiness`` specifies the stickiness of this 431affinity. If ``loose`` is given, which is the default, removing or 432adding a backend server might break affinity. While ``strict`` is 433given, removing the designated backend server breaks affinity, but 434adding new backend server does not cause breakage. 435 436PSK cipher suites 437----------------- 438 439nghttpx supports pre-shared key (PSK) cipher suites for both frontend 440and backend TLS connections. For frontend connection, use 441:option:`--psk-secrets` option to specify a file which contains PSK 442identity and secrets. The format of the file is 443``<identity>:<hex-secret>``, where ``<identity>`` is PSK identity, and 444``<hex-secret>`` is PSK secret in hex, like so: 445 446.. code-block:: text 447 448 client1:9567800e065e078085c241d54a01c6c3f24b3bab71a606600f4c6ad2c134f3b9 449 client2:b1376c3f8f6dcf7c886c5bdcceecd1e6f1d708622b6ddd21bda26ebd0c0bca99 450 451nghttpx server accepts any of the identity and secret pairs in the 452file. The default cipher suite list does not contain PSK cipher 453suites. In order to use PSK, PSK cipher suite must be enabled by 454using :option:`--ciphers` option. The desired PSK cipher suite may be 455listed in `HTTP/2 cipher block list 456<https://tools.ietf.org/html/rfc7540#appendix-A>`_. In order to use 457such PSK cipher suite with HTTP/2, disable HTTP/2 cipher block list by 458using :option:`--no-http2-cipher-block-list` option. But you should 459understand its implications. 460 461At the time of writing, even if only PSK cipher suites are specified 462in :option:`--ciphers` option, certificate and private key are still 463required. 464 465For backend connection, use :option:`--client-psk-secrets` option to 466specify a file which contains single PSK identity and secret. The 467format is the same as the file used by :option:`--psk-secrets` 468described above, but only first identity and secret pair is solely 469used, like so: 470 471.. code-block:: text 472 473 client2:b1376c3f8f6dcf7c886c5bdcceecd1e6f1d708622b6ddd21bda26ebd0c0bca99 474 475The default cipher suite list does not contain PSK cipher suites. In 476order to use PSK, PSK cipher suite must be enabled by using 477:option:`--client-ciphers` option. The desired PSK cipher suite may 478be listed in `HTTP/2 cipher block list 479<https://tools.ietf.org/html/rfc7540#appendix-A>`_. In order to use 480such PSK cipher suite with HTTP/2, disable HTTP/2 cipher block list by 481using :option:`--client-no-http2-cipher-block-list` option. But you 482should understand its implications. 483 484TLSv1.3 485------- 486 487As of nghttpx v1.34.0, if it is built with OpenSSL 1.1.1 or later, it 488supports TLSv1.3. 0-RTT data is supported, but by default its 489processing is postponed until TLS handshake completes to mitigate 490replay attack. This costs extra round trip and reduces effectiveness 491of 0-RTT data. :option:`--tls-no-postpone-early-data` makes nghttpx 492not wait for handshake to complete before forwarding request included 493in 0-RTT to get full potential of 0-RTT data. In this case, nghttpx 494adds ``Early-Data: 1`` header field when forwarding a request to a 495backend server. All backend servers should recognize this header 496field and understand that there is a risk for replay attack. See `RFC 4978470 <https://tools.ietf.org/html/rfc8470>`_ for ``Early-Data`` header 498field. 499 500nghttpx disables anti replay protection provided by OpenSSL. The anti 501replay protection of OpenSSL requires that a resumed request must hit 502the same server which generates the session ticket. Therefore it 503might not work nicely in a deployment where there are multiple nghttpx 504instances sharing ticket encryption keys via memcached. 505 506Because TLSv1.3 completely changes the semantics of cipher suite 507naming scheme and structure, nghttpx provides the new option 508:option:`--tls13-ciphers` and :option:`--tls13-client-ciphers` to 509change preferred cipher list for TLSv1.3. 510 511WebSockets over HTTP/2 512---------------------- 513 514nghttpx supports `RFC 8441 <https://tools.ietf.org/html/rfc8441>`_ 515Bootstrapping WebSockets with HTTP/2 for both frontend and backend 516connections. This feature is enabled by default and no configuration 517is required. 518 519WebSockets over HTTP/3 is also supported. 520 521HTTP/3 522------ 523 524nghttpx supports HTTP/3 if it is built with HTTP/3 support enabled. 525HTTP/3 support is experimental. 526 527In order to listen UDP port to receive HTTP/3 traffic, 528:option:`--frontend` option must have ``quic`` parameter: 529 530.. code-block:: text 531 532 frontend=*,443;quic 533 534The above example makes nghttpx receive HTTP/3 traffic on UDP 535port 443. 536 537nghttpx does not support HTTP/3 on backend connection. 538 539Hot swapping (SIGUSR2) or configuration reload (SIGHUP) require eBPF 540program. Without eBPF, old worker processes keep getting HTTP/3 541traffic and do not work as intended. The QUIC keying material to 542encrypt Connection ID must be set with 543:option:`--frontend-quic-secret-file` and must provide the existing 544keys in order to keep the existing connections alive during reload. 545 546The construction of Connection ID closely follows Block Cipher CID 547Algorithm described in `QUIC-LB draft 548<https://datatracker.ietf.org/doc/html/draft-ietf-quic-load-balancers>`_. 549A Connection ID that nghttpx generates is always 20 bytes long. It 550uses first 2 bits as a configuration ID. The remaining bits in the 551first byte are reserved and random. The next 4 bytes are server ID. 552The next 4 bytes are used to route UDP datagram to a correct 553``SO_REUSEPORT`` socket. The remaining bytes are randomly generated. 554The server ID and the next 12 bytes are encrypted with AES-ECB. The 555key is derived from the keying materials stored in a file specified by 556:option:`--frontend-quic-secret-file`. The first 2 bits of keying 557material in the file is used as a configuration ID. The remaining 558bits and following 3 bytes are reserved and unused. The next 32 bytes 559are used as an initial secret. The remaining 32 bytes are used as a 560salt. The encryption key is generated by `HKDF 561<https://datatracker.ietf.org/doc/html/rfc5869>`_ with SHA256 and 562these keying materials and ``connection id encryption key`` as info. 563 564In order announce that HTTP/3 endpoint is available, you should 565specify alt-svc header field. For example, the following options send 566alt-svc header field in HTTP/1.1 and HTTP/2 response: 567 568.. code-block:: text 569 570 altsvc=h3,443,,,ma=3600 571 http2-altsvc=h3,443,,,ma=3600 572 573Migration from nghttpx v1.18.x or earlier 574----------------------------------------- 575 576As of nghttpx v1.19.0, :option:`--ciphers` option only changes cipher 577list for frontend TLS connection. In order to change cipher list for 578backend connection, use :option:`--client-ciphers` option. 579 580Similarly, :option:`--no-http2-cipher-block-list` option only disables 581HTTP/2 cipher block list for frontend connection. In order to disable 582HTTP/2 cipher block list for backend connection, use 583:option:`--client-no-http2-cipher-block-list` option. 584 585``--accept-proxy-protocol`` option was deprecated. Instead, use 586``proxyproto`` parameter in :option:`--frontend` option to enable 587PROXY protocol support per frontend. 588 589Migration from nghttpx v1.8.0 or earlier 590---------------------------------------- 591 592As of nghttpx 1.9.0, ``--frontend-no-tls`` and ``--backend-no-tls`` 593have been removed. 594 595To disable encryption on frontend connection, use ``no-tls`` keyword 596in :option:`--frontend` potion: 597 598.. code-block:: text 599 600 frontend=*,3000;no-tls 601 602The TLS encryption is now disabled on backend connection in all modes 603by default. To enable encryption on backend connection, use ``tls`` 604keyword in :option:`--backend` option: 605 606.. code-block:: text 607 608 backend=127.0.0.1,8080;tls 609 610As of nghttpx 1.9.0, ``--http2-bridge``, ``--client`` and 611``--client-proxy`` options have been removed. These functionality can 612be used using combinations of options. 613 614Use following option instead of ``--http2-bridge``: 615 616.. code-block:: text 617 618 backend=<ADDR>,<PORT>;;proto=h2;tls 619 620Use following options instead of ``--client``: 621 622.. code-block:: text 623 624 frontend=<ADDR>,<PORT>;no-tls 625 backend=<ADDR>,<PORT>;;proto=h2;tls 626 627Use following options instead of ``--client-proxy``: 628 629.. code-block:: text 630 631 http2-proxy=yes 632 frontend=<ADDR>,<PORT>;no-tls 633 backend=<ADDR>,<PORT>;;proto=h2;tls 634 635We also removed ``--backend-http2-connections-per-worker`` option. It 636was present because previously the number of backend h2 connection was 637statically configured, and defaulted to 1. Now the number of backend 638h2 connection is increased on demand. We know the maximum number of 639concurrent streams per connection. When we push as many request as 640the maximum concurrency to the one connection, we create another new 641connection so that we can distribute load and avoid delay the request 642processing. This is done automatically without any configuration. 643