Lines Matching full:to
7 The increasing amount of applications moving to the web has made "HTTP
8 Scripting" more frequently requested and wanted. To be able to automatically
9 extract information from the web, to fake users, to post or upload data to
13 transfers, but this particular document will focus on how to use it when
15 how to invoke `curl --help` or `curl --manual` to get basic information about
18 Curl is not written to do everything for you. It makes the requests, it gets
20 to glue everything together using some kind of script language or repeated
25 HTTP is the protocol used to fetch data from web servers. It is a simple
26 protocol that is built upon TCP/IP. The protocol also allows information to
27 get sent to the server from the client using a few different methods, as will
30 HTTP is plain ASCII text lines being sent by the client to a server to
32 before the actual requested content is sent to the client.
43 (`-v` as a short option) will display what kind of commands curl sends to the
46 `--verbose` is the single most useful option when it comes to debug or even
60 want to know the amount of milliseconds between two points in a transfer. For
63 is what you need. It will prepend the time to each trace output line:
69 When doing parallel transfers, it is relevant to see which transfer is
70 doing what. When response headers are received (and logged) you need to
74 to each trace output line:
80 By default curl sends the response to stdout. You need to redirect it
81 somewhere to avoid that, most often that is done with `-o` or `-O`.
94 The hostname is usually resolved using DNS or your /etc/hosts file to an IP
98 For development and other trying out situations, you can point to a different
107 or in some cases UDP. Normally you do not have to take that into
110 number immediately following the hostname. Like when doing HTTP to port
115 The port number you specify in the URL is the number that the server uses to
117 need to specify that proxy's port number separately from what curl needs to
118 connect to the server. Like when using an HTTP proxy on port 4321:
124 Some services are setup to require HTTP authentication and then you need to
125 provide name and password which is then transferred to the remote site in
128 You can opt to either insert the user and password in the URL or you can
137 You need to pay attention that this kind of HTTP authentication is not what
139 to use forms and cookies instead.
143 The path part is just sent off to the server to request that it sends back
144 the associated response. The path is what is to the right side of the slash
151 The simplest and most common request/operation made using HTTP is to GET a
152 URL. The URL could itself refer to a webpage, an image or a file. The client
153 issues a GET request to the server and receives the document it asked for.
163 option to display them as well as the rest of the document.
180 is probably to just use one, but you can specify any amount of URLs. Yes
188 If you use [`--data`](https://curl.se/docs/manpage.html#-d) to POST to
189 the URL, using multiple URLs means that you send that same POST to all the
199 Sometimes you need to operate on several URLs in a single command line and do
209 Perhaps this is best shown with a few examples. To send first a HEAD and then
214 To first send a POST and then a GET:
223 the user to enter data in, and then press some kind of 'OK' or 'Submit'
224 button to get that data sent to the server. The server then typically uses
225 the posted data to decide how to act. Like using the entered words to search
226 in a database, or to add the info in a bug tracking system, display the
228 the user is allowed to see what it is about to see.
230 Of course there has to be some kind of program on the server end to receive
244 In your favorite browser, this form will appear with a text box to fill in
246 button, your browser will then create a new URL to get for you. The URL will
247 get `junk.cgi?birthyear=1905&press=OK` appended to the path part of the
256 To make curl do the GET form post for you, just enter the expected created
264 your browser. That is generally a good thing when you want to be able to
273 The form would look similar to the previous one:
282 And to use curl to post this form with the same data filled in as before, we
290 The data you send to the server MUST already be properly encoded, curl will
291 not do that for you. For example, if you want the data to contain a space,
292 you need to replace that space with `%20`, etc. Failing to comply with this will
293 most likely cause your data to be received wrongly and messed up.
305 Back in late 1995 they defined an additional way to post data over HTTP. It
306 is documented in the RFC 1867, why this method sometimes is referred to as
309 This method is mainly designed to better support file uploads. A form that
310 allows a user to upload a file could be written like this in HTML:
317 This clearly shows that the Content-Type about to be sent is
320 To post to a form like this with curl, you enter a command line like:
326 A common way for HTML based applications to pass state information between
327 pages is to add hidden fields to the forms. Hidden fields are already filled
328 in, they are not displayed to the user and they get passed along just as all
342 To POST this with curl, you will not have to think about if the fields are
343 hidden or not. To curl they are all the same:
349 When you are about to fill in a form and send it to a server by using curl
353 An easy way to get to see this, is to save the HTML page with the form on
354 your local disk, modify the 'method' to a GET, and press the submit button
355 (you could also change the action URL if you want to).
357 You will then clearly see the data get appended to the URL, separated with a
358 `?`-letter as GET forms are supposed to.
364 Perhaps the best way to upload data to an HTTP server is to use PUT. Then
366 server end that knows how to receive an HTTP PUT stream.
368 Put a file to an HTTP server with curl:
376 HTTP Authentication is the ability to tell the server your username and
377 password so that it can verify that you are allowed to do the request you are
383 To tell curl to use a user and password for authentication:
400 proxy. This seems to be especially common at various companies. An HTTP proxy
401 may require its own user and password to allow the client to get through to
402 the Internet. To specify those with curl, run something like:
406 If your proxy requires the authentication to be done using the NTLM method,
416 Do note that when a program is run, its parameters might be possible to see
418 able to watch your passwords if you pass them as plain command line
419 options. There are ways to circumvent this.
430 can be used to tell from which URL the client got to this particular
431 resource. Some programs/scripts check the referer field of requests to verify
433 this is a stupid way to check something so easily forged, many scripts still
435 thus more easily be able to fool the server into serving your request.
437 Use curl to set the referer field with:
443 Similar to the referer field, all HTTP requests may set the User-Agent
445 applications use this information to decide how to display pages. Silly web
446 programmers try to make different pages for users of different browsers to
452 is time to set the User Agent field to fool the server into thinking you are
455 To make curl look like Internet Explorer 5 on a Windows 2000 box:
468 include a hint about where the browser should go next to find this page, or a
469 new page keeping newly generated output. The header that tells the browser to
474 feature an option that will make it attempt to follow the `Location:`
477 To tell curl to follow a Location:
481 If you use curl to POST to a site that immediately redirects you to another
485 then revert to GET in the following operations.
491 to load a specific URL after a set number of seconds, or it may use
492 JavaScript to do it.
500 sent to the client by the server. The server tells the client for what path
506 contents to the server, unless of course they are expired.
508 Many applications and servers use this method to connect a series of requests
509 into a single logical session. To be able to use curl in such occasions, we
510 must be able to record and send back cookies the way the web application
515 The simplest way to send a few cookies to the server when getting a page with
516 curl is to add them on the command line like:
521 to record cookies simply by recording headers. Record cookies with curl by
529 below is a better way to store cookies.)
532 want to reconnect to a server and use cookies that were stored from a
533 previous connection (or hand-crafted manually to fool the server into
534 believing you had a previous connection). To use previously stored cookies,
541 want curl to understand received cookies, use `--cookie` with a file that
542 does not exist. Example, if you want to let curl understand cookies from a
548 Curl has the ability to read and write cookie files that use the same file
549 format that Netscape and Mozilla once used. It is a convenient way to share
562 There are a few ways to do secure HTTP transfers. By far the most common
565 thus makes it harder for attackers to spy on sensitive information.
568 advanced features to do secure transfers over HTTP.
570 Curl supports encrypted fetches when built to use a TLS library and it can be
571 built to use one out of a fairly large set of libraries - `curl -V` will show
572 which one your curl was built to use (if any!). To get a page from an HTTPS
579 In the HTTPS world, you use certificates to validate that you are the one
580 you claim to be, as an addition to normal passwords. Curl supports client-
582 need to enter before the certificate can be used by curl. The pass phrase
588 curl also tries to verify that the server is who it claims to be, by
590 bundle. Failing the verification will cause curl to deny the connection. You
592 (`-k`) in case you want to tell curl to ignore that the server cannot be
599 curl to use that to verify the server's certificate:
607 Doing fancy stuff, you may need to add or change elements of a single curl
610 For example, you can change the POST method to `PROPFIND` and send the data
629 It should be noted that curl selects which methods to use on its own
630 depending on what action to ask for. `-d` will do POST, `-I` will do HEAD and
633 modify curl's behavior. This means that if you for example use -d "data" to
634 do a POST, you can modify the method to a `PROPFIND` with `-X` and curl will
635 still think it sends a POST. You can change the normal GET to a POST method
649 login forms work and how to login to them using curl.
651 It can also be noted that to do this properly in an automated fashion, you
652 will most certainly need to script things and do multiple curl invokes etc.
654 First, servers mostly use cookies to track the logged-in status of the
655 client, so you will need to capture the cookies you receive in the
656 responses. Then, many sites also set a special cookie on the login page (to
658 of first getting the login-form page to capture the cookies set there.
661 sometimes they use such code to set or modify cookie contents. Possibly they
662 do that to prevent programmed logins, like this manual describes how to...
663 Anyway, if reading the code is not enough to let you repeat the behavior
665 sent cookies is usually a working method to work out how to shortcut the
670 to first capture the HTML code for the login form and extract all the hidden
671 fields to be able to do a proper login POST. Remember that the contents need
672 to be URL encoded when sent in a normal POST.
679 seem to respond the same way to your curl requests as it does to your
682 Then you need to start making your curl requests more similar to your
685 - Use the `--trace-ascii` option to store fully detailed logs of the requests
691 - Set user-agent (with [`-A`](https://curl.se/docs/manpage.html#-A)) to
702 A good helper to make sure you do this right, is the web browsers' developers
706 A more raw approach is to capture the HTTP traffic on the network with tools
708 received by the browser. (HTTPS forces you to use `SSLKEYLOGFILE` to do