Artifactspublic-v0

curl

master
ply4 git calls359 ms301 KiB2 cachedprotocol v2
998 lines35 KiBRaw

curl tutorial

Simple Usage

Get the main page from a web-server:

curl https://www.example.com/

Get a README file from an FTP server:

curl ftp://ftp.example.com/README

Get a webpage from a server using port 8000:

curl http://www.example.com:8000/

Get a directory listing of an FTP site:

curl ftp://ftp.example.com/

Get all terms matching curl from a dictionary:

curl dict://dict.example.com/m:curl

Get the definition of curl from a dictionary:

curl dict://dict.example.com/d:curl

Fetch two documents at once:

curl ftp://ftp.example.com/ https://www.example.com:8000/

Get a file off an FTPS server:

curl ftps://files.are.example.com/secrets.txt

or use the more appropriate FTPS way to get the same file:

curl --ssl-reqd ftp://files.are.example.com/secrets.txt

Get a file from an SSH server using SFTP:

curl -u username sftp://example.com/etc/issue

Get a file from an SSH server using SCP using a private key (not password-protected) to authenticate:

curl -u username: --key ~/.ssh/id_rsa scp://example.com/~/file.txt

Get a file from an SSH server using SCP using a private key (password-protected) to authenticate:

curl -u username: --key ~/.ssh/id_rsa --pass private_key_password
  scp://example.com/~/file.txt

Get the main page from an IPv6 web server:

curl "http://[2001:1890:1112:1::20]/"

Download to a File

Get a webpage and store in a local file with a specific name:

curl -o thatpage.html https://www.example.com/

Get a webpage and store in a local file, make the local file get the name of the remote document (if no filename part is specified in the URL, this fails):

curl -O https://www.example.com/index.html

Fetch two files and store them with their remote names:

curl -O www.haxx.se/index.html -O curl.se/download.html

Using Passwords

FTP

To ftp files using name and password, include them in the URL like:

curl ftp://name:passwd@ftp.server.example:port/full/path/to/file

or specify them with the -u flag like

curl -u name:passwd ftp://ftp.server.example:port/full/path/to/file

FTPS

It is like FTP, but you may also want to specify and use SSL-specific options for certificates etc.

Note that using ftps:// as prefix is the implicit way as described in the standards while the recommended explicit way is done by using ftp:// and the --ssl-reqd option.

SFTP / SCP

This is similar to FTP, but you can use the --key option to specify a private key to use instead of a password. Note that the private key may itself be protected by a password that is unrelated to the login password of the remote system; this password is specified using the --pass option. Typically, curl automatically extracts the public key from the private key file, but in cases where curl does not have the proper library support, a matching public key file must be specified using the --pubkey option.

HTTP

curl also supports user and password in HTTP(S) URLs. You can download a file like:

curl https://name:passwd@http.server.example/full/path/to/file

or specify user and password separately like in

curl -u name:passwd https://http.server.example/full/path/to/file

HTTP offers many different methods of authentication and curl supports several: Basic, Digest and Negotiate (SPNEGO). Without telling which method to use, curl defaults to Basic. You can also ask curl to pick the most secure ones out of the ones that the server accepts for the given URL, by using --anyauth.

Note! According to the URL specification, HTTP URLs can not contain a user and password, so that style does not work when using curl via a proxy, even though curl allows it at other times. When using a proxy, you must use the -u style for user and password.

HTTPS

Probably most commonly used with private certificates, as explained below.

Proxy

curl supports both HTTP and SOCKS proxy servers, with optional authentication. It does not have special support for FTP proxy servers since there are no standards for those, but it can still be made to work with many of them. You can also use both HTTP and SOCKS proxies to transfer files to and from FTP servers.

Get an ftp file using an HTTP proxy named my-proxy that uses port 888:

curl -x my-proxy:888 ftp://ftp.example.com/README

Get a file from an HTTPS server that requires user and password, using the same proxy as above:

curl -u user:passwd -x my-proxy:888 https://www.example.com/

Some proxies require special authentication. Specify by using -U as above:

curl -U user:passwd -x my-proxy:888 https://www.example.com/

A comma-separated list of hosts and domains which do not use the proxy can be specified as:

curl --noproxy example.com -x my-proxy:888 https://www.example.com/

If the proxy is specified with --proxy1.0 instead of --proxy or -x, then curl uses HTTP/1.0 instead of HTTP/1.1 for any CONNECT attempts.

curl also supports SOCKS4 and SOCKS5 proxies with --socks4 and --socks5.

See also the environment variables curl supports that offer further proxy control.

Most FTP proxy servers are set up to appear as a normal FTP server from the client's perspective, with special commands to select the remote FTP server. curl supports the -u, -Q and --ftp-account options that can be used to set up transfers through many FTP proxies. For example, a file can be uploaded to a remote FTP server using a Blue Coat FTP proxy with the options:

curl -u "username@ftp.server.example Proxy-Username:Remote-Pass"
  --ftp-account Proxy-Password --upload-file local-file
  ftp://my-ftp.proxy.example:21/remote/upload/path/

See the manual for your FTP proxy to determine the form it expects to set up transfers, and curl's -v option to see exactly what curl is sending.

Piping

Get a key file and install it as a trusted one (when on a system that uses apt for package management):

curl -L https://apt.example.org/llvm-snapshot.gpg.key | sudo tee
  /etc/apt/trusted.gpg.d/llvm-snapshot.asc >/dev/null

The '|' pipes the output to stdin. tee reads from stdin.

Ranges

HTTP 1.1 introduced byte-ranges. Using this, a client can request to get only one or more sub-parts of a specified document. curl supports this with the -r flag.

Get the first 100 bytes of a document:

curl -r 0-99 https://www.example.com/

Get the last 500 bytes of a document:

curl -r -500 https://www.example.com/

curl also supports simple ranges for FTP files as well. Then you can only specify start and stop position.

Get the first 100 bytes of a document using FTP:

curl -r 0-99 ftp://www.example.com/README

Uploading

FTP / FTPS / SFTP / SCP

Upload all data on stdin to a specified server:

curl -T - ftp://ftp.example.com/myfile

Upload data from a specified file, login with user and password:

curl -T uploadfile -u user:passwd ftp://ftp.example.com/myfile

Upload a local file to the remote site, and use the local filename at the remote site too:

curl -T uploadfile -u user:passwd ftp://ftp.example.com/

Upload a local file to get appended to the remote file:

curl -T localfile -a ftp://ftp.example.com/remotefile

curl also supports ftp upload through a proxy, but only if the proxy is configured to allow that kind of tunneling. If it does, you can run curl in a fashion similar to:

curl --proxytunnel -x proxy:port -T localfile ftp.example.com

HTTP

Upload all data on stdin to a specified HTTPS site:

curl -T - https://www.example.com/myfile

Note that the HTTP server must have been configured to accept PUT before this can be done successfully.

For other ways to do HTTP data upload, see the POST section below.

Verbose / Debug

If curl fails where it is not supposed to, if the servers do not let you in, if you cannot understand the responses: use the -v flag to get verbose fetching. curl outputs lots of info and what it sends and receives in order to let the user see all client-server interaction (but it does not show you the actual data).

curl -v ftp://ftp.example.com/

To get even more details and information on what curl does, try using the --trace or --trace-ascii options with a given filename to log to, like this:

curl --trace my-trace.txt www.haxx.se

Detailed Information

Different protocols provide different ways of getting detailed information about specific files/documents. To get curl to show detailed information about a single file, you should use -I/--head option. It displays all available info on a single file for HTTP and FTP. The HTTP information is a lot more extensive.

For HTTP, you can get the header information (the same as -I would show) shown before the data by using -i/--include. curl understands the -D/--dump-header option when getting files from both FTP and HTTP, and it then stores the headers in the specified file.

Store the HTTP headers in a separate file (headers.txt in the example):

  curl --dump-header headers.txt curl.se

Note that headers stored in a separate file can be useful at a later time if you want curl to use cookies sent by the server. More about that in the cookies section.

POST (HTTP)

It is easy to post data using curl. This is done using the -d <data> option. The post data must be urlencoded.

Post a simple name and phone guestbook.

curl -d "name=Rafael%20Sagula&phone=3320780" https://www.example.com/guest.cgi

Or automatically URL encode the data.

curl --data-urlencode "name=Rafael Sagula&phone=3320780"
  https://www.example.com/guest.cgi

How to post a form with curl, lesson #1:

Dig out all the <input> tags in the form that you want to fill in.

If there is a normal post, you use -d to post. -d takes a full post string, which is in the format

<variable1>=<data1>&<variable2>=<data2>&...

The variable names are the names set with "name=" in the <input> tags, and the data is the contents you want to fill in for the inputs. The data must be properly URL encoded. That means you replace space with + and that you replace weird letters with %XX where XX is the hexadecimal representation of the letter's ASCII code.

Example:

(say if https://example.com had the following html)

html
<form action="post.cgi" method="post">
  <input name=user size=10>
  <input name=pass type=password size=10>
  <input name=id type=hidden value="blablabla">
  <input name=ding value="submit">
</form>

We want to enter user foobar with password 12345.

To post to this, you would enter a curl command line like:

curl -d "user=foobar&pass=12345&id=blablabla&ding=submit"
  https://example.com/post.cgi

While -d uses the application/x-www-form-urlencoded mime-type, generally understood by CGI's and similar, curl also supports the more capable multipart/form-data type. This latter type supports things like file upload.

-F accepts parameters like -F "name=contents". If you want the contents to be read from a file, use @filename as contents. When specifying a file, you can also specify the file content type by appending ;type=<mime type> to the filename. You can also post the contents of several files in one field. For example, the field name coolfiles is used to send three files, with different content types using the following syntax:

curl -F "coolfiles=@fil1.gif;type=image/gif,fil2.txt,fil3.html"
  https://www.example.com/postit.cgi

If the content-type is not specified, curl tries to guess from the file extension (it only knows a few), or use the previously specified type (from an earlier file if several files are specified in a list) or else it uses the default type application/octet-stream.

Emulate a fill-in form with -F. Let's say you fill in three fields in a form. One field is a filename which to post, one field is your name and one field is a file description. We want to post the file we have written named cooltext.txt. To let curl do the posting of this data instead of your favorite browser, you have to read the HTML source of the form page and find the names of the input fields. In our example, the input field names are file, yourname and filedescription.

curl -F "file=@cooltext.txt" -F "yourname=Daniel"
  -F "filedescription=Cool text file with cool text inside"
  https://www.example.com/postit.cgi

To send two files in one post you can do it in two ways:

Send multiple files in a single field with a single field name:

curl -F "pictures=@dog.gif,cat.gif" $URL

Send two fields with two field names

curl -F "docpicture=@dog.gif" -F "catpicture=@cat.gif" $URL

To send a field value literally without interpreting a leading @ or <, or an embedded ;type=, use --form-string instead of -F. This is recommended when the value is obtained from a user or some other unpredictable source. Under these circumstances, using -F instead of --form-string could allow a user to trick curl into uploading a file.

Referrer

An HTTP request has the option to include information about which address referred it to the actual page. curl allows you to specify the referrer to be used on the command line. It is especially useful to fool or trick stupid servers or CGI scripts that rely on that information being available or contain certain data.

curl -e www.example.org https://www.example.com/

User Agent

An HTTP request has the option to include information about the browser that generated the request. curl allows it to be specified on the command line. It is especially useful to fool or trick stupid servers or CGI scripts that only accept certain browsers.

Example:

curl -A 'Mozilla/3.0 (Win95; I)' https://www.bank.example.com/

Other common strings:

  • Mozilla/3.0 (Win95; I) - Netscape Version 3 for Windows 95
  • Mozilla/3.04 (Win95; U) - Netscape Version 3 for Windows 95
  • Mozilla/2.02 (OS/2; U) - Netscape Version 2 for OS/2
  • Mozilla/4.04 [en] (X11; U; AIX 4.2; Nav) - Netscape for AIX
  • Mozilla/4.05 [en] (X11; U; Linux 2.0.32 i586) - Netscape for Linux

Note that Internet Explorer tries hard to be compatible in every way:

  • Mozilla/4.0 (compatible; MSIE 4.01; Windows 95) - MSIE for W95

Mozilla is not the only possible User-Agent name:

  • Konqueror/1.0 - KDE File Manager desktop client
  • Lynx/2.7.1 libwww-FM/2.14 - Lynx command line browser

Cookies

Cookies are generally used by web servers to keep state information at the client's side. The server sets cookies by sending a response line in the headers that looks like Set-Cookie: <data> where the data part then typically contains a set of NAME=VALUE pairs (separated by semicolons ; like NAME1=VALUE1; NAME2=VALUE2;). The server can also specify for what path the cookie should be used for (by specifying path=value), when the cookie should expire (expire=DATE), for what domain to use it (domain=NAME) and if it should be used on secure connections only (secure).

If you have received a page from a server that contains a header like:

http
Set-Cookie: sessionid=boo123; path="/foo";

it means the server wants that first pair passed on when we get anything in a path beginning with /foo.

Example, get a page that wants my name passed in a cookie:

curl -b "name=Daniel" www.example.com

curl also has the ability to use previously received cookies in following sessions. If you get cookies from a server and store them in a file in a manner similar to:

curl --dump-header headers www.example.com

... you can then in a second connect to that (or another) site, use the cookies from the headers.txt file like:

curl -b headers.txt www.example.com

While saving headers to a file is a working way to store cookies, it is however error-prone and not the preferred way to do this. Instead, make curl save the incoming cookies using the well-known Netscape cookie format like this:

curl -c cookies.txt www.example.com

Note that by specifying -b you enable the cookie engine and with -L you can make curl follow a location: (which often is used in combination with cookies). If a site sends cookies and a location field, you can use a non-existing file to trigger the cookie awareness like:

curl -L -b empty.txt www.example.com

The file to read cookies from must be formatted using plain HTTP headers OR as Netscape's cookie file. curl determines what kind it is based on the file contents. In the above command, curl parses the header and store the cookies received from www.example.com. curl sends the stored cookies which match the request to the server as it follows the location. The file empty.txt may be a nonexistent file.

To read and write cookies from a Netscape cookie file, you can set both -b and -c to use the same file:

curl -b cookies.txt -c cookies.txt www.example.com

Progress Meter

The progress meter exists to show a user that something actually is happening. The different fields in the output have the following meaning:

% Total    % Received % Xferd  Average Speed          Time             Curr.
                               Dload  Upload Total    Current  Left    Speed
0  151M    0 38608    0     0   9406      0  4:41:43  0:00:04  4:41:39  9287

From left-to-right:

  • % - percentage completed of the whole transfer
  • Total - total size of the whole expected transfer
  • % - percentage completed of the download
  • Received - currently downloaded amount of bytes
  • % - percentage completed of the upload
  • Xferd - currently uploaded amount of bytes
  • Average Speed Dload - the average transfer speed of the download
  • Average Speed Upload - the average transfer speed of the upload
  • Time Total - expected time to complete the operation
  • Time Current - time passed since the invoke
  • Time Left - expected time left to completion
  • Curr.Speed - the average transfer speed the last 5 seconds (the first 5 seconds of a transfer is based on less time of course.)

The -# option displays a totally different progress bar that does not need much explanation!

Speed Limit

curl allows the user to set the transfer speed conditions that must be met to let the transfer keep going. By using the switch -y and -Y you can make curl abort transfers if the transfer speed is below the specified lowest limit for a specified time.

To have curl abort the download if the speed is slower than 3000 bytes per second for 1 minute, run:

curl -Y 3000 -y 60 www.far-away.example.com

This can be used in combination with the overall time limit, so that the above operation must be completed in whole within 30 minutes:

curl -m 1800 -Y 3000 -y 60 www.far-away.example.com

Forcing curl not to transfer data faster than a given rate is also possible, which might be useful if you are using a limited bandwidth connection and you do not want your transfer to use all of it (sometimes referred to as bandwidth throttle).

Make curl transfer data no faster than 10 kilobytes per second:

curl --limit-rate 10K www.far-away.example.com

or

curl --limit-rate 10240 www.far-away.example.com

Or prevent curl from uploading data faster than 1 megabyte per second:

curl -T upload --limit-rate 1M ftp://uploads.example.com

When using the --limit-rate option, the transfer rate is regulated on a per-second basis, which causes the total transfer speed to become lower than the given number. Sometimes of course substantially lower, if your transfer stalls during periods.

Config File

curl automatically tries to read the .curlrc file (or _curlrc file on Microsoft Windows systems) from the user's home directory on startup.

The config file could be made up with normal command line switches, but you can also specify the long options without the dashes to make it more readable. You can separate the options and the parameter with spaces, or with = or :. Comments can be used within the file. If the first letter on a line is a #-symbol the rest of the line is treated as a comment.

If you want the parameter to contain spaces, you must enclose the entire parameter within double quotes ("). Within those quotes, you specify a quote as \".

NOTE: You must specify options and their arguments on the same line.

Example, set default time out and proxy in a config file:

# We want a 30 minute timeout:
-m 1800
# ... and we use a proxy for all accesses:
proxy = proxy.our.domain.example.com:8080

Whitespaces ARE significant at the end of lines, but all whitespace leading up to the first characters of each line are ignored.

Prevent curl from reading the default file by using -q as the first command line parameter, like:

curl -q www.example.org

Force curl to get and display a local help page in case it is invoked without URL by making a config file similar to:

# default URL to get
url = "https://help.with.curl.example.com/curlhelp.html"

You can specify another config file to be read by using the -K/--config flag. If you set config filename to - it reads the config from stdin, which can be handy if you want to hide options from being visible in process tables etc:

echo "user = user:passwd" | curl -K - https://that.secret.example.com

Extra Headers

When using curl in your own programs, you may end up needing to pass on your own custom headers when getting a webpage. You can do this by using the -H flag.

Example, send the header X-you-and-me: yes to the server when getting a page:

curl -H "X-you-and-me: yes" love.example.com

This can also be useful in case you want curl to send a different text in a header than it normally does. The -H header you specify then replaces the header curl would normally send. If you replace an internal header with an empty one, you prevent that header from being sent. To prevent the Host: header from being used:

curl -H "Host:" server.example.com

FTP and Path Names

Do note that when getting files with a ftp:// URL, the given path is relative to the directory you enter. To get the file README from your home directory at your ftp site, do:

curl ftp://user:passwd@my.example.com/README

If you want the README file from the root directory of that same site, you need to specify the absolute filename:

curl ftp://user:passwd@my.example.com//README

(I.e with an extra slash in front of the filename.)

SFTP and SCP and Path Names

With sftp: and scp: URLs, the path name given is the absolute name on the server. To access a file relative to the remote user's home directory, prefix the file with /~/ , such as:

curl -u $USER sftp://home.example.com/~/.bashrc

FTP and Firewalls

The FTP protocol requires one of the involved parties to open a second connection as soon as data is about to get transferred. There are two ways to do this.

The default way for curl is to issue the PASV command which causes the server to open another port and await another connection performed by the client. This is good if the client is behind a firewall that does not allow incoming connections.

curl ftp.example.com

If the server, for example, is behind a firewall that does not allow connections on ports other than 21 (or if it does not support the PASV command), the other way to do it is to use the PORT command and instruct the server to connect to the client on the given IP number and port (as parameters to the PORT command).

The -P flag to curl supports a few different options. Your machine may have several IP-addresses and/or network interfaces and curl allows you to select which of them to use. Default address can also be used:

curl -P - ftp.example.com

Download with PORT but use the IP address of our le0 interface (this does not work on Windows):

curl -P le0 ftp.example.com

Download with PORT but use 192.168.0.10 as our IP address to use:

curl -P 192.168.0.10 ftp.example.com

Network Interface

Get a webpage from a server using a specified port for the interface:

curl --interface eth0:1 https://www.example.com/

or

curl --interface 192.168.1.10 https://www.example.com/

HTTPS

Secure HTTP requires a TLS library to be installed and used when curl is built. If that is done, curl is capable of retrieving and posting documents using the HTTPS protocol.

Example:

curl https://secure.example.com

curl is also capable of using client certificates to get/post files from sites that require valid certificates. The only drawback is that the certificate needs to be in PEM-format. PEM is a standard and open format to store certificates with, but it is not used by the most commonly used browsers. If you want curl to use the certificates you use with your favorite browser, you may need to download/compile a converter that can convert your browser's formatted certificates to PEM formatted ones.

Example on how to automatically retrieve a document using a certificate with a personal password:

curl -E /path/to/cert.pem:password https://secure.example.com/

If you neglect to specify the password on the command line, you are prompted for the correct password before any data can be received.

Many older HTTPS servers have problems with specific SSL or TLS versions, which newer versions of OpenSSL etc use, therefore it is sometimes useful to specify what TLS version curl should use.:

curl --tlv1.0 https://secure.example.com/

Otherwise, curl attempts to use a sensible TLS default version.

Resuming File Transfers

To continue a file transfer where it was previously aborted, curl supports resume on HTTP(S) downloads as well as FTP uploads and downloads.

Continue downloading a document:

curl -C - -o file ftp://ftp.example.com/path/file

Continue uploading a document:

curl -C - -T file ftp://ftp.example.com/path/file

Continue downloading a document from a web server

curl -C - -o file https://www.example.com/

Time Conditions

HTTP allows a client to specify a time condition for the document it requests. It is If-Modified-Since or If-Unmodified-Since. curl allows you to specify them with the -z/--time-cond flag.

For example, you can easily make a download that only gets performed if the remote file is newer than a local copy. It would be made like:

curl -z local.html https://remote.example.com/remote.html

Or you can download a file only if the local file is newer than the remote one. Do this by prepending the date string with a -, as in:

curl -z -local.html https://remote.example.com/remote.html

You can specify a plain text date as condition. Tell curl to only download the file if it was updated since January 12, 2012:

curl -z "Jan 12 2012" https://remote.example.com/remote.html

curl accepts a wide range of date formats. You always make the date check the other way around by prepending it with a dash (-).

DICT

For fun try

curl dict://dict.org/m:curl
curl dict://dict.org/d:heisenbug:jargon
curl dict://dict.org/d:daniel:gcide

Aliases for m are match and find, and aliases for d are define and lookup. For example,

curl dict://dict.org/find:curl

Commands that break the URL description of the RFC (but not the DICT protocol) are

curl dict://dict.org/show:db
curl dict://dict.org/show:strat

Authentication support is still missing

LDAP

If you have installed the OpenLDAP library, curl can take advantage of it and offer ldap:// support. On Windows, curl uses WinLDAP from Platform SDK by default.

Default protocol version used by curl is LDAP version 3. Version 2 is used as a fallback mechanism in case version 3 fails to connect.

LDAP is a complex thing and writing an LDAP query is not an easy task. Familiarize yourself with the exact syntax description elsewhere. One such place might be: RFC 2255, The LDAP URL Format

To show you an example, this is how to get all people from an LDAP server that has a certain subdomain in their email address:

curl -B "ldap://ldap.example.com/o=frontec??sub?mail=*sth.example.com"

You also can use authentication when accessing LDAP catalog:

curl -u user:passwd "ldap://ldap.example.com/o=frontec??sub?mail=*"
curl "ldap://user:passwd@ldap.example.com/o=frontec??sub?mail=*"

By default, if user and password are provided, OpenLDAP/WinLDAP uses basic authentication. On Windows you can control this behavior by providing one of --basic, --ntlm or --digest option in curl command line

curl --ntlm "ldap://user:passwd@ldap.example.com/o=frontec??sub?mail=*"

On Windows, if no user/password specified, auto-negotiation mechanism is used with current logon credentials (SSPI/SPNEGO).

Environment Variables

curl reads and understands the following proxy related environment variables:

http_proxy, HTTPS_PROXY, FTP_PROXY

They should be set for protocol-specific proxies. General proxy should be set with

ALL_PROXY

A comma-separated list of hostnames that should not go through any proxy is set in (only an asterisk, * matches all hosts)

NO_PROXY

If the hostname matches one of these strings, or the host is within the domain of one of these strings, transactions with that node is not done over the proxy. When a domain is used, it needs to start with a period. A user can specify that both www.example.com and foo.example.com should not use a proxy by setting NO_PROXY to .example.com. By including the full name you can exclude specific hostnames, so to make www.example.com not use a proxy but still have foo.example.com do it, set NO_PROXY to www.example.com.

The usage of the -x/--proxy flag overrides the environment variables.

Netrc

Unix introduced the .netrc concept a long time ago. It is a way for a user to specify name and password for commonly visited FTP sites in a file so that you do not have to type them in each time you visit those sites. You realize this is a big security risk if someone else gets hold of your passwords, therefore most Unix programs do not read this file unless it is only readable by yourself (curl does not care though).

curl supports .netrc files if told to (using the -n/--netrc and --netrc-optional options). This is not restricted to FTP, so curl can use it for all protocols where authentication is used.

A simple .netrc file could look something like:

machine curl.se login iamdaniel password mysecret

Custom Output

To better allow script programmers to get to know about the progress of curl, the -w/--write-out option was introduced. Using this, you can specify what information from the previous transfer you want to extract.

To display the amount of bytes downloaded together with some text and an ending newline:

curl -w 'We downloaded %{size_download} bytes\n' www.example.com

Kerberos FTP Transfer

curl supports kerberos4 and kerberos5/GSSAPI for FTP transfers. You need the kerberos package installed and used at curl build time for it to be available.

First, get the krb-ticket the normal way, like with the kinit/kauth tool. Then use curl in way similar to:

curl --krb private ftp://krb4site.example.com -u username:fakepwd

There is no use for a password on the -u switch, but a blank one makes curl ask for one and you already entered the real password to kinit/kauth.

TELNET

The curl telnet support is basic and easy to use. curl passes all data passed to it on stdin to the remote server. Connect to a remote telnet server using a command line similar to:

curl telnet://remote.example.com

Enter the data to pass to the server on stdin. The result is sent to stdout or to the file you specify with -o.

You might want the -N/--no-buffer option to switch off the buffered output for slow connections or similar.

Pass options to the telnet protocol negotiation, by using the -t option. To tell the server we use a vt100 terminal, try something like:

curl -tTTYPE=vt100 telnet://remote.example.com

Other interesting options for it -t include:

  • XDISPLOC=<X display> Sets the X display location.
  • NEW_ENV=<var,val> Sets an environment variable.

NOTE: The telnet protocol does not specify any way to login with a specified user and password so curl cannot do that automatically. To do that, you need to track when the login prompt is received and send the username and password accordingly.

Persistent Connections

Specifying multiple files on a single command line makes curl transfer all of them, one after the other in the specified order.

libcurl attempts to use persistent connections for the transfers so that the second transfer to the same host can use the same connection that was already initiated and was left open in the previous transfer. This greatly decreases connection time for all but the first transfer and it makes a far better use of the network.

Note that curl cannot use persistent connections for transfers that are used in subsequent curl invokes. Try to stuff as many URLs as possible on the same command line if they are using the same host, as that makes the transfers faster. If you use an HTTP proxy for file transfers, practically all transfers are persistent.

Multiple Transfers With A Single Command Line

As is mentioned above, you can download multiple files with one command line by adding more URLs. If you want those to get saved to a local file instead of printed to stdout, you need to add one save option for each URL you specify. Note that this also goes for the -O option (but not --remote-name-all).

For example: get two files and use -O for the first and a custom file name for the second:

curl -O https://example.com/file.txt ftp://example.com/moo.exe -o moo.jpg

You can also upload multiple files in a similar fashion:

curl -T local1 ftp://example.com/moo.exe -T local2 ftp://example.com/moo2.txt

IPv6

curl connects to a server with IPv6 when a host lookup returns an IPv6 address and fall back to IPv4 if the connection fails. The --ipv4 and --ipv6 options can specify which address to use when both are available. IPv6 addresses can also be specified directly in URLs using the syntax:

https://[2001:1890:1112:1::20]/overview.html

When this style is used, the -g option must be given to stop curl from interpreting the square brackets as special globbing characters. Link local and site local addresses including a scope identifier, such as fe80::1234%1, may also be used, but the scope portion must be numeric or match an existing network interface on Linux and the percent character must be URL escaped. The previous example in an SFTP URL might look like:

sftp://[fe80::1234%251]/

IPv6 addresses provided other than in URLs (e.g. to the --proxy, --interface or --ftp-port options) should not be URL encoded.

Mailing Lists

For your convenience, we have several open mailing lists to discuss curl, its development and things relevant to this. Get all info at https://curl.se/mail/.

Please direct curl questions, feature requests and trouble reports to one of these mailing lists instead of mailing any individual.

Available lists include:

curl-users

Users of the command line tool. How to use it, what does not work, new features, related tools, questions, news, installations, compilations, running, porting etc.

curl-library

Developers using or developing libcurl. Bugs, extensions, improvements.

curl-announce

Low-traffic. Only receives announcements of new public versions. At worst, that makes something like one or two mails per month, but usually only one mail every second month.

curl-and-php

Using the curl functions in PHP. Everything curl with a PHP angle. Or PHP with a curl angle.

curl-and-python

Python hackers using curl with or without the python binding pycurl.

1<!--
2Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
4SPDX-License-Identifier: curl
5-->
7# curl tutorial
9## Simple Usage
11Get the main page from a web-server:
13 curl https://www.example.com/
15Get a README file from an FTP server:
17 curl ftp://ftp.example.com/README
19Get a webpage from a server using port 8000:
21 curl http://www.example.com:8000/
23Get a directory listing of an FTP site:
25 curl ftp://ftp.example.com/
27Get all terms matching curl from a dictionary:
29 curl dict://dict.example.com/m:curl
31Get the definition of curl from a dictionary:
33 curl dict://dict.example.com/d:curl
35Fetch two documents at once:
37 curl ftp://ftp.example.com/ https://www.example.com:8000/
39Get a file off an FTPS server:
41 curl ftps://files.are.example.com/secrets.txt
43or use the more appropriate FTPS way to get the same file:
45 curl --ssl-reqd ftp://files.are.example.com/secrets.txt
47Get a file from an SSH server using SFTP:
49 curl -u username sftp://example.com/etc/issue
51Get a file from an SSH server using SCP using a private key (not
52password-protected) to authenticate:
54 curl -u username: --key ~/.ssh/id_rsa scp://example.com/~/file.txt
56Get a file from an SSH server using SCP using a private key
57(password-protected) to authenticate:
59 curl -u username: --key ~/.ssh/id_rsa --pass private_key_password
60 scp://example.com/~/file.txt
62Get the main page from an IPv6 web server:
64 curl "http://[2001:1890:1112:1::20]/"
66## Download to a File
68Get a webpage and store in a local file with a specific name:
70 curl -o thatpage.html https://www.example.com/
72Get a webpage and store in a local file, make the local file get the name of
73the remote document (if no filename part is specified in the URL, this fails):
75 curl -O https://www.example.com/index.html
77Fetch two files and store them with their remote names:
79 curl -O www.haxx.se/index.html -O curl.se/download.html
81## Using Passwords
83### FTP
85To ftp files using name and password, include them in the URL like:
87 curl ftp://name:passwd@ftp.server.example:port/full/path/to/file
89or specify them with the `-u` flag like
91 curl -u name:passwd ftp://ftp.server.example:port/full/path/to/file
93### FTPS
95It is like FTP, but you may also want to specify and use SSL-specific options
96for certificates etc.
98Note that using `ftps://` as prefix is the *implicit* way as described in the
99standards while the recommended *explicit* way is done by using `ftp://` and
100the `--ssl-reqd` option.
102### SFTP / SCP
104This is similar to FTP, but you can use the `--key` option to specify a
105private key to use instead of a password. Note that the private key may itself
106be protected by a password that is unrelated to the login password of the
107remote system; this password is specified using the `--pass` option.
108Typically, curl automatically extracts the public key from the private key
109file, but in cases where curl does not have the proper library support, a
110matching public key file must be specified using the `--pubkey` option.
112### HTTP
114curl also supports user and password in HTTP(S) URLs. You can download a file
115like:
117 curl https://name:passwd@http.server.example/full/path/to/file
119or specify user and password separately like in
121 curl -u name:passwd https://http.server.example/full/path/to/file
123HTTP offers many different methods of authentication and curl supports
124several: Basic, Digest and Negotiate (SPNEGO). Without telling which method to
125use, curl defaults to Basic. You can also ask curl to pick the most secure
126ones out of the ones that the server accepts for the given URL, by using
127`--anyauth`.
129**Note**! According to the URL specification, HTTP URLs can not contain a user
130and password, so that style does not work when using curl via a proxy, even
131though curl allows it at other times. When using a proxy, you _must_ use the
132`-u` style for user and password.
134### HTTPS
136Probably most commonly used with private certificates, as explained below.
138## Proxy
140curl supports both HTTP and SOCKS proxy servers, with optional authentication.
141It does not have special support for FTP proxy servers since there are no
142standards for those, but it can still be made to work with many of them. You
143can also use both HTTP and SOCKS proxies to transfer files to and from FTP
144servers.
146Get an ftp file using an HTTP proxy named my-proxy that uses port 888:
148 curl -x my-proxy:888 ftp://ftp.example.com/README
150Get a file from an HTTPS server that requires user and password, using the
151same proxy as above:
153 curl -u user:passwd -x my-proxy:888 https://www.example.com/
155Some proxies require special authentication. Specify by using -U as above:
157 curl -U user:passwd -x my-proxy:888 https://www.example.com/
159A comma-separated list of hosts and domains which do not use the proxy can be
160specified as:
162 curl --noproxy example.com -x my-proxy:888 https://www.example.com/
164If the proxy is specified with `--proxy1.0` instead of `--proxy` or `-x`, then
165curl uses HTTP/1.0 instead of HTTP/1.1 for any `CONNECT` attempts.
167curl also supports SOCKS4 and SOCKS5 proxies with `--socks4` and `--socks5`.
169See also the environment variables curl supports that offer further proxy
170control.
172Most FTP proxy servers are set up to appear as a normal FTP server from the
173client's perspective, with special commands to select the remote FTP server.
174curl supports the `-u`, `-Q` and `--ftp-account` options that can be used to
175set up transfers through many FTP proxies. For example, a file can be uploaded
176to a remote FTP server using a Blue Coat FTP proxy with the options:
178 curl -u "username@ftp.server.example Proxy-Username:Remote-Pass"
179 --ftp-account Proxy-Password --upload-file local-file
180 ftp://my-ftp.proxy.example:21/remote/upload/path/
182See the manual for your FTP proxy to determine the form it expects to set up
183transfers, and curl's `-v` option to see exactly what curl is sending.
185## Piping
187Get a key file and install it as a trusted one (when on a system that uses
188`apt` for package management):
190 curl -L https://apt.example.org/llvm-snapshot.gpg.key | sudo tee
191 /etc/apt/trusted.gpg.d/llvm-snapshot.asc >/dev/null
193The '|' pipes the output to stdin. `tee` reads from stdin.
195## Ranges
197HTTP 1.1 introduced byte-ranges. Using this, a client can request to get only
198one or more sub-parts of a specified document. curl supports this with the
199`-r` flag.
201Get the first 100 bytes of a document:
203 curl -r 0-99 https://www.example.com/
205Get the last 500 bytes of a document:
207 curl -r -500 https://www.example.com/
209curl also supports simple ranges for FTP files as well. Then you can only
210specify start and stop position.
212Get the first 100 bytes of a document using FTP:
214 curl -r 0-99 ftp://www.example.com/README
216## Uploading
218### FTP / FTPS / SFTP / SCP
220Upload all data on stdin to a specified server:
222 curl -T - ftp://ftp.example.com/myfile
224Upload data from a specified file, login with user and password:
226 curl -T uploadfile -u user:passwd ftp://ftp.example.com/myfile
228Upload a local file to the remote site, and use the local filename at the
229remote site too:
231 curl -T uploadfile -u user:passwd ftp://ftp.example.com/
233Upload a local file to get appended to the remote file:
235 curl -T localfile -a ftp://ftp.example.com/remotefile
237curl also supports ftp upload through a proxy, but only if the proxy is
238configured to allow that kind of tunneling. If it does, you can run curl in a
239fashion similar to:
241 curl --proxytunnel -x proxy:port -T localfile ftp.example.com
243### HTTP
245Upload all data on stdin to a specified HTTPS site:
247 curl -T - https://www.example.com/myfile
249Note that the HTTP server must have been configured to accept PUT before this
250can be done successfully.
252For other ways to do HTTP data upload, see the POST section below.
254## Verbose / Debug
256If curl fails where it is not supposed to, if the servers do not let you in,
257if you cannot understand the responses: use the `-v` flag to get verbose
258fetching. curl outputs lots of info and what it sends and receives in order to
259let the user see all client-server interaction (but it does not show you the
260actual data).
262 curl -v ftp://ftp.example.com/
264To get even more details and information on what curl does, try using the
265`--trace` or `--trace-ascii` options with a given filename to log to, like
266this:
268 curl --trace my-trace.txt www.haxx.se
270## Detailed Information
272Different protocols provide different ways of getting detailed information
273about specific files/documents. To get curl to show detailed information about
274a single file, you should use `-I`/`--head` option. It displays all available
275info on a single file for HTTP and FTP. The HTTP information is a lot more
276extensive.
278For HTTP, you can get the header information (the same as `-I` would show)
279shown before the data by using `-i`/`--include`. curl understands the
280`-D`/`--dump-header` option when getting files from both FTP and HTTP, and it
281then stores the headers in the specified file.
283Store the HTTP headers in a separate file (headers.txt in the example):
285 curl --dump-header headers.txt curl.se
287Note that headers stored in a separate file can be useful at a later time if
288you want curl to use cookies sent by the server. More about that in the
289cookies section.
291## POST (HTTP)
293It is easy to post data using curl. This is done using the `-d <data>` option.
294The post data must be urlencoded.
296Post a simple `name` and `phone` guestbook.
298 curl -d "name=Rafael%20Sagula&phone=3320780" https://www.example.com/guest.cgi
300Or automatically [URL encode the data](https://everything.curl.dev/http/post/url-encode.html).
302 curl --data-urlencode "name=Rafael Sagula&phone=3320780"
303 https://www.example.com/guest.cgi
305How to post a form with curl, lesson #1:
307Dig out all the `<input>` tags in the form that you want to fill in.
309If there is a normal post, you use `-d` to post. `-d` takes a full post
310string, which is in the format
312 <variable1>=<data1>&<variable2>=<data2>&...
314The variable names are the names set with `"name="` in the `<input>` tags, and
315the data is the contents you want to fill in for the inputs. The data *must*
316be properly URL encoded. That means you replace space with + and that you
317replace weird letters with `%XX` where `XX` is the hexadecimal representation
318of the letter's ASCII code.
320Example:
322(say if `https://example.com` had the following html)
324```html
325<form action="post.cgi" method="post">
326 <input name=user size=10>
327 <input name=pass type=password size=10>
328 <input name=id type=hidden value="blablabla">
329 <input name=ding value="submit">
330</form>
331```
333We want to enter user `foobar` with password `12345`.
335To post to this, you would enter a curl command line like:
337 curl -d "user=foobar&pass=12345&id=blablabla&ding=submit"
338 https://example.com/post.cgi
340While `-d` uses the application/x-www-form-urlencoded mime-type, generally
341understood by CGI's and similar, curl also supports the more capable
342multipart/form-data type. This latter type supports things like file upload.
344`-F` accepts parameters like `-F "name=contents"`. If you want the contents to
345be read from a file, use `@filename` as contents. When specifying a file, you
346can also specify the file content type by appending `;type=<mime type>` to the
347filename. You can also post the contents of several files in one field. For
348example, the field name `coolfiles` is used to send three files, with
349different content types using the following syntax:
351 curl -F "coolfiles=@fil1.gif;type=image/gif,fil2.txt,fil3.html"
352 https://www.example.com/postit.cgi
354If the content-type is not specified, curl tries to guess from the file
355extension (it only knows a few), or use the previously specified type (from an
356earlier file if several files are specified in a list) or else it uses the
357default type `application/octet-stream`.
359Emulate a fill-in form with `-F`. Let's say you fill in three fields in a
360form. One field is a filename which to post, one field is your name and one
361field is a file description. We want to post the file we have written named
362`cooltext.txt`. To let curl do the posting of this data instead of your
363favorite browser, you have to read the HTML source of the form page and find
364the names of the input fields. In our example, the input field names are
365`file`, `yourname` and `filedescription`.
367 curl -F "file=@cooltext.txt" -F "yourname=Daniel"
368 -F "filedescription=Cool text file with cool text inside"
369 https://www.example.com/postit.cgi
371To send two files in one post you can do it in two ways:
373Send multiple files in a single field with a single field name:
375 curl -F "pictures=@dog.gif,cat.gif" $URL
377Send two fields with two field names
379 curl -F "docpicture=@dog.gif" -F "catpicture=@cat.gif" $URL
381To send a field value literally without interpreting a leading `@` or `<`, or
382an embedded `;type=`, use `--form-string` instead of `-F`. This is recommended
383when the value is obtained from a user or some other unpredictable
384source. Under these circumstances, using `-F` instead of `--form-string` could
385allow a user to trick curl into uploading a file.
387## Referrer
389An HTTP request has the option to include information about which address
390referred it to the actual page. curl allows you to specify the referrer to be
391used on the command line. It is especially useful to fool or trick stupid
392servers or CGI scripts that rely on that information being available or
393contain certain data.
395 curl -e www.example.org https://www.example.com/
397## User Agent
399An HTTP request has the option to include information about the browser that
400generated the request. curl allows it to be specified on the command line. It
401is especially useful to fool or trick stupid servers or CGI scripts that only
402accept certain browsers.
404Example:
406 curl -A 'Mozilla/3.0 (Win95; I)' https://www.bank.example.com/
408Other common strings:
410- `Mozilla/3.0 (Win95; I)` - Netscape Version 3 for Windows 95
411- `Mozilla/3.04 (Win95; U)` - Netscape Version 3 for Windows 95
412- `Mozilla/2.02 (OS/2; U)` - Netscape Version 2 for OS/2
413- `Mozilla/4.04 [en] (X11; U; AIX 4.2; Nav)` - Netscape for AIX
414- `Mozilla/4.05 [en] (X11; U; Linux 2.0.32 i586)` - Netscape for Linux
416Note that Internet Explorer tries hard to be compatible in every way:
418- `Mozilla/4.0 (compatible; MSIE 4.01; Windows 95)` - MSIE for W95
420Mozilla is not the only possible User-Agent name:
422- `Konqueror/1.0` - KDE File Manager desktop client
423- `Lynx/2.7.1 libwww-FM/2.14` - Lynx command line browser
425## Cookies
427Cookies are generally used by web servers to keep state information at the
428client's side. The server sets cookies by sending a response line in the
429headers that looks like `Set-Cookie: <data>` where the data part then
430typically contains a set of `NAME=VALUE` pairs (separated by semicolons `;`
431like `NAME1=VALUE1; NAME2=VALUE2;`). The server can also specify for what path
432the cookie should be used for (by specifying `path=value`), when the cookie
433should expire (`expire=DATE`), for what domain to use it (`domain=NAME`) and
434if it should be used on secure connections only (`secure`).
436If you have received a page from a server that contains a header like:
438```http
439Set-Cookie: sessionid=boo123; path="/foo";
440```
442it means the server wants that first pair passed on when we get anything in a
443path beginning with `/foo`.
445Example, get a page that wants my name passed in a cookie:
447 curl -b "name=Daniel" www.example.com
449curl also has the ability to use previously received cookies in following
450sessions. If you get cookies from a server and store them in a file in a
451manner similar to:
453 curl --dump-header headers www.example.com
455... you can then in a second connect to that (or another) site, use the
456cookies from the `headers.txt` file like:
458 curl -b headers.txt www.example.com
460While saving headers to a file is a working way to store cookies, it is
461however error-prone and not the preferred way to do this. Instead, make curl
462save the incoming cookies using the well-known Netscape cookie format like
463this:
465 curl -c cookies.txt www.example.com
467Note that by specifying `-b` you enable the cookie engine and with `-L` you
468can make curl follow a `location:` (which often is used in combination with
469cookies). If a site sends cookies and a location field, you can use a
470non-existing file to trigger the cookie awareness like:
472 curl -L -b empty.txt www.example.com
474The file to read cookies from must be formatted using plain HTTP headers OR as
475Netscape's cookie file. curl determines what kind it is based on the file
476contents. In the above command, curl parses the header and store the cookies
477received from www.example.com. curl sends the stored cookies which match the
478request to the server as it follows the location. The file `empty.txt` may be
479a nonexistent file.
481To read and write cookies from a Netscape cookie file, you can set both `-b`
482and `-c` to use the same file:
484 curl -b cookies.txt -c cookies.txt www.example.com
486## Progress Meter
488The progress meter exists to show a user that something actually is
489happening. The different fields in the output have the following meaning:
491 % Total % Received % Xferd Average Speed Time Curr.
492 Dload Upload Total Current Left Speed
493 0 151M 0 38608 0 0 9406 0 4:41:43 0:00:04 4:41:39 9287
495From left-to-right:
497- `%` - percentage completed of the whole transfer
498- `Total` - total size of the whole expected transfer
499- `%` - percentage completed of the download
500- `Received` - currently downloaded amount of bytes
501- `%` - percentage completed of the upload
502- `Xferd` - currently uploaded amount of bytes
503- `Average Speed Dload` - the average transfer speed of the download
504- `Average Speed Upload` - the average transfer speed of the upload
505- `Time Total` - expected time to complete the operation
506- `Time Current` - time passed since the invoke
507- `Time Left` - expected time left to completion
508- `Curr.Speed` - the average transfer speed the last 5 seconds (the first
509 5 seconds of a transfer is based on less time of course.)
511The `-#` option displays a totally different progress bar that does not need
512much explanation!
514## Speed Limit
516curl allows the user to set the transfer speed conditions that must be met to
517let the transfer keep going. By using the switch `-y` and `-Y` you can make
518curl abort transfers if the transfer speed is below the specified lowest limit
519for a specified time.
521To have curl abort the download if the speed is slower than 3000 bytes per
522second for 1 minute, run:
524 curl -Y 3000 -y 60 www.far-away.example.com
526This can be used in combination with the overall time limit, so that the above
527operation must be completed in whole within 30 minutes:
529 curl -m 1800 -Y 3000 -y 60 www.far-away.example.com
531Forcing curl not to transfer data faster than a given rate is also possible,
532which might be useful if you are using a limited bandwidth connection and you
533do not want your transfer to use all of it (sometimes referred to as
534*bandwidth throttle*).
536Make curl transfer data no faster than 10 kilobytes per second:
538 curl --limit-rate 10K www.far-away.example.com
540or
542 curl --limit-rate 10240 www.far-away.example.com
544Or prevent curl from uploading data faster than 1 megabyte per second:
546 curl -T upload --limit-rate 1M ftp://uploads.example.com
548When using the `--limit-rate` option, the transfer rate is regulated on a
549per-second basis, which causes the total transfer speed to become lower than
550the given number. Sometimes of course substantially lower, if your transfer
551stalls during periods.
553## Config File
555curl automatically tries to read the `.curlrc` file (or `_curlrc` file on
556Microsoft Windows systems) from the user's home directory on startup.
558The config file could be made up with normal command line switches, but you
559can also specify the long options without the dashes to make it more
560readable. You can separate the options and the parameter with spaces, or with
561`=` or `:`. Comments can be used within the file. If the first letter on a
562line is a `#`-symbol the rest of the line is treated as a comment.
564If you want the parameter to contain spaces, you must enclose the entire
565parameter within double quotes (`"`). Within those quotes, you specify a quote
566as `\"`.
568NOTE: You must specify options and their arguments on the same line.
570Example, set default time out and proxy in a config file:
572 # We want a 30 minute timeout:
573 -m 1800
574 # ... and we use a proxy for all accesses:
575 proxy = proxy.our.domain.example.com:8080
577Whitespaces ARE significant at the end of lines, but all whitespace leading
578up to the first characters of each line are ignored.
580Prevent curl from reading the default file by using -q as the first command
581line parameter, like:
583 curl -q www.example.org
585Force curl to get and display a local help page in case it is invoked without
586URL by making a config file similar to:
588 # default URL to get
589 url = "https://help.with.curl.example.com/curlhelp.html"
591You can specify another config file to be read by using the `-K`/`--config`
592flag. If you set config filename to `-` it reads the config from stdin, which
593can be handy if you want to hide options from being visible in process tables
594etc:
596 echo "user = user:passwd" | curl -K - https://that.secret.example.com
598## Extra Headers
600When using curl in your own programs, you may end up needing to pass on your
601own custom headers when getting a webpage. You can do this by using the `-H`
602flag.
604Example, send the header `X-you-and-me: yes` to the server when getting a
605page:
607 curl -H "X-you-and-me: yes" love.example.com
609This can also be useful in case you want curl to send a different text in a
610header than it normally does. The `-H` header you specify then replaces the
611header curl would normally send. If you replace an internal header with an
612empty one, you prevent that header from being sent. To prevent the `Host:`
613header from being used:
615 curl -H "Host:" server.example.com
617## FTP and Path Names
619Do note that when getting files with a `ftp://` URL, the given path is
620relative to the directory you enter. To get the file `README` from your home
621directory at your ftp site, do:
623 curl ftp://user:passwd@my.example.com/README
625If you want the README file from the root directory of that same site, you
626need to specify the absolute filename:
628 curl ftp://user:passwd@my.example.com//README
630(I.e with an extra slash in front of the filename.)
632## SFTP and SCP and Path Names
634With sftp: and scp: URLs, the path name given is the absolute name on the
635server. To access a file relative to the remote user's home directory, prefix
636the file with `/~/` , such as:
638 curl -u $USER sftp://home.example.com/~/.bashrc
640## FTP and Firewalls
642The FTP protocol requires one of the involved parties to open a second
643connection as soon as data is about to get transferred. There are two ways to
644do this.
646The default way for curl is to issue the PASV command which causes the server
647to open another port and await another connection performed by the
648client. This is good if the client is behind a firewall that does not allow
649incoming connections.
651 curl ftp.example.com
653If the server, for example, is behind a firewall that does not allow
654connections on ports other than 21 (or if it does not support the `PASV`
655command), the other way to do it is to use the `PORT` command and instruct the
656server to connect to the client on the given IP number and port (as parameters
657to the PORT command).
659The `-P` flag to curl supports a few different options. Your machine may have
660several IP-addresses and/or network interfaces and curl allows you to select
661which of them to use. Default address can also be used:
663 curl -P - ftp.example.com
665Download with `PORT` but use the IP address of our `le0` interface (this does
666not work on Windows):
668 curl -P le0 ftp.example.com
670Download with `PORT` but use 192.168.0.10 as our IP address to use:
672 curl -P 192.168.0.10 ftp.example.com
674## Network Interface
676Get a webpage from a server using a specified port for the interface:
678 curl --interface eth0:1 https://www.example.com/
680or
682 curl --interface 192.168.1.10 https://www.example.com/
684## HTTPS
686Secure HTTP requires a TLS library to be installed and used when curl is
687built. If that is done, curl is capable of retrieving and posting documents
688using the HTTPS protocol.
690Example:
692 curl https://secure.example.com
694curl is also capable of using client certificates to get/post files from sites
695that require valid certificates. The only drawback is that the certificate
696needs to be in PEM-format. PEM is a standard and open format to store
697certificates with, but it is not used by the most commonly used browsers. If
698you want curl to use the certificates you use with your favorite browser, you
699may need to download/compile a converter that can convert your browser's
700formatted certificates to PEM formatted ones.
702Example on how to automatically retrieve a document using a certificate with a
703personal password:
705 curl -E /path/to/cert.pem:password https://secure.example.com/
707If you neglect to specify the password on the command line, you are prompted
708for the correct password before any data can be received.
710Many older HTTPS servers have problems with specific SSL or TLS versions,
711which newer versions of OpenSSL etc use, therefore it is sometimes useful to
712specify what TLS version curl should use.:
714 curl --tlv1.0 https://secure.example.com/
716Otherwise, curl attempts to use a sensible TLS default version.
718## Resuming File Transfers
720To continue a file transfer where it was previously aborted, curl supports
721resume on HTTP(S) downloads as well as FTP uploads and downloads.
723Continue downloading a document:
725 curl -C - -o file ftp://ftp.example.com/path/file
727Continue uploading a document:
729 curl -C - -T file ftp://ftp.example.com/path/file
731Continue downloading a document from a web server
733 curl -C - -o file https://www.example.com/
735## Time Conditions
737HTTP allows a client to specify a time condition for the document it requests.
738It is `If-Modified-Since` or `If-Unmodified-Since`. curl allows you to specify
739them with the `-z`/`--time-cond` flag.
741For example, you can easily make a download that only gets performed if the
742remote file is newer than a local copy. It would be made like:
744 curl -z local.html https://remote.example.com/remote.html
746Or you can download a file only if the local file is newer than the remote
747one. Do this by prepending the date string with a `-`, as in:
749 curl -z -local.html https://remote.example.com/remote.html
751You can specify a plain text date as condition. Tell curl to only download the
752file if it was updated since January 12, 2012:
754 curl -z "Jan 12 2012" https://remote.example.com/remote.html
756curl accepts a wide range of date formats. You always make the date check the
757other way around by prepending it with a dash (`-`).
759## DICT
761For fun try
763 curl dict://dict.org/m:curl
764 curl dict://dict.org/d:heisenbug:jargon
765 curl dict://dict.org/d:daniel:gcide
767Aliases for `m` are `match` and `find`, and aliases for `d` are `define` and
768`lookup`. For example,
770 curl dict://dict.org/find:curl
772Commands that break the URL description of the RFC (but not the DICT
773protocol) are
775 curl dict://dict.org/show:db
776 curl dict://dict.org/show:strat
778Authentication support is still missing
780## LDAP
782If you have installed the OpenLDAP library, curl can take advantage of it and
783offer `ldap://` support. On Windows, curl uses WinLDAP from Platform SDK by
784default.
786Default protocol version used by curl is LDAP version 3. Version 2 is used as
787a fallback mechanism in case version 3 fails to connect.
789LDAP is a complex thing and writing an LDAP query is not an easy
790task. Familiarize yourself with the exact syntax description elsewhere. One
791such place might be: [RFC 2255, The LDAP URL
792Format](https://curl.se/rfc/rfc2255.txt)
794To show you an example, this is how to get all people from an LDAP server that
795has a certain subdomain in their email address:
797 curl -B "ldap://ldap.example.com/o=frontec??sub?mail=*sth.example.com"
799You also can use authentication when accessing LDAP catalog:
801 curl -u user:passwd "ldap://ldap.example.com/o=frontec??sub?mail=*"
802 curl "ldap://user:passwd@ldap.example.com/o=frontec??sub?mail=*"
804By default, if user and password are provided, OpenLDAP/WinLDAP uses basic
805authentication. On Windows you can control this behavior by providing one of
806`--basic`, `--ntlm` or `--digest` option in curl command line
808 curl --ntlm "ldap://user:passwd@ldap.example.com/o=frontec??sub?mail=*"
810On Windows, if no user/password specified, auto-negotiation mechanism is used
811with current logon credentials (SSPI/SPNEGO).
813## Environment Variables
815curl reads and understands the following proxy related environment variables:
817 http_proxy, HTTPS_PROXY, FTP_PROXY
819They should be set for protocol-specific proxies. General proxy should be set
820with
822 ALL_PROXY
824A comma-separated list of hostnames that should not go through any proxy is
825set in (only an asterisk, `*` matches all hosts)
827 NO_PROXY
829If the hostname matches one of these strings, or the host is within the domain
830of one of these strings, transactions with that node is not done over the
831proxy. When a domain is used, it needs to start with a period. A user can
832specify that both www.example.com and foo.example.com should not use a proxy
833by setting `NO_PROXY` to `.example.com`. By including the full name you can
834exclude specific hostnames, so to make `www.example.com` not use a proxy but
835still have `foo.example.com` do it, set `NO_PROXY` to `www.example.com`.
837The usage of the `-x`/`--proxy` flag overrides the environment variables.
839## Netrc
841Unix introduced the `.netrc` concept a long time ago. It is a way for a user
842to specify name and password for commonly visited FTP sites in a file so that
843you do not have to type them in each time you visit those sites. You realize
844this is a big security risk if someone else gets hold of your passwords,
845therefore most Unix programs do not read this file unless it is only readable
846by yourself (curl does not care though).
848curl supports `.netrc` files if told to (using the `-n`/`--netrc` and
849`--netrc-optional` options). This is not restricted to FTP, so curl can use it
850for all protocols where authentication is used.
852A simple `.netrc` file could look something like:
854 machine curl.se login iamdaniel password mysecret
856## Custom Output
858To better allow script programmers to get to know about the progress of curl,
859the `-w`/`--write-out` option was introduced. Using this, you can specify what
860information from the previous transfer you want to extract.
862To display the amount of bytes downloaded together with some text and an
863ending newline:
865 curl -w 'We downloaded %{size_download} bytes\n' www.example.com
867## Kerberos FTP Transfer
869curl supports kerberos4 and kerberos5/GSSAPI for FTP transfers. You need the
870kerberos package installed and used at curl build time for it to be available.
872First, get the krb-ticket the normal way, like with the `kinit`/`kauth` tool.
873Then use curl in way similar to:
875 curl --krb private ftp://krb4site.example.com -u username:fakepwd
877There is no use for a password on the `-u` switch, but a blank one makes curl
878ask for one and you already entered the real password to `kinit`/`kauth`.
880## TELNET
882The curl telnet support is basic and easy to use. curl passes all data passed
883to it on stdin to the remote server. Connect to a remote telnet server using a
884command line similar to:
886 curl telnet://remote.example.com
888Enter the data to pass to the server on stdin. The result is sent to stdout or
889to the file you specify with `-o`.
891You might want the `-N`/`--no-buffer` option to switch off the buffered output
892for slow connections or similar.
894Pass options to the telnet protocol negotiation, by using the `-t` option. To
895tell the server we use a vt100 terminal, try something like:
897 curl -tTTYPE=vt100 telnet://remote.example.com
899Other interesting options for it `-t` include:
901- `XDISPLOC=<X display>` Sets the X display location.
902- `NEW_ENV=<var,val>` Sets an environment variable.
904NOTE: The telnet protocol does not specify any way to login with a specified
905user and password so curl cannot do that automatically. To do that, you need to
906track when the login prompt is received and send the username and password
907accordingly.
909## Persistent Connections
911Specifying multiple files on a single command line makes curl transfer all of
912them, one after the other in the specified order.
914libcurl attempts to use persistent connections for the transfers so that the
915second transfer to the same host can use the same connection that was already
916initiated and was left open in the previous transfer. This greatly decreases
917connection time for all but the first transfer and it makes a far better use
918of the network.
920Note that curl cannot use persistent connections for transfers that are used
921in subsequent curl invokes. Try to stuff as many URLs as possible on the same
922command line if they are using the same host, as that makes the transfers
923faster. If you use an HTTP proxy for file transfers, practically all transfers
924are persistent.
926## Multiple Transfers With A Single Command Line
928As is mentioned above, you can download multiple files with one command line
929by adding more URLs. If you want those to get saved to a local file
930instead of printed to stdout, you need to add one save option for each URL you
931specify. Note that this also goes for the `-O` option (but not
932`--remote-name-all`).
934For example: get two files and use `-O` for the first and a custom file
935name for the second:
937 curl -O https://example.com/file.txt ftp://example.com/moo.exe -o moo.jpg
939You can also upload multiple files in a similar fashion:
941 curl -T local1 ftp://example.com/moo.exe -T local2 ftp://example.com/moo2.txt
943## IPv6
945curl connects to a server with IPv6 when a host lookup returns an IPv6 address
946and fall back to IPv4 if the connection fails. The `--ipv4` and `--ipv6`
947options can specify which address to use when both are available. IPv6
948addresses can also be specified directly in URLs using the syntax:
950 https://[2001:1890:1112:1::20]/overview.html
952When this style is used, the `-g` option must be given to stop curl from
953interpreting the square brackets as special globbing characters. Link local
954and site local addresses including a scope identifier, such as `fe80::1234%1`,
955may also be used, but the scope portion must be numeric or match an existing
956network interface on Linux and the percent character must be URL escaped. The
957previous example in an SFTP URL might look like:
959 sftp://[fe80::1234%251]/
961IPv6 addresses provided other than in URLs (e.g. to the `--proxy`,
962`--interface` or `--ftp-port` options) should not be URL encoded.
964## Mailing Lists
966For your convenience, we have several open mailing lists to discuss curl, its
967development and things relevant to this. Get all info at
968https://curl.se/mail/.
970Please direct curl questions, feature requests and trouble reports to one of
971these mailing lists instead of mailing any individual.
973Available lists include:
975### `curl-users`
977Users of the command line tool. How to use it, what does not work, new
978features, related tools, questions, news, installations, compilations,
979running, porting etc.
981### `curl-library`
983Developers using or developing libcurl. Bugs, extensions, improvements.
985### `curl-announce`
987Low-traffic. Only receives announcements of new public versions. At worst,
988that makes something like one or two mails per month, but usually only one
989mail every second month.
991### `curl-and-php`
993Using the curl functions in PHP. Everything curl with a PHP angle. Or PHP with
994a curl angle.
996### `curl-and-python`
998Python hackers using curl with or without the python binding pycurl.
Diagnostics4 git calls · 359 ms
CallStatusSizeTime0359 ms
discover200152 B3 ms
git.upload-pack discover
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=3
git.response.bytes=152
git.server.agent=gitty/1.0
git.server.object_format=sha1
git.server.caps=agent ls-refs fetch object-format
git.server.fetch=shallow filter sideband-all
git.protocol.version=2
git.pkt.frames=8
git.pkt.data=6
git.pkt.flush=2
git.pkt.delim=0
git.pkt.bytes=152
git.pkt.payload.max=34
ls-refs200149 B · hit3 ms
git.upload-pack ls-refs
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=3
git.response.bytes=149
git.response.cache=HIT
git.response.duration_ms=3
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=149
git.pkt.payload.max=78
git.ls_refs.count=2
git.ls_refs.head=refs/heads/master
fetchblob:none · deepen 10200287 KiB · hit3 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=353
git.response.bytes=293470
git.response.cache=HIT
git.response.duration_ms=3
git.fetch.filter=blob:none
git.fetch.filter.requested=tree:0
git.fetch.filter.fallback=blob:none
git.fetch.filter.learned=false
git.fetch.wants=31c911efe5652725e930f4d2e60c9060e5f6646d
git.fetch.haves.count=0
git.fetch.deepen=10
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=10
git.pkt.data=8
git.pkt.flush=1
git.pkt.delim=1
git.pkt.bytes=293470
git.pkt.payload.max=65516
git.sideband.band1.bytes=293354
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=87
git.pack.bytes=293354
git.pack.ofs_delta=0
git.pack.ref_delta=1
git.pack.chain.max=1
git.pack.trailer=6a646bcda6740bab42e321b51998b52ea15146c0
git.pack.full=86
Open pack map
fetch20014 KiB353 ms
git.upload-pack fetch
http.response.status_code=200
git.response.bytes=14231
git.response.cache=MISS
git.response.duration_ms=353
git.fetch.wants=c6082dda3d2ade33df68d296469d0ee94096e494
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=14231
git.pkt.payload.max=14210
git.sideband.band1.bytes=14209
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0

Each row is one HTTP round trip ply made to the git server for this page, on one timeline. discover asks what the server speaks, ls-refs lists branch tips, fetch brings objects by id. A hit was served by Workers Cache. Tap a row for its wire details.

Remote
https://5ad2236b39de4c8bd8f6270ea63f3ea9.artifacts.cloudflare.net/git/public-v0/curl.git
Ref
refs/heads/master
Commit
31c911efe5652725e930f4d2e60c9060e5f6646d
cf-ray
a5e170c639a96102
Request
e716dec7…
Protocol
v2 · sha1
Agent
gitty/1.0
Filter
advertised
Fetch
shallow filter sideband-all
Capabilities
agent ls-refs fetch object-format