Artifactspublic-v0

curl

master
ply4 git calls26 ms298 KiB3 cachedprotocol v2
829 lines30 KiBRaw

curl test suite file format

The curl test suite's file format is simple and extendable, closely resembling XML. All data for a single test case resides in a single ASCII file. Labels mark the beginning and the end of all sections, and each label must be written in its own line. Comments are either XML-style (enclosed with <!-- and -->) or shell script style (beginning with #) and must appear on their own lines and not alongside actual test data. Test data files are syntactically valid XML; lack of support for character entities is a big difference but macros like %CR fill that particular role here.

Each test case source exists as a file matching the format tests/data/testNUM, where NUM is the unique test number, and must begin with a testcase tag, which encompasses the remainder of the file.

Preprocessing

When a test is to be executed, the source file is first preprocessed and variables are substituted by their respective contents and the output version of the test file is stored as %LOGDIR/testNUM. That version is what is read and used by the test servers.

Base64 Encoding

In the preprocess stage, a special instruction can be used to have runtests.pl base64 encode a certain section and insert in the generated output file. This is in particular good for test cases where the test tool is expected to pass in base64 encoded content that might use dynamic information that is unique for this particular test invocation, like the server port number.

To insert a base64 encoded string into the output, use this syntax:

%b64[ data to encode ]b64%

The data to encode can then use any of the existing variables mentioned below, or even percent-encoded individual bytes. As an example, insert the HTTP server's port number (in ASCII) followed by a space and the hexadecimal byte 9a:

%b64[%HTTPPORT %9a]b64%

Hexadecimal decoding

In the preprocess stage, a special instruction can be used to have runtests.pl generate a sequence of binary bytes.

To insert a sequence of bytes from a hex encoded string, use this syntax:

%hex[ %XX-encoded data to decode ]hex%

Other bytes within the brackets that not percent encoded are inserted as-is.

For example, to insert the binary octets 0, 1 and 255 into the test file:

%hex[%00%01%FF]hex%

Repeat content

In the preprocess stage, a special instruction can be used to have runtests.pl generate a repetitive sequence of bytes.

To insert a sequence of repeat bytes, use this syntax to make the <string> get repeated <number> of times. The number has to be 1 or larger and the string may contain %HH hexadecimal codes:

%repeat[<number> x <string>]%

For example, to insert the word hello 100 times:

%repeat[100 x hello]%

Whitespace

To force CRLF newline, add this macro to the end of the line:

%CR  - carriage return

To add significant whitespace characters at the end of the line, or to empty lines:

%SP  - space
%TAB - horizontal tab

Special characters

Macros to help keep data files XML-compliant:

%AMP - Ampersand: `&`
%GT  - Greater-than sign: `>`
%LT  - Less-than sign: `<`

Insert capped epoch days

Mostly to test capped cookie expire dates: %days[NUM] inserts the number of seconds for the given number of days into the future, aligned to the nearest minute. That is the same calculation the cookie engine uses to cap expiration dates.

Include file

This instruction allows a test case to include another file. It is helpful to remember that the ordinary variables are expanded before the include happens so %LOGDIR and the others can be used in the include line.

The filename cannot contain % as that letter is used to end the name for the include instruction:

%include filename%

Or, a variant of the above where the file is loaded as a newline-agnostic text file, and whitespace, special character macros and variables expanded after inclusion:

%includetext filename%

Conditional lines

Lines in the test file can be made to appear conditionally on a specific feature (see the "features" section below) being set or not set. If the specific feature is present, the following lines are output, otherwise it outputs nothing, until a following else or endif clause. Like this:

%if brotli
Accept-Encoding
%endif

It can also check for the inverse condition, so if the feature is not set by the use of an exclamation mark:

%if !brotli
Accept-Encoding: not-brotli
%endif

You can also make an "else" clause to get output for the opposite condition, like:

%if brotli
Accept-Encoding: brotli
%else
Accept-Encoding: nothing
%endif

Nested conditions are supported.

Variables

When the test is preprocessed, a range of "variables" in the test file is replaced by their content at that time.

Available substitute variables include:

  • %CLIENT6IP - IPv6 address of the client running curl (including brackets)
  • %CLIENT6IP-NB - IPv6 address of the client running curl (no brackets)
  • %CLIENTIP - IPv4 address of the client running curl
  • %CURL - Path to the curl executable
  • %DATE - current YYYY-MM-DD date
  • %DEV_NULL - Null device (e.g. /dev/null)
  • %FILE_PWD - Current directory, on Windows prefixed with a slash
  • %FTP6PORT - IPv6 port number of the FTP server
  • %FTPPORT - Port number of the FTP server
  • %FTPSPORT - Port number of the FTPS server
  • %FTPTIME2 - Timeout in seconds that should be sufficient to receive a response from the test FTP server
  • %GOPHER6PORT - IPv6 port number of the Gopher server
  • %GOPHERPORT - Port number of the Gopher server
  • %GOPHERSPORT - Port number of the Gophers server
  • %HOST6IP - IPv6 address of the host running this test
  • %HOSTIP - IPv4 address of the host running this test
  • %HTTP2PORT - Port number of the HTTP/2 server
  • %HTTP6PORT - IPv6 port number of the HTTP server
  • %HTTPPORT - Port number of the HTTP server
  • %HTTPSPORT - Port number of the HTTPS server
  • %HTTPSPROXYPORT - Port number of the HTTPS-proxy
  • %HTTPTLS6PORT - IPv6 port number of the HTTP TLS server
  • %HTTPTLSPORT - Port number of the HTTP TLS server
  • %HTTPUNIXPATH - Path to the Unix socket of the HTTP server
  • %IMAP6PORT - IPv6 port number of the IMAP server
  • %IMAPPORT - Port number of the IMAP server
  • %LOGDIR - Log directory relative to %PWD
  • %MQTTPORT - Port number of the MQTT server
  • %MQTTSPORT - Port number of the MQTTS server
  • %NOLISTENPORT - Port number where no service is listening
  • %POP36PORT - IPv6 port number of the POP3 server
  • %POP3PORT - Port number of the POP3 server
  • %PROXYPORT - Port number of the HTTP proxy
  • %PWD - Current directory
  • %RESOLVE - server/resolve command
  • %RTSP6PORT - IPv6 port number of the RTSP server
  • %RTSPPORT - Port number of the RTSP server
  • %SMTP6PORT - IPv6 port number of the SMTP server
  • %SMTPPORT - Port number of the SMTP server
  • %SOCKSPORT - Port number of the SOCKS4/5 server
  • %SOCKSUNIXPATH - Path to the Unix socket of the SOCKS server
  • %SRCDIR - Full path to the source dir
  • %SCP_PWD - Current directory friendly for the SSH server for the scp:// protocol
  • %SFTP_PWD - Current directory friendly for the SSH server for the sftp:// protocol
  • %SSHKEYALGO - SSH host and client key algorithm, e.g. ssh-rsa or ssh-ed25519
  • %SSHPORT - Port number of the SCP/SFTP server
  • %SSHSRVMD5 - MD5 of SSH server's public key
  • %SSHSRVSHA256 - SHA256 of SSH server's public key
  • %TELNETPORT - Port number of the telnet server
  • %TESTNUMBER - Number of the test case
  • %TFTP6PORT - IPv6 port number of the TFTP server
  • %TFTPPORT - Port number of the TFTP server
  • %USER - Login ID of the user running the test
  • %VERNUM - the version number of the tested curl (without -DEV)
  • %VERSION - the full version number of the tested curl

<testcase>

Each test is always specified entirely within the testcase tag. Each test case is split up in four main sections: info, reply, client and verify.

  • info provides information about the test case

  • reply is used for the server to know what to send as a reply for the requests curl sends

  • client defines how the client should behave

  • verify defines how to verify that the data stored after a command has been run ended up correct

Each main section has a number of available subsections that can be specified, that are checked/used if specified.

<info>

<keywords>

A newline-separated list of keywords describing what this test case uses and tests. Try to use already used keywords. These keywords are used for statistical/informational purposes and for choosing or skipping classes of tests. Keywords must begin with an alphabetic character, -, [ or { and may actually consist of multiple words separated by spaces which are treated together as a single identifier. Most keywords are only there to provide a way for users to skip certain classes of tests, if desired, but a few are treated specially by the test harness or build system.

When running a unit test and the keywords include unittest, the <tool> section can be left empty to use the standard unit test tool name unitN where N is the test number. Unit tests means libcurl unit tests. The command line tool's unit tests are marked as tunittest.

The test-ci make target automatically skips test with the flaky keyword.

Tests that have strict timing dependencies have the timing-dependent keyword. These are intended to eventually be treated specially on CI builds which are often run on overloaded machines with unpredictable timing.

Tests using non-7-bit-ASCII characters must provide them with %hex[] or similar.

<reply>

<data [nocheck="yes"] [sendzero="yes"] [nonewline="yes"] [crlf="yes|headers"]>

data to be sent to the client on its request and later verified that it arrived safely. Set nocheck="yes" to prevent the test script from verifying the arrival of this data.

If the data contains swsclose anywhere within the start and end tag, and this is an HTTP test, then the connection is closed by the server after this response is sent. If not, the connection is kept persistent.

If the data contains swsbounce anywhere within the start and end tag, then the HTTP server overrides the part number response returned for a subsequent request made by the same test to previous part number + 1. For example, if a test makes a request which causes the server to return <data> that contains keyword swsbounce then for the next response it ignores the requested part number and instead returns <data1>. If <data1> contains keyword swsbounce then the next response is <data2> and so on. This is useful for auth tests and similar.

sendzero=yes means that the (FTP) server "sends" the data even if the size is zero bytes. Used to verify curl's behavior on zero bytes transfers.

nonewline=yes means that the last byte (the trailing newline character) should be cut off from the data before sending or comparing it.

crlf=yes forces the newlines to become CRLF even if not written so in the test.

crlf=headers forces header newlines to become CRLF even if not written so in the source file. Note that this makes runtests.pl parse and "guess" what is a header and what is not in order to apply the CRLF line endings appropriately.

For FTP file listings, the <data> section is be used only if you make sure that there has been a CWD done first to a directory named test-[NUM] where NUM is the test case number. Otherwise the ftp server cannot know from which test file to load the list content.

<dataNUM [crlf="yes|headers"]>

Send back this contents instead of the <data> one. The NUM is set by:

  • The test number in the request line is >10000 and this is the remainder of [test case number]%10000.
  • The request was HTTP and included digest details, which adds 1000 to NUM
  • If an HTTP request is Basic and NUM is already >=1000, it adds 1 to NUM
  • If an HTTP request is Negotiate, NUM gets incremented by one for each request with Negotiate authorization header on the same test case.

Dynamically changing NUM in this way allows the test harness to be used to test authentication negotiation where several different requests must be sent to complete a transfer. The response to each request is found in its own data section. Validating the entire negotiation sequence can be done by specifying a datacheck section.

<connect [crlf="yes|headers"]>

The connect section is used instead of the 'data' for all CONNECT requests. The remainder of the rules for the data section then apply but with a connect prefix.

crlf=yes forces the newlines to become CRLF even if not written so in the test.

crlf=headers forces header newlines to become CRLF even if not written so in the source file. Note that this makes runtests.pl parse and "guess" what is a header and what is not in order to apply the CRLF line endings appropriately.

<socks>

Address type and address details as logged by the SOCKS proxy.

<datacheck [mode="text"] [nonewline="yes"] [crlf="yes|headers"]>

if the data is sent but this is what should be checked afterwards. If nonewline=yes is set, runtests cuts off the trailing newline from the data before comparing with the one actually received by the client.

Use the mode="text" attribute if the output is in text mode on platforms that have a text/binary difference.

<datacheckNUM [nonewline="yes"] [mode="text"] [crlf="yes|headers"]>

The contents of numbered datacheck sections are appended to the non-numbered one.

<size>

number to return on an ftp SIZE command (set to -1 to make this command fail)

<mdtm>

what to send back if the client sends a (FTP) MDTM command, set to -1 to have it return that the file does not exist

<postcmd>

special purpose server-command to control its behavior after the reply is sent For HTTP/HTTPS, these are supported:

wait [secs] - Pause for the given time

<servercmd>

Special-commands for the server.

The first line of this file is always set to Testnum [number] by the test script, to allow servers to read that to know what test the client is about to issue.

For FTP/SMTP/POP/IMAP
  • REPLY [command] [return value] [response string] - Changes how the server responds to the [command]. [response string] is evaluated as a perl string, so it can contain embedded \r\n, for example. There is a special [command] named "welcome" (without quotes) which is the string sent immediately on connect as a welcome.
  • REPLYLF (like above but sends the response terminated with LF-only and not CRLF)
  • COUNT [command] [num] - Do the REPLY change for [command] only [num] times and then go back to the built-in approach
  • DELAY [command] [secs] - Delay responding to this command for the given time
  • RETRWEIRDO - Enable the "weirdo" RETR case when multiple response lines appear at once when a file is transferred
  • RETRNOSIZE - Make sure the RETR response does not contain the size of the file
  • RETRSIZE [size] - Force RETR response to contain the specified size
  • NOSAVE - Do not actually save what is received
  • SLOWDOWN - Send FTP responses with 0.01 sec delay between each byte
  • SLOWDOWNDATA - Send FTP responses with 0.01 sec delay between each data byte
  • PASVBADIP - makes PASV send back an illegal IP in its 227 response
  • CAPA [capabilities] - Enables support for and specifies a list of space separated capabilities to return to the client for the IMAP CAPABILITY, POP3 CAPA and SMTP EHLO commands
  • AUTH [mechanisms] - Enables support for SASL authentication and specifies a list of space separated mechanisms for IMAP, POP3 and SMTP
  • STOR [msg] respond with this instead of default after STOR
For HTTP/HTTPS
  • auth_required if this is set and a POST/PUT is made without auth, the server does NOT wait for the full request body to get sent
  • delay: [msecs] - delay this amount after connection
  • idle - do nothing after receiving the request, "sit idle"
  • stream - continuously send data to the client, never-ending
  • writedelay: [msecs] delay this amount between reply packets
  • skip: [num] - instructs the server to ignore reading this many bytes from a PUT or POST request
  • rtp: part [num] channel [num] size [num] - stream a fake RTP packet for the given part on a chosen channel with the given payload size
  • connection-monitor - When used, this logs [DISCONNECT] to the server.input log when the connection is disconnected.
  • upgrade - when an HTTP upgrade header is found, the server upgrades to http2
  • swsclose - instruct server to close connection after response
  • no-expect - do not read the request body if Expect: is present
For TFTP

writedelay: [secs] delay this amount between reply packets (each packet being 512 bytes payload)

<dns>

Commands for the test DNS server.

  • A: [dotted IPv4 address] - set IPv4 address to return
  • AAAA: [numerical IPv6 address] - set IPv6 address to return, with or without []

<client>

<server>

What server(s) this test case requires/uses. Available servers:

  • dict
  • file
  • ftp
  • ftp-ipv6
  • ftps
  • gopher
  • gopher-ipv6
  • gophers
  • http
  • http/2
  • http-ipv6
  • http-proxy
  • https
  • https-proxy
  • https-mtls
  • http-unix
  • imap
  • mqtt
  • pop3
  • rtsp
  • rtsp-ipv6
  • scp
  • sftp
  • smtp
  • socks4
  • socks5
  • socks5unix
  • telnet
  • tftp

Give only one per line. If a test does not require any servers, the <server> subsection should be omitted.

Servers that require a special server certificate can have the PEM certificate filename (found in the certs directory) appended to the server name separated by a space.

<features>

A list of features that MUST be present in the client/library for this test to be able to run. If a required feature is not present then the test is SKIPPED.

Alternatively a feature can be prefixed with an exclamation mark to indicate a feature is NOT required. If the feature is present then the test is SKIPPED.

Features testable here are:

  • --libcurl
  • alt-svc
  • aws - built with aws-sigv4 support
  • AppleIDN
  • asyn-rr - c-ares is used for additional records only
  • brotli
  • c-ares - c-ares is used for (all) name resolves
  • CharConv
  • codeset-utf8 - if the running codeset is UTF-8 capable.
  • cookies
  • crypto
  • cygwin
  • Debug
  • digest
  • DoH
  • getrlimit
  • GnuTLS
  • GSS-API
  • h2c
  • headers-api
  • HSTS
  • HTTP-auth
  • http/2
  • http/3
  • HTTPS-proxy
  • HTTPSRR
  • IDN
  • IPv6
  • Kerberos
  • Largefile
  • large-time - time_t is larger than 32-bit
  • large-size - size_t is larger than 32-bit
  • libssh2
  • libssh
  • badlibssh - libssh configuration incompatible with the test suite
  • libz
  • local-http - the HTTP server runs on 127.0.0.1
  • manual
  • mbedtls
  • Mime
  • netrc
  • nghttpx
  • nghttpx-h3
  • OpenSSL
  • override-dns - this build can use a "fake" DNS server
  • parsedate
  • proxy
  • PSL
  • rustls
  • Schannel
  • shuffle-dns
  • socks
  • SPNEGO
  • SSL
  • SSLpinning
  • SSLS-EXPORT
  • SSPI
  • threaded-resolver
  • TLS-SRP
  • torture - if runtests is running in memory test mode
  • TrackMemory
  • typecheck
  • threadsafe
  • Unicode
  • unittest - that feature needs to be enabled also for the tool unit tests marked as tunittest
  • UnixSockets
  • verbose-strings
  • wakeup
  • win32
  • WinIDN
  • wolfssl
  • xattr
  • zstd

as well as each protocol that curl supports. A protocol only needs to be specified if it is different from the server (useful when the server is none).

<killserver>

Using the same syntax as in <server> but when mentioned here these servers are explicitly KILLED when this test case is completed. Only use this if there is no other alternatives. Using this of course requires subsequent tests to restart servers.

<precheck>

A command line that if set gets run by the test script before the test. If an output is displayed by the command or if the return code is non-zero, the test is skipped and the (single-line) output is displayed as reason for not running the test.

<tool>

Name of tool to invoke instead of "curl". This tool must be built and exist either in the libtest/ directory (if the tool name starts with lib) or in the unit/ directory (if the tool name starts with unit).

If the keyword unittest is set in the test case, the default name for the tool is unit%TESTNUMBER.

If the keyword tunittest is set in the test case, the default name for the tool is tool%TESTNUMBER.

If the keyword libtest is set in the test case, the default name for the tool is lib%TESTNUMBER.

<name>

Brief test case description, shown when the test runs.

<setenv>

variable1=contents1
variable2=contents2
variable3

Set the given environment variables to the specified value before the actual command is run. They are restored back to their former values again after the command has been run.

If the variable name has no assignment, no =, then that variable is deleted.

<command [option="no-q/no-output/no-stdout/no-include/no-memdebug/force-output/binary-trace"] [timeout="secs"][delay="secs"][type="perl/shell"]>

Command line to run.

If the command spans multiple lines, they are concatenated with a space added between them.

Set type="perl" to write the test case as a perl script. It implies that there is no memory debugging and valgrind gets shut off for this test.

Set type="shell" to write the test case as a shell script. It implies that there is no memory debugging and valgrind gets shut off for this test.

Set option="no-output" to prevent the test script to slap on the --output argument that directs the output to a file. The --output is also not added if the verify/stdout section is used.

Set option="no-stdout" to prevent the normal stdout redirect template to be applied on the command line to run.

Set option="force-output" to make use of --output even when the test is otherwise written to verify stdout.

Set option="no-include" to prevent the test script to slap on the --include argument.

Set option="no-memdebug" to make the test run without the memdebug tracking system. For tests that are incompatible - multi-threaded for example.

Set option="no-q" avoid using -q as the first argument in the curl command line.

Set option="binary-trace" to use --trace instead of --trace-ascii for tracing. Suitable for binary-oriented protocols such as MQTT.

Set timeout="secs" to override default server logs advisor read lock timeout. This timeout is used by the test harness, once that the command has completed execution, to wait for the test server to write out server side log files and remove the lock that advised not to read them. The "secs" parameter is the not negative integer number of seconds for the timeout. This timeout attribute is documented for completeness sake, but is deep test harness stuff and only needed for singular and specific test cases. Avoid using it.

Set delay="secs" to introduce a time delay once that the command has completed execution and before the <postcheck> section runs. The "secs" parameter is the not negative integer number of seconds for the delay. This 'delay' attribute is intended for specific test cases, and normally not needed.

<file name="%LOGDIR/filename" [options]>

This creates the named file with this content before the test case is run, which is useful if the test case needs a file to act on.

If nonewline="yes" is used, the created file gets the final newline stripped off.

crlf=yes forces the newlines to become CRLF even if not written so in the test.

mode="text" normalizes the line endings to make them compare as text on all platforms.

<file1>

1 to 4 can be appended to 'file' to create more files.

<file2>

<file3>

<file4>

<stdin [nonewline="yes"][crlf="yes"]>

Pass this given data on stdin to the tool.

If nonewline is set, we cut off the trailing newline of this given data before comparing with the one actually received by the client

crlf=yes forces the newlines to become CRLF even if not written so in the test.

<disable>

If test-duphandle is a listed item here, this is not run when --test-duphandle is used.

<verify>

<errorcode>

numerical error code curl is supposed to return. Specify a list of accepted error codes by separating multiple numbers with comma. See test 237 for an example.

<strip>

One regex per line that is removed from the protocol dumps before the comparison is made. This is useful to remove dependencies on dynamically changing protocol data such as port numbers or user-agent strings.

<strippart>

One perl op per line that operates on the protocol dump. This is pretty advanced. Example: s/^EPRT .*/EPRT stripped/.

<postcheck>

A command line that if set gets run by the test script after the test. If the command exists with a non-zero status code, the test is considered failed.

<notexists>

A list of directory entries that are checked for after the test has completed and that must not exist. A listed entry existing causes the test to fail.

<protocol [nonewline="yes"][crlf="yes|headers"]>

the protocol dump curl should transmit, if nonewline is set, we cut off the trailing newline of this given data before comparing with the one actually sent by the client The <strip> and <strippart> rules are applied before comparisons are made.

crlf=yes forces the newlines to become CRLF even if not written so in the test.

crlf=headers forces header newlines to become CRLF even if not written so in the source file. Note that this makes runtests.pl parse and "guess" what is a header and what is not in order to apply the CRLF line endings appropriately.

<proxy [nonewline="yes"][crlf="yes|headers"]>

The protocol dump curl should transmit to an HTTP proxy (when the http-proxy server is used), if nonewline is set, we cut off the trailing newline of this given data before comparing with the one actually sent by the client The <strip> and <strippart> rules are applied before comparisons are made.

<stderr [mode="text/warn"] [nonewline="yes"] [crlf="yes|headers"]>

This verifies that this data was passed to stderr.

Use the mode="text" attribute if the output is in text mode on platforms that have a text/binary difference.

Use the mode="warn" attribute for curl warning output, as it then makes the check without newlines and the prefix to better handle that the lines may wrap at different points depending on the lengths of the lines and terminal width.

crlf=yes forces the newlines to become CRLF even if not written so in the test.

crlf=headers forces header newlines to become CRLF even if not written so in the source file. Note that this makes runtests.pl parse and "guess" what is a header and what is not in order to apply the CRLF line endings appropriately.

If nonewline is set, we cut off the trailing newline of this given data before comparing with the one actually received by the client

<stdout [mode="text"] [nonewline="yes"] [crlf="yes|headers"] [loadfile="filename"]>

This verifies that this data was passed to stdout.

Use the mode="text" attribute if the output is in text mode on platforms that have a text/binary difference.

If nonewline is set, we cut off the trailing newline of this given data before comparing with the one actually received by the client

crlf=yes forces the newlines to become CRLF even if not written so in the test.

crlf=headers forces header newlines to become CRLF even if not written so in the source file. Note that this makes runtests.pl parse and "guess" what is a header and what is not in order to apply the CRLF line endings appropriately.

loadfile="filename" makes loading the data from an external file.

To verify that there was nothing sent to stdout, put %EMPTY as the only content.

<limit>

When this test runs and curl was built with debug enabled, runtests make sure that the set limits are not exceeded. Supported limits:

Allocations: [number of allocation calls]
Maximum allocated: [maximum concurrent memory allocated]

<file name="%LOGDIR/filename" [mode="text"] [crlf="yes|headers"]>

The file's contents must be identical to this after the test is complete. Use the mode="text" attribute if the output is in text mode on platforms that have a text/binary difference.

crlf=yes forces the newlines to become CRLF even if not written so in the test.

crlf=headers forces header newlines to become CRLF even if not written so in the source file. Note that this makes runtests.pl parse and "guess" what is a header and what is not in order to apply the CRLF line endings appropriately.

<file1>

1 to 4 can be appended to 'file' to compare more files.

<file2>

<file3>

<file4>

<stripfile>

One perl op per line that operates on the output file or stdout before being compared with what is stored in the test file. This is pretty advanced. Example: "s/^EPRT .*/EPRT stripped/"

<stripfile1>

1 to 4 can be appended to stripfile to strip the corresponding <fileN> content

<stripfile2>

<stripfile3>

<stripfile4>

<upload [crlf="yes"] [nonewline="yes"]>

the contents of the upload data curl should have sent

crlf=yes forces upload newlines to become CRLF even if not written so in the source file.

nonewline=yes means that the last byte (the trailing newline character) should be cut off from the upload data before comparing it.

<valgrind>

disable - disables the valgrind log check for this test

<dns [host="name"]>

This specify the input the DNS server is expected to get from curl. Because of differences in implementations, this section is sorted automatically before compared.

Because of local configurations in machines running tests, there may be additional requests sent to [host].[custom suffix]. To prevent such requests to mess up comparisons, we can set the hostname to check in the <dns> tag.

1<!--
2Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
4SPDX-License-Identifier: curl
5-->
7# curl test suite file format
9The curl test suite's file format is simple and extendable, closely resembling
10XML. All data for a single test case resides in a single ASCII file. Labels
11mark the beginning and the end of all sections, and each label must be written
12in its own line. Comments are either XML-style (enclosed with `<!--` and
13`-->`) or shell script style (beginning with `#`) and must appear on their own
14lines and not alongside actual test data. Test data files are syntactically
15valid XML; lack of support for character entities is a big difference but macros
16like %CR fill that particular role here.
18Each test case source exists as a file matching the format
19`tests/data/testNUM`, where `NUM` is the unique test number, and must begin
20with a `testcase` tag, which encompasses the remainder of the file.
22# Preprocessing
24When a test is to be executed, the source file is first preprocessed and
25variables are substituted by their respective contents and the output version
26of the test file is stored as `%LOGDIR/testNUM`. That version is what is read
27and used by the test servers.
29## Base64 Encoding
31In the preprocess stage, a special instruction can be used to have runtests.pl
32base64 encode a certain section and insert in the generated output file. This
33is in particular good for test cases where the test tool is expected to pass
34in base64 encoded content that might use dynamic information that is unique
35for this particular test invocation, like the server port number.
37To insert a base64 encoded string into the output, use this syntax:
39 %b64[ data to encode ]b64%
41The data to encode can then use any of the existing variables mentioned below,
42or even percent-encoded individual bytes. As an example, insert the HTTP
43server's port number (in ASCII) followed by a space and the hexadecimal byte
449a:
46 %b64[%HTTPPORT %9a]b64%
48## Hexadecimal decoding
50In the preprocess stage, a special instruction can be used to have runtests.pl
51generate a sequence of binary bytes.
53To insert a sequence of bytes from a hex encoded string, use this syntax:
55 %hex[ %XX-encoded data to decode ]hex%
57Other bytes within the brackets that not percent encoded are inserted as-is.
59For example, to insert the binary octets 0, 1 and 255 into the test file:
61 %hex[%00%01%FF]hex%
63## Repeat content
65In the preprocess stage, a special instruction can be used to have runtests.pl
66generate a repetitive sequence of bytes.
68To insert a sequence of repeat bytes, use this syntax to make the `<string>`
69get repeated `<number>` of times. The number has to be 1 or larger and the
70string may contain `%HH` hexadecimal codes:
72 %repeat[<number> x <string>]%
74For example, to insert the word hello 100 times:
76 %repeat[100 x hello]%
78## Whitespace
80To force CRLF newline, add this macro to the end of the line:
82 %CR - carriage return
84To add significant whitespace characters at the end of the line, or to empty
85lines:
87 %SP - space
88 %TAB - horizontal tab
90## Special characters
92Macros to help keep data files XML-compliant:
94 %AMP - Ampersand: `&`
95 %GT - Greater-than sign: `>`
96 %LT - Less-than sign: `<`
98## Insert capped epoch days
100Mostly to test capped cookie expire dates: `%days[NUM]` inserts the number of
101seconds for the given number of days into the future, aligned to the nearest
102minute. That is the same calculation the cookie engine uses to cap expiration
103dates.
105## Include file
107This instruction allows a test case to include another file. It is helpful to
108remember that the ordinary variables are expanded before the include happens
109so `%LOGDIR` and the others can be used in the include line.
111The filename cannot contain `%` as that letter is used to end the name for
112the include instruction:
114 %include filename%
116Or, a variant of the above where the file is loaded as a newline-agnostic
117text file, and whitespace, special character macros and variables expanded
118after inclusion:
120 %includetext filename%
122## Conditional lines
124Lines in the test file can be made to appear conditionally on a specific
125feature (see the "features" section below) being set or not set. If the
126specific feature is present, the following lines are output, otherwise it
127outputs nothing, until a following else or `endif` clause. Like this:
129 %if brotli
130 Accept-Encoding
131 %endif
133It can also check for the inverse condition, so if the feature is *not* set by
134the use of an exclamation mark:
136 %if !brotli
137 Accept-Encoding: not-brotli
138 %endif
140You can also make an "else" clause to get output for the opposite condition,
141like:
143 %if brotli
144 Accept-Encoding: brotli
145 %else
146 Accept-Encoding: nothing
147 %endif
149Nested conditions are supported.
151# Variables
153When the test is preprocessed, a range of "variables" in the test file is
154replaced by their content at that time.
156Available substitute variables include:
158- `%CLIENT6IP` - IPv6 address of the client running curl (including brackets)
159- `%CLIENT6IP-NB` - IPv6 address of the client running curl (no brackets)
160- `%CLIENTIP` - IPv4 address of the client running curl
161- `%CURL` - Path to the curl executable
162- `%DATE` - current YYYY-MM-DD date
163- `%DEV_NULL` - Null device (e.g. /dev/null)
164- `%FILE_PWD` - Current directory, on Windows prefixed with a slash
165- `%FTP6PORT` - IPv6 port number of the FTP server
166- `%FTPPORT` - Port number of the FTP server
167- `%FTPSPORT` - Port number of the FTPS server
168- `%FTPTIME2` - Timeout in seconds that should be sufficient to receive a
169 response from the test FTP server
170- `%GOPHER6PORT` - IPv6 port number of the Gopher server
171- `%GOPHERPORT` - Port number of the Gopher server
172- `%GOPHERSPORT` - Port number of the Gophers server
173- `%HOST6IP` - IPv6 address of the host running this test
174- `%HOSTIP` - IPv4 address of the host running this test
175- `%HTTP2PORT` - Port number of the HTTP/2 server
176- `%HTTP6PORT` - IPv6 port number of the HTTP server
177- `%HTTPPORT` - Port number of the HTTP server
178- `%HTTPSPORT` - Port number of the HTTPS server
179- `%HTTPSPROXYPORT` - Port number of the HTTPS-proxy
180- `%HTTPTLS6PORT` - IPv6 port number of the HTTP TLS server
181- `%HTTPTLSPORT` - Port number of the HTTP TLS server
182- `%HTTPUNIXPATH` - Path to the Unix socket of the HTTP server
183- `%IMAP6PORT` - IPv6 port number of the IMAP server
184- `%IMAPPORT` - Port number of the IMAP server
185- `%LOGDIR` - Log directory relative to %PWD
186- `%MQTTPORT` - Port number of the MQTT server
187- `%MQTTSPORT` - Port number of the MQTTS server
188- `%NOLISTENPORT` - Port number where no service is listening
189- `%POP36PORT` - IPv6 port number of the POP3 server
190- `%POP3PORT` - Port number of the POP3 server
191- `%PROXYPORT` - Port number of the HTTP proxy
192- `%PWD` - Current directory
193- `%RESOLVE` - server/resolve command
194- `%RTSP6PORT` - IPv6 port number of the RTSP server
195- `%RTSPPORT` - Port number of the RTSP server
196- `%SMTP6PORT` - IPv6 port number of the SMTP server
197- `%SMTPPORT` - Port number of the SMTP server
198- `%SOCKSPORT` - Port number of the SOCKS4/5 server
199- `%SOCKSUNIXPATH` - Path to the Unix socket of the SOCKS server
200- `%SRCDIR` - Full path to the source dir
201- `%SCP_PWD` - Current directory friendly for the SSH server for the `scp://` protocol
202- `%SFTP_PWD` - Current directory friendly for the SSH server for the `sftp://` protocol
203- `%SSHKEYALGO` - SSH host and client key algorithm, e.g. `ssh-rsa` or `ssh-ed25519`
204- `%SSHPORT` - Port number of the SCP/SFTP server
205- `%SSHSRVMD5` - MD5 of SSH server's public key
206- `%SSHSRVSHA256` - SHA256 of SSH server's public key
207- `%TELNETPORT` - Port number of the telnet server
208- `%TESTNUMBER` - Number of the test case
209- `%TFTP6PORT` - IPv6 port number of the TFTP server
210- `%TFTPPORT` - Port number of the TFTP server
211- `%USER` - Login ID of the user running the test
212- `%VERNUM` - the version number of the tested curl (without -DEV)
213- `%VERSION` - the full version number of the tested curl
215# `<testcase>`
217Each test is always specified entirely within the `testcase` tag. Each test
218case is split up in four main sections: `info`, `reply`, `client` and
219`verify`.
221- **info** provides information about the test case
223- **reply** is used for the server to know what to send as a reply for the
224requests curl sends
226- **client** defines how the client should behave
228- **verify** defines how to verify that the data stored after a command has
229been run ended up correct
231Each main section has a number of available subsections that can be specified,
232that are checked/used if specified.
234## `<info>`
236### `<keywords>`
237A newline-separated list of keywords describing what this test case uses and
238tests. Try to use already used keywords. These keywords are used for
239statistical/informational purposes and for choosing or skipping classes of
240tests. Keywords must begin with an alphabetic character, `-`, `[` or `{` and
241may actually consist of multiple words separated by spaces which are treated
242together as a single identifier. Most keywords are only there to provide a way
243for users to skip certain classes of tests, if desired, but a few are treated
244specially by the test harness or build system.
246When running a unit test and the keywords include `unittest`, the `<tool>`
247section can be left empty to use the standard unit test tool name `unitN`
248where `N` is the test number. Unit tests means *libcurl* unit tests. The
249command line tool's unit tests are marked as `tunittest`.
251The `test-ci` make target automatically skips test with the `flaky` keyword.
253Tests that have strict timing dependencies have the `timing-dependent` keyword.
254These are intended to eventually be treated specially on CI builds which are
255often run on overloaded machines with unpredictable timing.
257Tests using non-7-bit-ASCII characters must provide them with `%hex[]` or
258similar.
260## `<reply>`
262### `<data [nocheck="yes"] [sendzero="yes"] [nonewline="yes"] [crlf="yes|headers"]>`
264data to be sent to the client on its request and later verified that it
265arrived safely. Set `nocheck="yes"` to prevent the test script from verifying
266the arrival of this data.
268If the data contains `swsclose` anywhere within the start and end tag, and
269this is an HTTP test, then the connection is closed by the server after this
270response is sent. If not, the connection is kept persistent.
272If the data contains `swsbounce` anywhere within the start and end tag, then
273the HTTP server overrides the part number response returned for a subsequent
274request made by the same test to `previous part number + 1`. For example, if a
275test makes a request which causes the server to return `<data>` that contains
276keyword `swsbounce` then for the next response it ignores the requested part
277number and instead returns `<data1>`. If `<data1>` contains keyword
278`swsbounce` then the next response is `<data2>` and so on. This is useful for
279auth tests and similar.
281`sendzero=yes` means that the (FTP) server "sends" the data even if the size
282is zero bytes. Used to verify curl's behavior on zero bytes transfers.
284`nonewline=yes` means that the last byte (the trailing newline character)
285should be cut off from the data before sending or comparing it.
287`crlf=yes` forces the newlines to become CRLF even if not written so in the
288test.
290`crlf=headers` forces *header* newlines to become CRLF even if not written so
291in the source file. Note that this makes runtests.pl parse and "guess" what is
292a header and what is not in order to apply the CRLF line endings appropriately.
294For FTP file listings, the `<data>` section is be used *only* if you make sure
295that there has been a CWD done first to a directory named `test-[NUM]` where
296`NUM` is the test case number. Otherwise the ftp server cannot know from which
297test file to load the list content.
299### `<dataNUM [crlf="yes|headers"]>`
301Send back this contents instead of the `<data>` one. The `NUM` is set by:
303- The test number in the request line is >10000 and this is the remainder of
304 [test case number]%10000.
305- The request was HTTP and included digest details, which adds 1000 to `NUM`
306- If an HTTP request is Basic and `NUM` is already >=1000, it adds 1 to `NUM`
307- If an HTTP request is Negotiate, `NUM` gets incremented by one for each
308 request with Negotiate authorization header on the same test case.
310Dynamically changing `NUM` in this way allows the test harness to be used to
311test authentication negotiation where several different requests must be sent
312to complete a transfer. The response to each request is found in its own data
313section. Validating the entire negotiation sequence can be done by specifying
314a `datacheck` section.
316### `<connect [crlf="yes|headers"]>`
317The connect section is used instead of the 'data' for all CONNECT
318requests. The remainder of the rules for the data section then apply but with
319a connect prefix.
321`crlf=yes` forces the newlines to become CRLF even if not written so in the
322test.
324`crlf=headers` forces *header* newlines to become CRLF even if not written so
325in the source file. Note that this makes runtests.pl parse and "guess" what is
326a header and what is not in order to apply the CRLF line endings appropriately.
328### `<socks>`
329Address type and address details as logged by the SOCKS proxy.
331### `<datacheck [mode="text"] [nonewline="yes"] [crlf="yes|headers"]>`
332if the data is sent but this is what should be checked afterwards. If
333`nonewline=yes` is set, runtests cuts off the trailing newline from the data
334before comparing with the one actually received by the client.
336Use the `mode="text"` attribute if the output is in text mode on platforms
337that have a text/binary difference.
339### `<datacheckNUM [nonewline="yes"] [mode="text"] [crlf="yes|headers"]>`
340The contents of numbered `datacheck` sections are appended to the non-numbered
341one.
343### `<size>`
344number to return on an ftp SIZE command (set to -1 to make this command fail)
346### `<mdtm>`
347what to send back if the client sends a (FTP) `MDTM` command, set to -1 to
348have it return that the file does not exist
350### `<postcmd>`
351special purpose server-command to control its behavior *after* the
352reply is sent
353For HTTP/HTTPS, these are supported:
355`wait [secs]` - Pause for the given time
357### `<servercmd>`
358Special-commands for the server.
360The first line of this file is always set to `Testnum [number]` by the test
361script, to allow servers to read that to know what test the client is about to
362issue.
364#### For FTP/SMTP/POP/IMAP
366- `REPLY [command] [return value] [response string]` - Changes how the server
367 responds to the [command]. [response string] is evaluated as a perl string,
368 so it can contain embedded \r\n, for example. There is a special [command]
369 named "welcome" (without quotes) which is the string sent immediately on
370 connect as a welcome.
371- `REPLYLF` (like above but sends the response terminated with LF-only and not
372 CRLF)
373- `COUNT [command] [num]` - Do the `REPLY` change for `[command]` only `[num]`
374 times and then go back to the built-in approach
375- `DELAY [command] [secs]` - Delay responding to this command for the given
376 time
377- `RETRWEIRDO` - Enable the "weirdo" RETR case when multiple response lines
378 appear at once when a file is transferred
379- `RETRNOSIZE` - Make sure the RETR response does not contain the size of the
380 file
381- `RETRSIZE [size]` - Force RETR response to contain the specified size
382- `NOSAVE` - Do not actually save what is received
383- `SLOWDOWN` - Send FTP responses with 0.01 sec delay between each byte
384- `SLOWDOWNDATA` - Send FTP responses with 0.01 sec delay between each data
385 byte
386- `PASVBADIP` - makes PASV send back an illegal IP in its 227 response
387- `CAPA [capabilities]` - Enables support for and specifies a list of space
388 separated capabilities to return to the client for the IMAP `CAPABILITY`,
389 POP3 `CAPA` and SMTP `EHLO` commands
390- `AUTH [mechanisms]` - Enables support for SASL authentication and specifies
391 a list of space separated mechanisms for IMAP, POP3 and SMTP
392- `STOR [msg]` respond with this instead of default after `STOR`
394#### For HTTP/HTTPS
396- `auth_required` if this is set and a POST/PUT is made without auth, the
397 server does NOT wait for the full request body to get sent
398- `delay: [msecs]` - delay this amount after connection
399- `idle` - do nothing after receiving the request, "sit idle"
400- `stream` - continuously send data to the client, never-ending
401- `writedelay: [msecs]` delay this amount between reply packets
402- `skip: [num]` - instructs the server to ignore reading this many bytes from
403 a PUT or POST request
404- `rtp: part [num] channel [num] size [num]` - stream a fake RTP packet for
405 the given part on a chosen channel with the given payload size
406- `connection-monitor` - When used, this logs `[DISCONNECT]` to the
407 `server.input` log when the connection is disconnected.
408- `upgrade` - when an HTTP upgrade header is found, the server upgrades to
409 http2
410- `swsclose` - instruct server to close connection after response
411- `no-expect` - do not read the request body if Expect: is present
413#### For TFTP
414`writedelay: [secs]` delay this amount between reply packets (each packet
415 being 512 bytes payload)
417### `<dns>`
419Commands for the test DNS server.
421- `A: [dotted IPv4 address]` - set IPv4 address to return
422- `AAAA: [numerical IPv6 address]` - set IPv6 address to return, with or
423 without `[]`
425## `<client>`
427### `<server>`
428What server(s) this test case requires/uses. Available servers:
430- `dict`
431- `file`
432- `ftp`
433- `ftp-ipv6`
434- `ftps`
435- `gopher`
436- `gopher-ipv6`
437- `gophers`
438- `http`
439- `http/2`
440- `http-ipv6`
441- `http-proxy`
442- `https`
443- `https-proxy`
444- `https-mtls`
445- `http-unix`
446- `imap`
447- `mqtt`
448- `pop3`
449- `rtsp`
450- `rtsp-ipv6`
451- `scp`
452- `sftp`
453- `smtp`
454- `socks4`
455- `socks5`
456- `socks5unix`
457- `telnet`
458- `tftp`
460Give only one per line. If a test does not require any servers, the `<server>`
461subsection should be omitted.
463Servers that require a special server certificate can
464have the PEM certificate filename (found in the `certs` directory) appended to
465the server name separated by a space.
467### `<features>`
468A list of features that MUST be present in the client/library for this test to
469be able to run. If a required feature is not present then the test is SKIPPED.
471Alternatively a feature can be prefixed with an exclamation mark to indicate a
472feature is NOT required. If the feature is present then the test is SKIPPED.
474Features testable here are:
476- `--libcurl`
477- `alt-svc`
478- `aws` - built with **aws-sigv4** support
479- `AppleIDN`
480- `asyn-rr` - c-ares is used for additional records only
481- `brotli`
482- `c-ares` - c-ares is used for (all) name resolves
483- `CharConv`
484- `codeset-utf8` - if the running codeset is UTF-8 capable.
485- `cookies`
486- `crypto`
487- `cygwin`
488- `Debug`
489- `digest`
490- `DoH`
491- `getrlimit`
492- `GnuTLS`
493- `GSS-API`
494- `h2c`
495- `headers-api`
496- `HSTS`
497- `HTTP-auth`
498- `http/2`
499- `http/3`
500- `HTTPS-proxy`
501- `HTTPSRR`
502- `IDN`
503- `IPv6`
504- `Kerberos`
505- `Largefile`
506- `large-time` - time_t is larger than 32-bit
507- `large-size` - size_t is larger than 32-bit
508- `libssh2`
509- `libssh`
510- `badlibssh` - libssh configuration incompatible with the test suite
511- `libz`
512- `local-http` - the HTTP server runs on 127.0.0.1
513- `manual`
514- `mbedtls`
515- `Mime`
516- `netrc`
517- `nghttpx`
518- `nghttpx-h3`
519- `OpenSSL`
520- `override-dns` - this build can use a "fake" DNS server
521- `parsedate`
522- `proxy`
523- `PSL`
524- `rustls`
525- `Schannel`
526- `shuffle-dns`
527- `socks`
528- `SPNEGO`
529- `SSL`
530- `SSLpinning`
531- `SSLS-EXPORT`
532- `SSPI`
533- `threaded-resolver`
534- `TLS-SRP`
535- `torture` - if runtests is running in memory test mode
536- `TrackMemory`
537- `typecheck`
538- `threadsafe`
539- `Unicode`
540- `unittest` - that feature needs to be enabled also for the tool unit tests marked as `tunittest`
541- `UnixSockets`
542- `verbose-strings`
543- `wakeup`
544- `win32`
545- `WinIDN`
546- `wolfssl`
547- `xattr`
548- `zstd`
550as well as each protocol that curl supports. A protocol only needs to be
551specified if it is different from the server (useful when the server is
552`none`).
554### `<killserver>`
555Using the same syntax as in `<server>` but when mentioned here these servers
556are explicitly KILLED when this test case is completed. Only use this if there
557is no other alternatives. Using this of course requires subsequent tests to
558restart servers.
560### `<precheck>`
561A command line that if set gets run by the test script before the test. If an
562output is displayed by the command or if the return code is non-zero, the test
563is skipped and the (single-line) output is displayed as reason for not running
564the test.
566### `<tool>`
568Name of tool to invoke instead of "curl". This tool must be built and exist
569either in the `libtest/` directory (if the tool name starts with `lib`) or in
570the `unit/` directory (if the tool name starts with `unit`).
572If the keyword `unittest` is set in the test case, the default name for the
573tool is `unit%TESTNUMBER`.
575If the keyword `tunittest` is set in the test case, the default name for the
576tool is `tool%TESTNUMBER`.
578If the keyword `libtest` is set in the test case, the default name for the
579tool is `lib%TESTNUMBER`.
581### `<name>`
582Brief test case description, shown when the test runs.
584### `<setenv>`
586 variable1=contents1
587 variable2=contents2
588 variable3
590Set the given environment variables to the specified value before the actual
591command is run. They are restored back to their former values again after the
592command has been run.
594If the variable name has no assignment, no `=`, then that variable is deleted.
596### `<command [option="no-q/no-output/no-stdout/no-include/no-memdebug/force-output/binary-trace"] [timeout="secs"][delay="secs"][type="perl/shell"]>`
597Command line to run.
599If the command spans multiple lines, they are concatenated with a space added
600between them.
602Set `type="perl"` to write the test case as a perl script. It implies that
603there is no memory debugging and valgrind gets shut off for this test.
605Set `type="shell"` to write the test case as a shell script. It implies that
606there is no memory debugging and valgrind gets shut off for this test.
608Set `option="no-output"` to prevent the test script to slap on the `--output`
609argument that directs the output to a file. The `--output` is also not added
610if the verify/stdout section is used.
612Set `option="no-stdout"` to prevent the normal stdout redirect template to be
613applied on the command line to run.
615Set `option="force-output"` to make use of `--output` even when the test is
616otherwise written to verify stdout.
618Set `option="no-include"` to prevent the test script to slap on the
619`--include` argument.
621Set `option="no-memdebug"` to make the test run without the memdebug tracking
622system. For tests that are incompatible - multi-threaded for example.
624Set `option="no-q"` avoid using `-q` as the first argument in the curl command
625line.
627Set `option="binary-trace"` to use `--trace` instead of `--trace-ascii` for
628tracing. Suitable for binary-oriented protocols such as MQTT.
630Set `timeout="secs"` to override default server logs advisor read lock
631timeout. This timeout is used by the test harness, once that the command has
632completed execution, to wait for the test server to write out server side log
633files and remove the lock that advised not to read them. The "secs" parameter
634is the not negative integer number of seconds for the timeout. This `timeout`
635attribute is documented for completeness sake, but is deep test harness stuff
636and only needed for singular and specific test cases. Avoid using it.
638Set `delay="secs"` to introduce a time delay once that the command has
639completed execution and before the `<postcheck>` section runs. The "secs"
640parameter is the not negative integer number of seconds for the delay. This
641'delay' attribute is intended for specific test cases, and normally not
642needed.
644### `<file name="%LOGDIR/filename" [options]>`
645This creates the named file with this content before the test case is run,
646which is useful if the test case needs a file to act on.
648If `nonewline="yes"` is used, the created file gets the final newline stripped
649off.
651`crlf=yes` forces the newlines to become CRLF even if not written so in the
652test.
654`mode="text"` normalizes the line endings to make them compare as text on all
655platforms.
657### `<file1>`
6581 to 4 can be appended to 'file' to create more files.
660### `<file2>`
662### `<file3>`
664### `<file4>`
666### `<stdin [nonewline="yes"][crlf="yes"]>`
667Pass this given data on stdin to the tool.
669If `nonewline` is set, we cut off the trailing newline of this given data
670before comparing with the one actually received by the client
672`crlf=yes` forces the newlines to become CRLF even if not written so in the
673test.
675## `<disable>`
677If `test-duphandle` is a listed item here, this is not run when
678`--test-duphandle` is used.
680## `<verify>`
681### `<errorcode>`
682numerical error code curl is supposed to return. Specify a list of accepted
683error codes by separating multiple numbers with comma. See test 237 for an
684example.
686### `<strip>`
687One regex per line that is removed from the protocol dumps before the
688comparison is made. This is useful to remove dependencies on dynamically
689changing protocol data such as port numbers or user-agent strings.
691### `<strippart>`
692One perl op per line that operates on the protocol dump. This is pretty
693advanced. Example: `s/^EPRT .*/EPRT stripped/`.
695### `<postcheck>`
696A command line that if set gets run by the test script after the test. If the
697command exists with a non-zero status code, the test is considered failed.
699### `<notexists>`
700A list of directory entries that are checked for after the test has completed
701and that must not exist. A listed entry existing causes the test to fail.
703### `<protocol [nonewline="yes"][crlf="yes|headers"]>`
705the protocol dump curl should transmit, if `nonewline` is set, we cut off the
706trailing newline of this given data before comparing with the one actually
707sent by the client The `<strip>` and `<strippart>` rules are applied before
708comparisons are made.
710`crlf=yes` forces the newlines to become CRLF even if not written so in the
711test.
713`crlf=headers` forces *header* newlines to become CRLF even if not written so
714in the source file. Note that this makes runtests.pl parse and "guess" what is
715a header and what is not in order to apply the CRLF line endings appropriately.
717### `<proxy [nonewline="yes"][crlf="yes|headers"]>`
719The protocol dump curl should transmit to an HTTP proxy (when the http-proxy
720server is used), if `nonewline` is set, we cut off the trailing newline of
721this given data before comparing with the one actually sent by the client The
722`<strip>` and `<strippart>` rules are applied before comparisons are made.
724### `<stderr [mode="text/warn"] [nonewline="yes"] [crlf="yes|headers"]>`
725This verifies that this data was passed to stderr.
727Use the mode="text" attribute if the output is in text mode on platforms that
728have a text/binary difference.
730Use the mode="warn" attribute for curl warning output, as it then makes the
731check without newlines and the prefix to better handle that the lines may wrap
732at different points depending on the lengths of the lines and terminal width.
734`crlf=yes` forces the newlines to become CRLF even if not written so in the
735test.
737`crlf=headers` forces *header* newlines to become CRLF even if not written so
738in the source file. Note that this makes runtests.pl parse and "guess" what is
739a header and what is not in order to apply the CRLF line endings appropriately.
741If `nonewline` is set, we cut off the trailing newline of this given data
742before comparing with the one actually received by the client
744### `<stdout [mode="text"] [nonewline="yes"] [crlf="yes|headers"] [loadfile="filename"]>`
745This verifies that this data was passed to stdout.
747Use the mode="text" attribute if the output is in text mode on platforms that
748have a text/binary difference.
750If `nonewline` is set, we cut off the trailing newline of this given data
751before comparing with the one actually received by the client
753`crlf=yes` forces the newlines to become CRLF even if not written so in the
754test.
756`crlf=headers` forces *header* newlines to become CRLF even if not written so
757in the source file. Note that this makes runtests.pl parse and "guess" what is
758a header and what is not in order to apply the CRLF line endings appropriately.
760`loadfile="filename"` makes loading the data from an external file.
762To verify that there was nothing sent to stdout, put `%EMPTY` as the only
763content.
765### `<limit>`
767When this test runs and curl was built with debug enabled, runtests make sure
768that the set limits are not exceeded. Supported limits:
770 Allocations: [number of allocation calls]
771 Maximum allocated: [maximum concurrent memory allocated]
773### `<file name="%LOGDIR/filename" [mode="text"] [crlf="yes|headers"]>`
774The file's contents must be identical to this after the test is complete. Use
775the mode="text" attribute if the output is in text mode on platforms that have
776a text/binary difference.
778`crlf=yes` forces the newlines to become CRLF even if not written so in the
779test.
781`crlf=headers` forces *header* newlines to become CRLF even if not written so
782in the source file. Note that this makes runtests.pl parse and "guess" what is
783a header and what is not in order to apply the CRLF line endings appropriately.
785### `<file1>`
7861 to 4 can be appended to 'file' to compare more files.
788### `<file2>`
790### `<file3>`
792### `<file4>`
794### `<stripfile>`
795One perl op per line that operates on the output file or stdout before being
796compared with what is stored in the test file. This is pretty
797advanced. Example: "s/^EPRT .*/EPRT stripped/"
799### `<stripfile1>`
8001 to 4 can be appended to `stripfile` to strip the corresponding `<fileN>`
801content
803### `<stripfile2>`
805### `<stripfile3>`
807### `<stripfile4>`
809### `<upload [crlf="yes"] [nonewline="yes"]>`
810the contents of the upload data curl should have sent
812`crlf=yes` forces *upload* newlines to become CRLF even if not written so in
813the source file.
815`nonewline=yes` means that the last byte (the trailing newline character)
816should be cut off from the upload data before comparing it.
818### `<valgrind>`
819disable - disables the valgrind log check for this test
821### `<dns [host="name"]>`
823This specify the input the DNS server is expected to get from curl. Because of
824differences in implementations, this section is sorted automatically before
825compared.
827Because of local configurations in machines running tests, there may be
828additional requests sent to `[host].[custom suffix]`. To prevent such requests
829to mess up comparisons, we can set the hostname to check in the `<dns>` tag.
Diagnostics4 git calls · 26 ms
CallStatusSizeTime026 ms
discover200152 B2 ms
git.upload-pack discover
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=3
git.response.bytes=152
git.server.agent=gitty/1.0
git.server.object_format=sha1
git.server.caps=agent ls-refs fetch object-format
git.server.fetch=shallow filter sideband-all
git.protocol.version=2
git.pkt.frames=8
git.pkt.data=6
git.pkt.flush=2
git.pkt.delim=0
git.pkt.bytes=152
git.pkt.payload.max=34
ls-refs200149 B · hit2 ms
git.upload-pack ls-refs
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=2
git.response.bytes=149
git.response.cache=HIT
git.response.duration_ms=2
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=149
git.pkt.payload.max=78
git.ls_refs.count=2
git.ls_refs.head=refs/heads/master
fetchblob:none · deepen 10200287 KiB · hit3 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=19
git.response.bytes=293470
git.response.cache=HIT
git.response.duration_ms=3
git.fetch.filter=blob:none
git.fetch.filter.requested=tree:0
git.fetch.filter.fallback=blob:none
git.fetch.filter.learned=false
git.fetch.wants=31c911efe5652725e930f4d2e60c9060e5f6646d
git.fetch.haves.count=0
git.fetch.deepen=10
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=10
git.pkt.data=8
git.pkt.flush=1
git.pkt.delim=1
git.pkt.bytes=293470
git.pkt.payload.max=65516
git.sideband.band1.bytes=293354
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=87
git.pack.bytes=293354
git.pack.ofs_delta=0
git.pack.ref_delta=1
git.pack.chain.max=1
git.pack.trailer=6a646bcda6740bab42e321b51998b52ea15146c0
git.pack.full=86
Open pack map
fetch20011 KiB · hit21 ms
git.upload-pack fetch
http.response.status_code=200
git.response.bytes=11018
git.response.cache=HIT
git.response.duration_ms=21
git.fetch.wants=e29b91dfba61d2e400b84766e69dd991c6171f6a
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=11018
git.pkt.payload.max=10997
git.sideband.band1.bytes=10996
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0

Each row is one HTTP round trip ply made to the git server for this page, on one timeline. discover asks what the server speaks, ls-refs lists branch tips, fetch brings objects by id. A hit was served by Workers Cache. Tap a row for its wire details.

Remote
https://5ad2236b39de4c8bd8f6270ea63f3ea9.artifacts.cloudflare.net/git/public-v0/curl.git
Ref
refs/heads/master
Commit
31c911efe5652725e930f4d2e60c9060e5f6646d
cf-ray
a5e2db453bf7c125
Request
0137a7e1…
Protocol
v2 · sha1
Agent
gitty/1.0
Filter
advertised
Fetch
shallow filter sideband-all
Capabilities
agent ls-refs fetch object-format