Artifactspublic-v0

curl

master
ply4 git calls212 ms288 KiB2 cachedprotocol v2
90 lines3.1 KiBRaw

scorecard.py

This is an internal script in tests/http/scorecard.py used for testing curl's performance in a set of cases. These are for exercising parts of curl/libcurl in a reproducible fashion to judge improvements or detect regressions. They are not intended to represent real world scenarios as such.

This script is not part of any official interface and we may change it in the future according to the project's needs.

setup

When you are able to run curl's pytest suite, scorecard should work for you as well. They start a local Apache httpd or Caddy server and invoke the locally build src/curl (by default).

invocation

A typical invocation for measuring performance of HTTP/2 downloads would be:

sh
curl> python3 tests/http/scorecard.py -d h2

and this prints a table with the results. The last argument is the protocol to test and it can be h1, h2 or h3. You can add --json to get results in JSON instead of text.

Help for all command line options are available via:

sh
curl> python3 tests/http/scorecard.py -h

scenarios

Apart from -d/--downloads there is -u/--uploads and -r/--requests. These are run with a variation of resource sizes and parallelism by default. You can specify these in some way if you are interested in a particular case.

For example, to run downloads of a 1 MB resource only, 100 times with at max 6 parallel transfers, use:

sh
curl> python3 tests/http/scorecard.py -d --download-sizes=1mb --download-count=100 --download-parallel=6 h2

Similar options are available for uploads and requests scenarios.

sockd

If you have configured curl with --with-test-danted=<danted-path> for a dante-server installed on your system, you can provide the scorecard with arguments --socks4 or --socks5 to test performance with a SOCKS proxy involved. (Note: this does not work for HTTP/3)

flame graphs

With the excellent Flame Graph by Brendan Gregg, scorecard can turn perf/dtrace samples into an interactive SVG. Either clone the Flamegraph repository next to your curl project or set the environment variable FLAMEGRAPH to the location of your clone. Then run scorecard with the --flame option, like

sh
curl> FLAMEGRAPH=/Users/sei/projects/FlameGraph python3 tests/http/scorecard.py \
   -r --request-count=50000 --request-parallels=100 --samples=1 --flame h2

and the SVG of the run is in tests/http/gen/curl/curl.flamegraph.svg. You can open that in Firefox and zoom in/out of stacks of interest.

The flame graph is about the last run of curl. That is why you should add scorecard arguments that restrict measurements to a single run.

Measures/Privileges

The --flame option uses perf on linux and dtrace on macOS. Since both tools require special privileges, they are run via the sudo command by scorecard. This means you need to issue a sudo recently enough before running scorecard, so no new password check is needed.

There is no support right now for measurements on other platforms.

1<!--
2Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
4SPDX-License-Identifier: curl
5-->
7# scorecard.py
9This is an internal script in `tests/http/scorecard.py` used for testing
10curl's performance in a set of cases. These are for exercising parts of
11curl/libcurl in a reproducible fashion to judge improvements or detect
12regressions. They are not intended to represent real world scenarios
13as such.
15This script is not part of any official interface and we may change it in
16the future according to the project's needs.
18## setup
20When you are able to run curl's `pytest` suite, scorecard should work for you
21as well. They start a local Apache httpd or Caddy server and invoke the
22locally build `src/curl` (by default).
24## invocation
26A typical invocation for measuring performance of HTTP/2 downloads would be:
28```sh
29curl> python3 tests/http/scorecard.py -d h2
30```
32and this prints a table with the results. The last argument is the protocol to
33test and it can be `h1`, `h2` or `h3`. You can add `--json` to get results in
34JSON instead of text.
36Help for all command line options are available via:
38```sh
39curl> python3 tests/http/scorecard.py -h
40```
42## scenarios
44Apart from `-d/--downloads` there is `-u/--uploads` and `-r/--requests`. These
45are run with a variation of resource sizes and parallelism by default. You can
46specify these in some way if you are interested in a particular case.
48For example, to run downloads of a 1 MB resource only, 100 times with at max 6
49parallel transfers, use:
51```sh
52curl> python3 tests/http/scorecard.py -d --download-sizes=1mb --download-count=100 --download-parallel=6 h2
53```
55Similar options are available for uploads and requests scenarios.
57## sockd
59If you have configured curl with `--with-test-danted=<danted-path>` for a
60`dante-server` installed on your system, you can provide the scorecard
61with arguments `--socks4` or `--socks5` to test performance with a SOCKS proxy
62involved. (Note: this does not work for HTTP/3)
64## flame graphs
66With the excellent [Flame Graph](https://github.com/brendangregg/FlameGraph)
67by Brendan Gregg, scorecard can turn `perf`/`dtrace` samples into an
68interactive SVG. Either clone the `Flamegraph` repository next to your `curl`
69project or set the environment variable `FLAMEGRAPH` to the location of your
70clone. Then run scorecard with the `--flame` option, like
72```sh
73curl> FLAMEGRAPH=/Users/sei/projects/FlameGraph python3 tests/http/scorecard.py \
74 -r --request-count=50000 --request-parallels=100 --samples=1 --flame h2
75```
77and the SVG of the run is in `tests/http/gen/curl/curl.flamegraph.svg`. You
78can open that in Firefox and zoom in/out of stacks of interest.
80The flame graph is about the last run of `curl`. That is why you should add
81scorecard arguments that restrict measurements to a single run.
83### Measures/Privileges
85The `--flame` option uses `perf` on linux and `dtrace` on macOS. Since both
86tools require special privileges, they are run via the `sudo` command by
87scorecard. This means you need to issue a `sudo` recently enough before
88running scorecard, so no new password check is needed.
90There is no support right now for measurements on other platforms.
Diagnostics4 git calls · 212 ms
CallStatusSizeTime0212 ms
discover200152 B2 ms
git.upload-pack discover
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=2
git.response.bytes=152
git.server.agent=gitty/1.0
git.server.object_format=sha1
git.server.caps=agent ls-refs fetch object-format
git.server.fetch=shallow filter sideband-all
git.protocol.version=2
git.pkt.frames=8
git.pkt.data=6
git.pkt.flush=2
git.pkt.delim=0
git.pkt.bytes=152
git.pkt.payload.max=34
ls-refs200149 B · hit2 ms
git.upload-pack ls-refs
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=2
git.response.bytes=149
git.response.cache=HIT
git.response.duration_ms=2
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=149
git.pkt.payload.max=78
git.ls_refs.count=2
git.ls_refs.head=refs/heads/master
fetchblob:none · deepen 10200287 KiB · hit3 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=3
git.response.bytes=293470
git.response.cache=HIT
git.response.duration_ms=3
git.fetch.filter=blob:none
git.fetch.filter.requested=tree:0
git.fetch.filter.fallback=blob:none
git.fetch.filter.learned=false
git.fetch.wants=31c911efe5652725e930f4d2e60c9060e5f6646d
git.fetch.haves.count=0
git.fetch.deepen=10
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=10
git.pkt.data=8
git.pkt.flush=1
git.pkt.delim=1
git.pkt.bytes=293470
git.pkt.payload.max=65516
git.sideband.band1.bytes=293354
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=87
git.pack.bytes=293354
git.pack.ofs_delta=0
git.pack.ref_delta=1
git.pack.chain.max=1
git.pack.trailer=6a646bcda6740bab42e321b51998b52ea15146c0
git.pack.full=86
Open pack map
fetch2001.6 KiB207 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=207
git.response.bytes=1646
git.response.cache=MISS
git.response.duration_ms=207
git.fetch.wants=049e1567b69ba3c380af4c5d9b007e5fbedfb1f3
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=1646
git.pkt.payload.max=1625
git.sideband.band1.bytes=1624
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0

Each row is one HTTP round trip ply made to the git server for this page, on one timeline. discover asks what the server speaks, ls-refs lists branch tips, fetch brings objects by id. A hit was served by Workers Cache. Tap a row for its wire details.

Remote
https://5ad2236b39de4c8bd8f6270ea63f3ea9.artifacts.cloudflare.net/git/public-v0/curl.git
Ref
refs/heads/master
Commit
31c911efe5652725e930f4d2e60c9060e5f6646d
cf-ray
a5e2c4895b99aedd
Request
8a6dfb94…
Protocol
v2 · sha1
Agent
gitty/1.0
Filter
advertised
Fetch
shallow filter sideband-all
Capabilities
agent ls-refs fetch object-format