Artifactspublic-v0

curl

master
ply4 git calls176 ms289 KiB2 cachedprotocol v2
100 lines3.4 KiBRaw

HTTPS RR

RFC 9460 documents the HTTPS DNS Resource Record.

curl features experimental support for HTTPS RR.

  • The ALPN list from the record is parsed and used
  • The ECH field is stored - and used if ECH is enabled in the build
  • The port number is not used (Firefox supports it, Chrome does not)
  • The target name is not used
  • The IP addresses (Ipv6hints, Ipv4hints) from the HTTPS RR are not used
  • It only supports a single HTTPS RR per hostname
  • Hostnames without A/AAAA records but with HTTPS RR fails
  • consider service profiles where the RR provides different addresses for TCP vs QUIC etc

HTTPSRR is listed as a feature in the curl -V output if curl contains HTTPS RR support. If c-ares is not included in the build, the HTTPS RR support is limited to DoH.

asyn-rr is listed as a feature in the curl -V output if c-ares is used for additional resolves in addition to a "normal" resolve done with the threaded resolver.

The data extracted from the HTTPS RR is stored in the in-memory DNS cache to be reused on subsequent uses of the same hostnames.

limitations

We have decided to work on the HTTPS RR support by following what seems to be (widely) used, and wait with implementing the details of the record that do not seem to be deployed. HTTPS RR is a DNS field with many odd corners and complexities and we might as well avoid them if no one seems to want them.

build

./configure --enable-httpsrr

or

cmake -DUSE_HTTPSRR=ON

ALPN

The list of ALPN IDs is parsed but may not be completely respected because of what the HTTP version preference is set to, which is a problem we are working on. Also, getting an HTTP/1.1 ALPN in the HTTPS RR field for an http:// transfer should imply switching to HTTPS, HSTS style. Which curl currently does not.

DoH

When HTTPS RR is enabled in the curl build, The DoH code asks for an HTTPS record in addition to the A and AAAA records, and if an HTTPS RR answer is returned, curl parses it and stores the retrieved information.

Non-DoH

If DoH is not used for name resolving in an HTTPS RR enabled build, we must provide the ability using the regular resolver backends. We use the c-ares DNS library for the HTTPS RR lookup. Version 1.28.0 or greater.

c-ares

If curl is built to use the c-ares library for name resolves, an HTTPS RR enabled build makes a request for the HTTPS RR in addition to the regular lookup.

Threaded resolver

When built to use the threaded resolver, which is the default, an HTTPS RR build still needs a c-ares installation provided so that a separate request for the HTTPS record can be done in parallel to the regular getaddrinfo() call.

This is done by specifying both c-ares and threaded resolver to configure:

./configure --enable-ares=... --enable-threaded-resolver

or to cmake:

cmake -DENABLE_ARES=ON -DENABLE_THREADED_RESOLVER=ON

Because the HTTPS record is handled separately from the A/AAAA record retrieval, by a separate library, there is a small risk for discrepancies.

When building curl using the threaded resolver with HTTPS RR support (using c-ares), the curl -V output looks exactly like a c-ares resolver build.

HTTPS RR Options

Because curl is a low level transfer tool for which users sometimes want detailed control, we need to offer options to control HTTPS RR use.

1<!--
2Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
4SPDX-License-Identifier: curl
5-->
7# HTTPS RR
9[RFC 9460](https://datatracker.ietf.org/doc/html/rfc9460) documents the HTTPS
10DNS Resource Record.
12curl features **experimental** support for HTTPS RR.
14- The ALPN list from the record is parsed and used
15- The ECH field is stored - and used if ECH is enabled in the build
16- The port number is not used (Firefox supports it, Chrome does not)
17- The target name is not used
18- The IP addresses (`Ipv6hints`, `Ipv4hints`) from the HTTPS RR are not used
19- It only supports a single HTTPS RR per hostname
20- Hostnames without A/AAAA records but *with* HTTPS RR fails
21- consider service profiles where the RR provides different addresses for TCP
22 vs QUIC etc
24`HTTPSRR` is listed as a feature in the `curl -V` output if curl contains
25HTTPS RR support. If c-ares is not included in the build, the HTTPS RR support
26is limited to DoH.
28`asyn-rr` is listed as a feature in the `curl -V` output if c-ares is used for
29additional resolves in addition to a "normal" resolve done with the threaded
30resolver.
32The data extracted from the HTTPS RR is stored in the in-memory DNS cache to
33be reused on subsequent uses of the same hostnames.
35## limitations
37We have decided to work on the HTTPS RR support by following what seems to be
38(widely) used, and wait with implementing the details of the record
39that do not seem to be deployed. HTTPS RR is a DNS field with many odd corners
40and complexities and we might as well avoid them if no one seems to want them.
42## build
44 ./configure --enable-httpsrr
46or
48 cmake -DUSE_HTTPSRR=ON
50## ALPN
52The list of ALPN IDs is parsed but may not be completely respected because of
53what the HTTP version preference is set to, which is a problem we are working
54on. Also, getting an `HTTP/1.1` ALPN in the HTTPS RR field for an `http://`
55transfer should imply switching to HTTPS, HSTS style. Which curl currently
56does not.
58## DoH
60When HTTPS RR is enabled in the curl build, The DoH code asks for an HTTPS
61record in addition to the A and AAAA records, and if an HTTPS RR answer is
62returned, curl parses it and stores the retrieved information.
64## Non-DoH
66If DoH is not used for name resolving in an HTTPS RR enabled build, we must
67provide the ability using the regular resolver backends. We use the c-ares DNS
68library for the HTTPS RR lookup. Version 1.28.0 or greater.
70### c-ares
72If curl is built to use the c-ares library for name resolves, an HTTPS RR
73enabled build makes a request for the HTTPS RR in addition to the regular
74lookup.
76### Threaded resolver
78When built to use the threaded resolver, which is the default, an HTTPS RR
79build still needs a c-ares installation provided so that a separate request
80for the HTTPS record can be done in parallel to the regular getaddrinfo()
81call.
83This is done by specifying both c-ares and threaded resolver to configure:
85 ./configure --enable-ares=... --enable-threaded-resolver
87or to cmake:
89 cmake -DENABLE_ARES=ON -DENABLE_THREADED_RESOLVER=ON
91Because the HTTPS record is handled separately from the A/AAAA record
92retrieval, by a separate library, there is a small risk for discrepancies.
94When building curl using the threaded resolver with HTTPS RR support (using
95c-ares), the `curl -V` output looks exactly like a c-ares resolver build.
97## HTTPS RR Options
99Because curl is a low level transfer tool for which users sometimes want
100detailed control, we need to offer options to control HTTPS RR use.
Diagnostics4 git calls · 176 ms
CallStatusSizeTime0176 ms
discover200152 B3 ms
git.upload-pack discover
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=3
git.response.bytes=152
git.server.agent=gitty/1.0
git.server.object_format=sha1
git.server.caps=agent ls-refs fetch object-format
git.server.fetch=shallow filter sideband-all
git.protocol.version=2
git.pkt.frames=8
git.pkt.data=6
git.pkt.flush=2
git.pkt.delim=0
git.pkt.bytes=152
git.pkt.payload.max=34
ls-refs200149 B · hit4 ms
git.upload-pack ls-refs
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=4
git.response.bytes=149
git.response.cache=HIT
git.response.duration_ms=4
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=149
git.pkt.payload.max=78
git.ls_refs.count=2
git.ls_refs.head=refs/heads/master
fetchblob:none · deepen 10200287 KiB · hit25 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=2
git.response.bytes=293470
git.response.cache=HIT
git.response.duration_ms=25
git.fetch.filter=blob:none
git.fetch.filter.requested=tree:0
git.fetch.filter.fallback=blob:none
git.fetch.filter.learned=false
git.fetch.wants=31c911efe5652725e930f4d2e60c9060e5f6646d
git.fetch.haves.count=0
git.fetch.deepen=10
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=10
git.pkt.data=8
git.pkt.flush=1
git.pkt.delim=1
git.pkt.bytes=293470
git.pkt.payload.max=65516
git.sideband.band1.bytes=293354
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=87
git.pack.bytes=293354
git.pack.ofs_delta=0
git.pack.ref_delta=1
git.pack.chain.max=1
git.pack.trailer=6a646bcda6740bab42e321b51998b52ea15146c0
git.pack.full=86
Open pack map
fetch2001.7 KiB147 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=147
git.response.bytes=1712
git.response.cache=MISS
git.response.duration_ms=147
git.fetch.wants=e046e512ea0cecb1a748c4a3167888537c2e1d73
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=1712
git.pkt.payload.max=1691
git.sideband.band1.bytes=1690
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0

Each row is one HTTP round trip ply made to the git server for this page, on one timeline. discover asks what the server speaks, ls-refs lists branch tips, fetch brings objects by id. A hit was served by Workers Cache. Tap a row for its wire details.

Remote
https://5ad2236b39de4c8bd8f6270ea63f3ea9.artifacts.cloudflare.net/git/public-v0/curl.git
Ref
refs/heads/master
Commit
31c911efe5652725e930f4d2e60c9060e5f6646d
cf-ray
a5e1575069c690ce
Request
eb542121…
Protocol
v2 · sha1
Agent
gitty/1.0
Filter
advertised
Fetch
shallow filter sideband-all
Capabilities
agent ls-refs fetch object-format