Artifactspublic-v0

curl

master
ply4 git calls28 ms290 KiB3 cachedprotocol v2
162 lines7.0 KiBRaw

TLS Sessions and Tickets

The TLS protocol offers methods of "resuming" a previous "session". A TLS "session" is a negotiated security context across a connection (which may be via TCP or UDP or other transports.)

By "resuming", the TLS protocol means that the security context from before can be fully or partially resurrected when the TLS client presents the proper crypto stuff to the server. This saves on the amount of TLS packets that need to be sent back and forth, reducing amount of data and even latency. In the case of QUIC, resumption may send application data without having seen any reply from the server, hence this is named 0-RTT data.

The exact mechanism of session tickets in TLSv1.2 (and earlier) and TLSv1.3 differs. TLSv1.2 tickets have several weaknesses (that can be exploited by attackers) which TLSv1.3 then fixed. See Session Tickets in the real world for an insight into this topic.

These difference between TLS protocol versions are reflected in curl's handling of session tickets. More below.

curl's ssl_peer_key

In order to find a ticket from a previous TLS session, curl needs a name for TLS sessions that uniquely identifies the peer it talks to.

This name has to reflect also the various TLS parameters that can be configured in curl for a connection. We do not want to use a ticket from an different configuration. Example: when setting the maximum TLS version to 1.2, we do not want to reuse a ticket we got from a TLSv1.3 session, although we are talking to the same host.

Internally, we call this name a ssl_peer_key. It is a printable string that carries hostname and port and any non-default TLS parameters involved in the connection.

Examples:

  • https:curl.se:443:KEY-V3:CA-17:/etc/ssl/cert.pem:IMPL-12:GnuTLS/3.8.7:G is a peer key for a connection to https://curl.se:443 using /etc/ssl/cert.pem as CA trust anchors and GnuTLS/3.8.7 as TLS backend.
  • https:curl.se:443:KEY-V3:TLSVER-6-6:CA-17:/etc/ssl/cert.pem:IMPL-12:GnuTLS/3.8.7:G is the same as the previous, except it is configured to use TLSv1.2 as min and max versions.

Variable string fields use NAME-length:value, where the length counts bytes in the value, so delimiters inside a value cannot introduce another field. The :G suffix marks keys that can be used across processes; :L marks local keys. Persisted sessions exported with legacy peer keys no longer match keys in this format and cannot be resumed from those cache entries.

Different configurations produce different keys which is what curl needs when handling SSL session tickets.

One important thing: peer keys do not contain confidential information.

Peer keys carry the hostnames you use curl for. They do leak the privacy of your communication. We recommend to not persist peer keys for this reason.

Caveat: The key may contain filenames or paths. It does not reflect the contents in the file system. If you change /etc/ssl/cert.pem and reuse a previous ticket, curl might trust a server which no longer has a root certificate in the file.

Session Cache Access

Lookups

When a new connection is being established, each SSL connection filter creates its own peer_key and calls into the cache. The cache then looks for a ticket with exactly this peer_key. Peer keys between proxy SSL filters and SSL filters talking through a tunnel differ, as they talk to different peers.

If the connection filter wants to use a client certificate, the cache checks those as well. If the cache peer carries client certs, the connection filter must have those with the same values (and vice versa).

On a match, the connection filter gets the session ticket and feeds that to the TLS implementation which, on accepting it, tries to resume it for a shorter handshake. In addition, the filter gets the ALPN used before and the amount of 0-RTT data that the server announced to be willing to accept. The filter can then decide if it wants to attempt 0-RTT or not. (The ALPN is needed to know if the server speaks the protocol you want to send in 0-RTT. It makes no sense to send HTTP/2 requests to a server that only knows HTTP/1.1.)

Updates

When a new TLS session ticket is received by a filter, it adds it to the cache using its peer_key and SSL configuration. The cache looks for a matching entry and, should it find one, adds the ticket for this peer.

Put, Take and Return

when a filter accesses the session cache, it takes a ticket from the cache, meaning a returned ticket is removed. The filter then configures its TLS backend and returns the ticket to the cache.

The cache needs to treat tickets from TLSv1.2 and 1.3 differently. 1.2 tickets should be reused, but 1.3 tickets SHOULD NOT (RFC 8446). The session cache drops 1.3 tickets when they are returned after use, but keeps a 1.2 ticket.

When a ticket is put into the cache, there is also a difference. There can be several 1.3 tickets at the same time, but only a single 1.2 ticket. TLSv1.2 tickets replace any other. 1.3 tickets accumulate up to a max amount.

By having a "put/take/return" we reflect the 1.3 use case nicely. Two concurrent connections do not reuse the same ticket.

Session Ticket Persistence

Privacy and Security

As mentioned above, SSL peer keys are not intended for storage in a file system. They clearly show which hosts the user talked to. This is not only privacy relevant, but also has security implications as an attacker might find worthy targets among your peer keys.

Also, we do not recommend to persist TLSv1.2 tickets.

Salted Hashes

The TLS session cache offers an alternative to storing peer keys: it provides a salted SHA256 hash of the peer key for import and export.

Export

The salt is generated randomly for each peer key on export. The SHA256 makes sure that the peer key cannot be reversed and that a slightly different key still produces a different result.

This means an attacker cannot "grep" a session file for a particular entry, e.g. if they want to know if you accessed a specific host. They can however compute the SHA256 hashes for all salts in the file and find a specific entry. They cannot find a hostname they do not know. They would have to brute force by guessing.

Import

When session tickets are imported from a file, curl only gets the salted hashes. The imported tickets belong to an unknown peer key.

When a connection filter tries to take a session ticket, it passes its peer key. This peer key initially does not match any tickets in the cache. The cache then checks all entries with unknown peer keys if the passed key matches their salted hash. If it does, the peer key is recovered and remembered at the cache entry.

This is a performance penalty in the order of "unknown" peer keys which diminishes over time when keys are rediscovered. Note that this also works for putting a new ticket into the cache: when no present entry matches, a new one with peer key is created. This peer key then no longer bears the cost of hash computes.

1<!--
2Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
4SPDX-License-Identifier: curl
5-->
7# TLS Sessions and Tickets
9The TLS protocol offers methods of "resuming" a previous "session". A
10TLS "session" is a negotiated security context across a connection
11(which may be via TCP or UDP or other transports.)
13By "resuming", the TLS protocol means that the security context from
14before can be fully or partially resurrected when the TLS client presents
15the proper crypto stuff to the server. This saves on the amount of
16TLS packets that need to be sent back and forth, reducing amount
17of data and even latency. In the case of QUIC, resumption may send
18application data without having seen any reply from the server, hence
19this is named 0-RTT data.
21The exact mechanism of session tickets in TLSv1.2 (and earlier) and
22TLSv1.3 differs. TLSv1.2 tickets have several weaknesses (that can
23be exploited by attackers) which TLSv1.3 then fixed. See
24[Session Tickets in the real world](https://words.filippo.io/we-need-to-talk-about-session-tickets/)
25for an insight into this topic.
27These difference between TLS protocol versions are reflected in curl's
28handling of session tickets. More below.
30## curl's `ssl_peer_key`
32In order to find a ticket from a previous TLS session, curl needs a name for
33TLS sessions that uniquely identifies the peer it talks to.
35This name has to reflect also the various TLS parameters that can be
36configured in curl for a connection. We do not want to use a ticket from an
37different configuration. Example: when setting the maximum TLS version to 1.2,
38we do not want to reuse a ticket we got from a TLSv1.3 session, although we
39are talking to the same host.
41Internally, we call this name a `ssl_peer_key`. It is a printable string that
42carries hostname and port and any non-default TLS parameters involved in the
43connection.
45Examples:
46- `https:curl.se:443:KEY-V3:CA-17:/etc/ssl/cert.pem:IMPL-12:GnuTLS/3.8.7:G` is
47 a peer key for a connection to `https://curl.se:443` using
48 `/etc/ssl/cert.pem` as CA trust anchors and GnuTLS/3.8.7 as TLS backend.
49- `https:curl.se:443:KEY-V3:TLSVER-6-6:CA-17:/etc/ssl/cert.pem:IMPL-12:GnuTLS/3.8.7:G`
50 is the same as the previous, except it is configured to use TLSv1.2 as
51 min and max versions.
53Variable string fields use `NAME-length:value`, where the length counts bytes
54in the value, so delimiters inside a value cannot introduce another field. The
55`:G` suffix marks keys that can be used across processes; `:L` marks local
56keys. Persisted sessions exported with legacy peer keys no longer match keys
57in this format and cannot be resumed from those cache entries.
59Different configurations produce different keys which is what curl needs when
60handling SSL session tickets.
62One important thing: peer keys do not contain confidential information.
64Peer keys carry the hostnames you use curl for. They *do* leak the privacy of
65your communication. We recommend to *not* persist peer keys for this reason.
67**Caveat**: The key may contain filenames or paths. It does not reflect the
68*contents* in the file system. If you change `/etc/ssl/cert.pem` and reuse
69a previous ticket, curl might trust a server which no longer has a root
70certificate in the file.
72## Session Cache Access
74#### Lookups
76When a new connection is being established, each SSL connection filter creates
77its own peer_key and calls into the cache. The cache then looks for a ticket
78with exactly this peer_key. Peer keys between proxy SSL filters and SSL
79filters talking through a tunnel differ, as they talk to different peers.
81If the connection filter wants to use a client certificate, the cache checks
82those as well. If the cache peer carries client certs, the connection filter
83must have those with the same values (and vice versa).
85On a match, the connection filter gets the session ticket and feeds that to
86the TLS implementation which, on accepting it, tries to resume it for a
87shorter handshake. In addition, the filter gets the ALPN used before and the
88amount of 0-RTT data that the server announced to be willing to accept. The
89filter can then decide if it wants to attempt 0-RTT or not. (The ALPN is
90needed to know if the server speaks the protocol you want to send in 0-RTT. It
91makes no sense to send HTTP/2 requests to a server that only knows HTTP/1.1.)
93#### Updates
95When a new TLS session ticket is received by a filter, it adds it to the
96cache using its peer_key and SSL configuration. The cache looks for
97a matching entry and, should it find one, adds the ticket for this
98peer.
100### Put, Take and Return
102when a filter accesses the session cache, it *takes*
103a ticket from the cache, meaning a returned ticket is removed. The filter
104then configures its TLS backend and *returns* the ticket to the cache.
106The cache needs to treat tickets from TLSv1.2 and 1.3 differently. 1.2 tickets
107should be reused, but 1.3 tickets SHOULD NOT (RFC 8446). The session cache
108drops 1.3 tickets when they are returned after use, but keeps a 1.2
109ticket.
111When a ticket is *put* into the cache, there is also a difference. There
112can be several 1.3 tickets at the same time, but only a single 1.2 ticket.
113TLSv1.2 tickets replace any other. 1.3 tickets accumulate up to a max
114amount.
116By having a "put/take/return" we reflect the 1.3 use case nicely. Two
117concurrent connections do not reuse the same ticket.
119## Session Ticket Persistence
121#### Privacy and Security
123As mentioned above, SSL peer keys are not intended for storage in a file
124system. They clearly show which hosts the user talked to. This is not only
125privacy relevant, but also has security implications as an attacker might find
126worthy targets among your peer keys.
128Also, we do not recommend to persist TLSv1.2 tickets.
130### Salted Hashes
132The TLS session cache offers an alternative to storing peer keys:
133it provides a salted SHA256 hash of the peer key for import and export.
135#### Export
137The salt is generated randomly for each peer key on export. The SHA256 makes
138sure that the peer key cannot be reversed and that a slightly different key
139still produces a different result.
141This means an attacker cannot "grep" a session file for a particular entry,
142e.g. if they want to know if you accessed a specific host. They *can* however
143compute the SHA256 hashes for all salts in the file and find a specific entry.
144They *cannot* find a hostname they do not know. They would have to brute force
145by guessing.
147#### Import
149When session tickets are imported from a file, curl only gets the salted
150hashes. The imported tickets belong to an *unknown* peer key.
152When a connection filter tries to *take* a session ticket, it passes its peer
153key. This peer key initially does not match any tickets in the cache. The
154cache then checks all entries with unknown peer keys if the passed key matches
155their salted hash. If it does, the peer key is recovered and remembered at the
156cache entry.
158This is a performance penalty in the order of "unknown" peer keys which
159diminishes over time when keys are rediscovered. Note that this also works for
160putting a new ticket into the cache: when no present entry matches, a new one
161with peer key is created. This peer key then no longer bears the cost of hash
162computes.
Diagnostics4 git calls · 28 ms
CallStatusSizeTime028 ms
discover200152 B3 ms
git.upload-pack discover
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=1
git.response.bytes=152
git.server.agent=gitty/1.0
git.server.object_format=sha1
git.server.caps=agent ls-refs fetch object-format
git.server.fetch=shallow filter sideband-all
git.protocol.version=2
git.pkt.frames=8
git.pkt.data=6
git.pkt.flush=2
git.pkt.delim=0
git.pkt.bytes=152
git.pkt.payload.max=34
ls-refs200149 B · hit2 ms
git.upload-pack ls-refs
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=2
git.response.bytes=149
git.response.cache=HIT
git.response.duration_ms=2
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=149
git.pkt.payload.max=78
git.ls_refs.count=2
git.ls_refs.head=refs/heads/master
fetchblob:none · deepen 10200287 KiB · hit1 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=24
git.response.bytes=293470
git.response.cache=HIT
git.response.duration_ms=1
git.fetch.filter=blob:none
git.fetch.filter.requested=tree:0
git.fetch.filter.fallback=blob:none
git.fetch.filter.learned=false
git.fetch.wants=31c911efe5652725e930f4d2e60c9060e5f6646d
git.fetch.haves.count=0
git.fetch.deepen=10
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=10
git.pkt.data=8
git.pkt.flush=1
git.pkt.delim=1
git.pkt.bytes=293470
git.pkt.payload.max=65516
git.sideband.band1.bytes=293354
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=87
git.pack.bytes=293354
git.pack.ofs_delta=0
git.pack.ref_delta=1
git.pack.chain.max=1
git.pack.trailer=6a646bcda6740bab42e321b51998b52ea15146c0
git.pack.full=86
Open pack map
fetch2003.2 KiB · hit24 ms
git.upload-pack fetch
http.response.status_code=200
git.response.bytes=3301
git.response.cache=HIT
git.response.duration_ms=24
git.fetch.wants=6ff3d83c9dd6820278b504ff87f9423de5e4578b
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=3301
git.pkt.payload.max=3280
git.sideband.band1.bytes=3279
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0

Each row is one HTTP round trip ply made to the git server for this page, on one timeline. discover asks what the server speaks, ls-refs lists branch tips, fetch brings objects by id. A hit was served by Workers Cache. Tap a row for its wire details.

Remote
https://5ad2236b39de4c8bd8f6270ea63f3ea9.artifacts.cloudflare.net/git/public-v0/curl.git
Ref
refs/heads/master
Commit
31c911efe5652725e930f4d2e60c9060e5f6646d
cf-ray
a5e2c9d8dc8f4b67
Request
33f41e86…
Protocol
v2 · sha1
Agent
gitty/1.0
Filter
advertised
Fetch
shallow filter sideband-all
Capabilities
agent ls-refs fetch object-format