Artifactspublic-v0

curl

master
ply4 git calls26 ms292 KiB3 cachedprotocol v2
343 lines14 KiBRaw

curl connection filters

Connection filters is a design in the internals of curl, not visible in its public API. They were added in curl 7.87.0. This document describes the concepts, its high level implementation and the motivations.

Filters

A "connection filter" is a piece of code that is responsible for handling a range of operations of curl's connections: reading, writing, waiting on external events, connecting and closing down - to name the most important ones.

The most important feat of connection filters is that they can be stacked on top of each other (or "chained" if you prefer that metaphor). In the common scenario that you want to retrieve a https: URL with curl, you need 2 basic things to send the request and get the response: a TCP connection, represented by a socket and an SSL instance en- and decrypt over that socket. You write your request to the SSL instance, which encrypts and writes that data to the socket, which then sends the bytes over the network.

With connection filters, curl's internal setup looks something like this (cf for connection filter):

Curl_easy *data         connectdata *conn        cf-ssl        cf-socket
+----------------+      +-----------------+      +-------+     +--------+
|https://curl.se/|----> | properties      |----> | keys  |---> | socket |--> OS --> network
+----------------+      +-----------------+      +-------+     +--------+

Curl_write(data, buffer)
  --> Curl_cfilter_write(data, data->conn, buffer)
        --> conn->filter->write(conn->filter, data, buffer)

While connection filters all do different things, they look the same from the "outside". The code in data and conn does not really know which filters are installed. conn writes into the first filter, whatever that is.

Same is true for filters. Each filter has a pointer to the next filter. When SSL has encrypted the data, it does not write to a socket, it writes to the next filter. If that is indeed a socket, or a file, or an HTTP/2 connection is of no concern to the SSL filter.

This allows stacking, as in:

Direct:
  http://localhost/      conn -> cf-socket
  https://curl.se/       conn -> cf-ssl -> cf-socket
Via http proxy tunnel:
  http://localhost/      conn -> cf-http-proxy -> cf-socket
  https://curl.se/       conn -> cf-ssl -> cf-http-proxy -> cf-socket
Via https proxy tunnel:
  http://localhost/      conn -> cf-http-proxy -> cf-ssl -> cf-socket
  https://curl.se/       conn -> cf-ssl -> cf-http-proxy -> cf-ssl -> cf-socket
Via http proxy tunnel via SOCKS proxy:
  http://localhost/      conn -> cf-http-proxy -> cf-socks -> cf-socket

Connecting/Closing

Before Curl_easy can send the request, the connection needs to be established. This means that all connection filters have done, whatever they need to do: waiting for the socket to be connected, doing the TLS handshake, performing the HTTP tunnel request, etc. This has to be done in reverse order: the last filter has to do its connect first, then the one above can start, etc.

Each filter does in principle the following:

c
static CURLcode myfilter_cf_connect(struct Curl_cfilter *cf,
                                    struct Curl_easy *data,
                                    bool *done)
{
  CURLcode result;

  if(cf->connected) {            /* we and all below are done */
    *done = TRUE;
    return CURLE_OK;
  }
                                 /* Let the filters below connect */
  result = cf->next->cft->connect(cf->next, data, blocking, done);
  if(result || !*done)
    return result;               /* below errored/not finished yet */

  /* MYFILTER CONNECT THINGS */  /* below connected, do out thing */
  *done = cf->connected = TRUE;  /* done, remember, return */
  return CURLE_OK;
}

Closing a connection then works similar. The conn tells the first filter to close. Contrary to connecting, the filter does its own things first, before telling the next filter to close.

Efficiency

There are two things curl is concerned about: efficient memory use and fast transfers.

The memory footprint of a filter is relatively small:

c
struct Curl_cfilter {
  const struct Curl_cftype *cft; /* the type providing implementation */
  struct Curl_cfilter *next;     /* next filter in chain */
  void *ctx;                     /* filter type specific settings */
  struct connectdata *conn;      /* the connection this filter belongs to */
  int sockindex;                 /* TODO: like to get rid off this */
  BIT(connected);                /* != 0 iff this filter is connected */
};

The filter type cft is a singleton, one static struct for each type of filter. The ctx is where a filter holds its specific data. That varies by filter type. An http-proxy filter keeps the ongoing state of the CONNECT here, free it after its has been established. The SSL filter keeps the SSL* (if OpenSSL is used) here until the connection is closed. This varies.

conn is a reference to the connection this filter belongs to, so nothing extra besides the pointer itself.

Several things, that before were kept in struct connectdata, now goes into the filter->ctx when needed. The memory footprint for connections that do not use an http proxy, or socks, or https is lower.

As to transfer efficiency, writing and reading through a filter comes at near zero cost if the filter does not transform the data. An http proxy or socks filter, once it is connected, passes the calls through. Those filters implementations look like this:

c
ssize_t Curl_cf_def_send(struct Curl_cfilter *cf, struct Curl_easy *data,
                         const void *buf, size_t len, CURLcode *err)
{
  return cf->next->cft->do_send(cf->next, data, buf, len, err);
}

The recv implementation is equivalent.

Filter Types

The currently existing filter types (curl 8.5.0) are:

  • TCP, UDP, UNIX: filters that operate on a socket, providing raw I/O.
  • SOCKET-ACCEPT: special TCP socket that has a socket that has been accept()ed in a listen()
  • SSL: filter that applies TLS en-/decryption and handshake. Manages the underlying TLS backend implementation.
  • HTTP-PROXY, H1-PROXY, H2-PROXY, H3-PROXY: the first manages the connection to an HTTP proxy server and uses the other depending on which ALPN protocol has been negotiated.
  • SOCKS-PROXY: filter for the various SOCKS proxy protocol variations
  • HAPROXY: filter for the protocol of the same name, providing client IP information to a server.
  • HTTP/2: filter for handling multiplexed transfers over an HTTP/2 connection
  • HTTP/3: filter for handling multiplexed transfers over an HTTP/3+QUIC connection
  • HAPPY-EYEBALLS: meta filter that implements IPv4/IPv6 "happy eyeballs". It creates up to 2 sub-filters that race each other for a connection.
  • SETUP: meta filter that manages the creation of sub-filter chains for a specific transport (e.g. TCP or QUIC).
  • HTTPS-CONNECT: meta filter that races a TCP+TLS and a QUIC connection against each other to determine if HTTP/1.1, HTTP/2 or HTTP/3 shall be used for a transfer.

Meta filters are combining other filters for a specific purpose, mostly during connection establishment. Other filters like TCP, UDP and UNIX are only to be found at the end of filter chains. SSL filters provide encryption, of course. Protocol filters change the bytes sent and received.

Filter Flags

Filter types carry flags that inform what they do. These are (for now):

  • CF_TYPE_IP_CONNECT: this filter type talks directly to a server. This does not have to be the server the transfer wants to talk to. For example when a proxy server is used.
  • CF_TYPE_SSL: this filter type provides encryption.
  • CF_TYPE_MULTIPLEX: this filter type can manage multiple transfers in parallel.

Filter types can combine these flags. For example, the HTTP/3 filter types have CF_TYPE_IP_CONNECT, CF_TYPE_SSL and CF_TYPE_MULTIPLEX set.

Flags are useful to extrapolate properties of a connection. To check if a connection is encrypted, libcurl inspect the filter chain in place, top down, for CF_TYPE_SSL. If it finds CF_TYPE_IP_CONNECT before any CF_TYPE_SSL, the connection is not encrypted.

For example, conn1 is for a http: request using a tunnel through an HTTP/2 https: proxy. conn2 is a https: HTTP/2 connection to the same proxy. conn3 uses HTTP/3 without proxy. The filter chains would look like this (simplified):

conn1 --> `HTTP-PROXY` --> `H2-PROXY` --> `SSL` --> `TCP`
flags:                     `IP_CONNECT`   `SSL`     `IP_CONNECT`

conn2 --> `HTTP/2` --> `SSL` --> `HTTP-PROXY` --> `H2-PROXY` --> `SSL` --> `TCP`
flags:                 `SSL`                      `IP_CONNECT`   `SSL`     `IP_CONNECT`

conn3 --> `HTTP/3`
flags:    `SSL|IP_CONNECT`

Inspecting the filter chains, conn1 is seen as unencrypted, since it contains an IP_CONNECT filter before any SSL. conn2 is clearly encrypted as an SSL flagged filter is seen first. conn3 is also encrypted as the SSL flag is checked before the presence of IP_CONNECT.

Similar checks can determine if a connection is multiplexed or not.

Adding CONNECT-UDP support

HTTP/3 on top of HTTP/1.1 (MASQUE CONNECT-UDP):

conn --> HTTP/3 --> CAPSULE --> HTTP-PROXY --> H1-PROXY --> SSL --> HAPPY-EYEBALLS --> TCP

HTTP/3 on top of HTTP/2 (MASQUE CONNECT-UDP):

conn --> HTTP/3 --> CAPSULE --> HTTP-PROXY --> H2-PROXY --> SSL --> HAPPY-EYEBALLS --> TCP

The CAPSULE filter handles RFC 9297 capsule protocol encapsulation and decapsulation of UDP datagrams. It is inserted automatically when the HTTP-PROXY filter completes a successful CONNECT-UDP tunnel.

Adding H3-PROXY support

HTTP/1.1 on top of HTTP/3 (CONNECT over QUIC):

conn --> HTTP/1.1 --> SSL --> HTTP-PROXY --> H3-PROXY --> HAPPY-EYEBALLS --> UDP

HTTP/2 on top of HTTP/3 (CONNECT over QUIC):

conn --> HTTP/2 --> SSL --> HTTP-PROXY --> H3-PROXY --> HAPPY-EYEBALLS --> UDP

HTTP/3 on top of HTTP/3 (MASQUE CONNECT-UDP over QUIC):

conn --> HTTP/3 --> CAPSULE --> HTTP-PROXY --> H3-PROXY --> HAPPY-EYEBALLS --> UDP

Filter Tracing

Filters may make use of special trace macros like CURL_TRC_CF(data, cf, msg, ...). With data being the transfer and cf being the filter instance. These traces are normally not active and their execution is guarded so that they are cheap to ignore.

Users of curl may activate them by adding the name of the filter type to the --trace-config argument. For example, in order to get more detailed tracing of an HTTP/2 request, invoke curl with:

sh
> curl -v --trace-config ids,time,http/2 https://curl.se/

Which gives you trace output with time information, transfer+connection ids and details from the HTTP/2 filter. Filter type names in the trace config are case insensitive. You may use all to enable tracing for all filter types. When using libcurl you may call curl_global_trace(config_string) at the start of your application to enable filter details.

Meta Filters

Meta filters is a catch-all name for filter types that do not change the transfer data in any way but provide other important services to curl. In general, it is possible to do all sorts of silly things with them. One of the commonly used, important things is "eyeballing".

The HAPPY-EYEBALLS filter is involved in the connect phase. Its job is to try the various IPv4 and IPv6 addresses that are known for a server. If only one address family is known (or configured), it tries the addresses one after the other with timeouts calculated from the amount of addresses and the overall connect timeout.

When more than one address family is to be tried, it splits the address list into IPv4 and IPv6 and makes parallel attempts. The connection filter chain looks like this:

* create connection for http://curl.se/
conn[curl.se] --> SETUP[TCP] --> HAPPY-EYEBALLS --> NULL
* start connect
conn[curl.se] --> SETUP[TCP] --> HAPPY-EYEBALLS --> NULL
                                 - ballerv4 --> TCP[151.101.1.91]:443
                                 - ballerv6 --> TCP[2a04:4e42:c00::347]:443
* v6 answers, connected
conn[curl.se] --> SETUP[TCP] --> HAPPY-EYEBALLS --> TCP[2a04:4e42:c00::347]:443
* transfer

The modular design of connection filters and that we can plug them into each other is used to control the parallel attempts. When a TCP filter does not connect (in time), it is torn down and another one is created for the next address. This keeps the TCP filter simple.

The HAPPY-EYEBALLS on the other hand stays focused on its side of the problem. We can use it also to make other type of connection by giving it another filter type to try to have happy eyeballing for QUIC:

* create connection for --http3-only https://curl.se/
conn[curl.se] --> SETUP[QUIC] --> HAPPY-EYEBALLS --> NULL
* start connect
conn[curl.se] --> SETUP[QUIC] --> HAPPY-EYEBALLS --> NULL
                                  - ballerv4 --> HTTP/3[151.101.1.91]:443
                                  - ballerv6 --> HTTP/3[2a04:4e42:c00::347]:443
* v6 answers, connected
conn[curl.se] --> SETUP[QUIC] --> HAPPY-EYEBALLS --> HTTP/3[2a04:4e42:c00::347]:443
* transfer

When we plug these two variants together, we get the HTTPS-CONNECT filter type that is used for --http3 when both HTTP/3 and HTTP/2 or HTTP/1.1 shall be attempted:

* create connection for --http3 https://curl.se/
conn[curl.se] --> HTTPS-CONNECT --> NULL
* start connect
conn[curl.se] --> HTTPS-CONNECT --> NULL
                  - SETUP[QUIC] --> HAPPY-EYEBALLS --> NULL
                                    - ballerv4 --> HTTP/3[151.101.1.91]:443
                                    - ballerv6 --> HTTP/3[2a04:4e42:c00::347]:443
                  - SETUP[TCP]  --> HAPPY-EYEBALLS --> NULL
                                    - ballerv4 --> TCP[151.101.1.91]:443
                                    - ballerv6 --> TCP[2a04:4e42:c00::347]:443
* v4 QUIC answers, connected
conn[curl.se] --> HTTPS-CONNECT --> SETUP[QUIC] --> HAPPY-EYEBALLS --> HTTP/3[151.101.1.91]:443
* transfer
1<!--
2Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
4SPDX-License-Identifier: curl
5-->
7# curl connection filters
9Connection filters is a design in the internals of curl, not visible in its
10public API. They were added in curl 7.87.0. This document describes the
11concepts, its high level implementation and the motivations.
13## Filters
15A "connection filter" is a piece of code that is responsible for handling a
16range of operations of curl's connections: reading, writing, waiting on
17external events, connecting and closing down - to name the most important
18ones.
20The most important feat of connection filters is that they can be stacked on
21top of each other (or "chained" if you prefer that metaphor). In the common
22scenario that you want to retrieve a `https:` URL with curl, you need 2 basic
23things to send the request and get the response: a TCP connection, represented
24by a `socket` and an SSL instance en- and decrypt over that socket. You write
25your request to the SSL instance, which encrypts and writes that data to the
26socket, which then sends the bytes over the network.
28With connection filters, curl's internal setup looks something like this (cf
29for connection filter):
31```
32Curl_easy *data connectdata *conn cf-ssl cf-socket
33+----------------+ +-----------------+ +-------+ +--------+
34|https://curl.se/|----> | properties |----> | keys |---> | socket |--> OS --> network
35+----------------+ +-----------------+ +-------+ +--------+
37Curl_write(data, buffer)
38 --> Curl_cfilter_write(data, data->conn, buffer)
39 --> conn->filter->write(conn->filter, data, buffer)
40```
42While connection filters all do different things, they look the same from the
43"outside". The code in `data` and `conn` does not really know **which**
44filters are installed. `conn` writes into the first filter, whatever that is.
46Same is true for filters. Each filter has a pointer to the `next` filter. When
47SSL has encrypted the data, it does not write to a socket, it writes to the
48next filter. If that is indeed a socket, or a file, or an HTTP/2 connection is
49of no concern to the SSL filter.
51This allows stacking, as in:
53```
54Direct:
55 http://localhost/ conn -> cf-socket
56 https://curl.se/ conn -> cf-ssl -> cf-socket
57Via http proxy tunnel:
58 http://localhost/ conn -> cf-http-proxy -> cf-socket
59 https://curl.se/ conn -> cf-ssl -> cf-http-proxy -> cf-socket
60Via https proxy tunnel:
61 http://localhost/ conn -> cf-http-proxy -> cf-ssl -> cf-socket
62 https://curl.se/ conn -> cf-ssl -> cf-http-proxy -> cf-ssl -> cf-socket
63Via http proxy tunnel via SOCKS proxy:
64 http://localhost/ conn -> cf-http-proxy -> cf-socks -> cf-socket
65```
67### Connecting/Closing
69Before `Curl_easy` can send the request, the connection needs to be
70established. This means that all connection filters have done, whatever they
71need to do: waiting for the socket to be connected, doing the TLS handshake,
72performing the HTTP tunnel request, etc. This has to be done in reverse order:
73the last filter has to do its connect first, then the one above can start,
74etc.
76Each filter does in principle the following:
78```c
79static CURLcode myfilter_cf_connect(struct Curl_cfilter *cf,
80 struct Curl_easy *data,
81 bool *done)
83 CURLcode result;
85 if(cf->connected) { /* we and all below are done */
86 *done = TRUE;
87 return CURLE_OK;
88 }
89 /* Let the filters below connect */
90 result = cf->next->cft->connect(cf->next, data, blocking, done);
91 if(result || !*done)
92 return result; /* below errored/not finished yet */
94 /* MYFILTER CONNECT THINGS */ /* below connected, do out thing */
95 *done = cf->connected = TRUE; /* done, remember, return */
96 return CURLE_OK;
98```
100Closing a connection then works similar. The `conn` tells the first filter to
101close. Contrary to connecting, the filter does its own things first, before
102telling the next filter to close.
104### Efficiency
106There are two things curl is concerned about: efficient memory use and fast
107transfers.
109The memory footprint of a filter is relatively small:
111```c
112struct Curl_cfilter {
113 const struct Curl_cftype *cft; /* the type providing implementation */
114 struct Curl_cfilter *next; /* next filter in chain */
115 void *ctx; /* filter type specific settings */
116 struct connectdata *conn; /* the connection this filter belongs to */
117 int sockindex; /* TODO: like to get rid off this */
118 BIT(connected); /* != 0 iff this filter is connected */
119};
120```
122The filter type `cft` is a singleton, one static struct for each type of
123filter. The `ctx` is where a filter holds its specific data. That varies by
124filter type. An http-proxy filter keeps the ongoing state of the CONNECT here,
125free it after its has been established. The SSL filter keeps the `SSL*` (if
126OpenSSL is used) here until the connection is closed. This varies.
128`conn` is a reference to the connection this filter belongs to, so nothing
129extra besides the pointer itself.
131Several things, that before were kept in `struct connectdata`, now goes into
132the `filter->ctx` *when needed*. The memory footprint for connections that do
133*not* use an http proxy, or socks, or https is lower.
135As to transfer efficiency, writing and reading through a filter comes at near
136zero cost *if the filter does not transform the data*. An http proxy or socks
137filter, once it is connected, passes the calls through. Those filters
138implementations look like this:
140```c
141ssize_t Curl_cf_def_send(struct Curl_cfilter *cf, struct Curl_easy *data,
142 const void *buf, size_t len, CURLcode *err)
144 return cf->next->cft->do_send(cf->next, data, buf, len, err);
146```
148The `recv` implementation is equivalent.
150## Filter Types
152The currently existing filter types (curl 8.5.0) are:
154* `TCP`, `UDP`, `UNIX`: filters that operate on a socket, providing raw I/O.
155* `SOCKET-ACCEPT`: special TCP socket that has a socket that has been
156 `accept()`ed in a `listen()`
157* `SSL`: filter that applies TLS en-/decryption and handshake. Manages the
158 underlying TLS backend implementation.
159* `HTTP-PROXY`, `H1-PROXY`, `H2-PROXY`, `H3-PROXY`: the first manages the
160 connection to an HTTP proxy server and uses the other depending on which
161 ALPN protocol has been negotiated.
162* `SOCKS-PROXY`: filter for the various SOCKS proxy protocol variations
163* `HAPROXY`: filter for the protocol of the same name, providing client IP
164 information to a server.
165* `HTTP/2`: filter for handling multiplexed transfers over an HTTP/2
166 connection
167* `HTTP/3`: filter for handling multiplexed transfers over an HTTP/3+QUIC
168 connection
169* `HAPPY-EYEBALLS`: meta filter that implements IPv4/IPv6 "happy eyeballs".
170 It creates up to 2 sub-filters that race each other for a connection.
171* `SETUP`: meta filter that manages the creation of sub-filter chains for a
172 specific transport (e.g. TCP or QUIC).
173* `HTTPS-CONNECT`: meta filter that races a TCP+TLS and a QUIC connection
174 against each other to determine if HTTP/1.1, HTTP/2 or HTTP/3 shall be used
175 for a transfer.
177Meta filters are combining other filters for a specific purpose, mostly during
178connection establishment. Other filters like `TCP`, `UDP` and `UNIX` are only
179to be found at the end of filter chains. SSL filters provide encryption, of
180course. Protocol filters change the bytes sent and received.
182## Filter Flags
184Filter types carry flags that inform what they do. These are (for now):
186* `CF_TYPE_IP_CONNECT`: this filter type talks directly to a server. This does
187 not have to be the server the transfer wants to talk to. For example when a
188 proxy server is used.
189* `CF_TYPE_SSL`: this filter type provides encryption.
190* `CF_TYPE_MULTIPLEX`: this filter type can manage multiple transfers in parallel.
192Filter types can combine these flags. For example, the HTTP/3 filter types
193have `CF_TYPE_IP_CONNECT`, `CF_TYPE_SSL` and `CF_TYPE_MULTIPLEX` set.
195Flags are useful to extrapolate properties of a connection. To check if a
196connection is encrypted, libcurl inspect the filter chain in place, top down,
197for `CF_TYPE_SSL`. If it finds `CF_TYPE_IP_CONNECT` before any `CF_TYPE_SSL`,
198the connection is not encrypted.
200For example, `conn1` is for a `http:` request using a tunnel through an HTTP/2
201`https:` proxy. `conn2` is a `https:` HTTP/2 connection to the same proxy.
202`conn3` uses HTTP/3 without proxy. The filter chains would look like this
203(simplified):
205```
206conn1 --> `HTTP-PROXY` --> `H2-PROXY` --> `SSL` --> `TCP`
207flags: `IP_CONNECT` `SSL` `IP_CONNECT`
209conn2 --> `HTTP/2` --> `SSL` --> `HTTP-PROXY` --> `H2-PROXY` --> `SSL` --> `TCP`
210flags: `SSL` `IP_CONNECT` `SSL` `IP_CONNECT`
212conn3 --> `HTTP/3`
213flags: `SSL|IP_CONNECT`
214```
216Inspecting the filter chains, `conn1` is seen as unencrypted, since it
217contains an `IP_CONNECT` filter before any `SSL`. `conn2` is clearly encrypted
218as an `SSL` flagged filter is seen first. `conn3` is also encrypted as the
219`SSL` flag is checked before the presence of `IP_CONNECT`.
221Similar checks can determine if a connection is multiplexed or not.
223## Adding CONNECT-UDP support
224HTTP/3 on top of HTTP/1.1 (MASQUE CONNECT-UDP):
225```
226conn --> HTTP/3 --> CAPSULE --> HTTP-PROXY --> H1-PROXY --> SSL --> HAPPY-EYEBALLS --> TCP
227```
229HTTP/3 on top of HTTP/2 (MASQUE CONNECT-UDP):
230```
231conn --> HTTP/3 --> CAPSULE --> HTTP-PROXY --> H2-PROXY --> SSL --> HAPPY-EYEBALLS --> TCP
232```
234The CAPSULE filter handles RFC 9297 capsule protocol encapsulation and
235decapsulation of UDP datagrams. It is inserted automatically when the
236HTTP-PROXY filter completes a successful CONNECT-UDP tunnel.
238## Adding H3-PROXY support
239HTTP/1.1 on top of HTTP/3 (CONNECT over QUIC):
240```
241conn --> HTTP/1.1 --> SSL --> HTTP-PROXY --> H3-PROXY --> HAPPY-EYEBALLS --> UDP
242```
244HTTP/2 on top of HTTP/3 (CONNECT over QUIC):
245```
246conn --> HTTP/2 --> SSL --> HTTP-PROXY --> H3-PROXY --> HAPPY-EYEBALLS --> UDP
247```
249HTTP/3 on top of HTTP/3 (MASQUE CONNECT-UDP over QUIC):
250```
251conn --> HTTP/3 --> CAPSULE --> HTTP-PROXY --> H3-PROXY --> HAPPY-EYEBALLS --> UDP
252```
254## Filter Tracing
256Filters may make use of special trace macros like `CURL_TRC_CF(data, cf, msg,
257...)`. With `data` being the transfer and `cf` being the filter instance.
258These traces are normally not active and their execution is guarded so that
259they are cheap to ignore.
261Users of `curl` may activate them by adding the name of the filter type to the
262`--trace-config` argument. For example, in order to get more detailed tracing
263of an HTTP/2 request, invoke curl with:
265```sh
266> curl -v --trace-config ids,time,http/2 https://curl.se/
267```
269Which gives you trace output with time information, transfer+connection ids
270and details from the `HTTP/2` filter. Filter type names in the trace config
271are case insensitive. You may use `all` to enable tracing for all filter
272types. When using `libcurl` you may call `curl_global_trace(config_string)` at
273the start of your application to enable filter details.
275## Meta Filters
277Meta filters is a catch-all name for filter types that do not change the
278transfer data in any way but provide other important services to curl. In
279general, it is possible to do all sorts of silly things with them. One of the
280commonly used, important things is "eyeballing".
282The `HAPPY-EYEBALLS` filter is involved in the connect phase. Its job is to
283try the various IPv4 and IPv6 addresses that are known for a server. If only
284one address family is known (or configured), it tries the addresses one after
285the other with timeouts calculated from the amount of addresses and the
286overall connect timeout.
288When more than one address family is to be tried, it splits the address list
289into IPv4 and IPv6 and makes parallel attempts. The connection filter chain
290looks like this:
292```
293* create connection for http://curl.se/
294conn[curl.se] --> SETUP[TCP] --> HAPPY-EYEBALLS --> NULL
295* start connect
296conn[curl.se] --> SETUP[TCP] --> HAPPY-EYEBALLS --> NULL
297 - ballerv4 --> TCP[151.101.1.91]:443
298 - ballerv6 --> TCP[2a04:4e42:c00::347]:443
299* v6 answers, connected
300conn[curl.se] --> SETUP[TCP] --> HAPPY-EYEBALLS --> TCP[2a04:4e42:c00::347]:443
301* transfer
302```
304The modular design of connection filters and that we can plug them into each
305other is used to control the parallel attempts. When a `TCP` filter does not
306connect (in time), it is torn down and another one is created for the next
307address. This keeps the `TCP` filter simple.
309The `HAPPY-EYEBALLS` on the other hand stays focused on its side of the
310problem. We can use it also to make other type of connection by giving it
311another filter type to try to have happy eyeballing for QUIC:
313```
314* create connection for --http3-only https://curl.se/
315conn[curl.se] --> SETUP[QUIC] --> HAPPY-EYEBALLS --> NULL
316* start connect
317conn[curl.se] --> SETUP[QUIC] --> HAPPY-EYEBALLS --> NULL
318 - ballerv4 --> HTTP/3[151.101.1.91]:443
319 - ballerv6 --> HTTP/3[2a04:4e42:c00::347]:443
320* v6 answers, connected
321conn[curl.se] --> SETUP[QUIC] --> HAPPY-EYEBALLS --> HTTP/3[2a04:4e42:c00::347]:443
322* transfer
323```
325When we plug these two variants together, we get the `HTTPS-CONNECT` filter
326type that is used for `--http3` when **both** HTTP/3 and HTTP/2 or HTTP/1.1
327shall be attempted:
329```
330* create connection for --http3 https://curl.se/
331conn[curl.se] --> HTTPS-CONNECT --> NULL
332* start connect
333conn[curl.se] --> HTTPS-CONNECT --> NULL
334 - SETUP[QUIC] --> HAPPY-EYEBALLS --> NULL
335 - ballerv4 --> HTTP/3[151.101.1.91]:443
336 - ballerv6 --> HTTP/3[2a04:4e42:c00::347]:443
337 - SETUP[TCP] --> HAPPY-EYEBALLS --> NULL
338 - ballerv4 --> TCP[151.101.1.91]:443
339 - ballerv6 --> TCP[2a04:4e42:c00::347]:443
340* v4 QUIC answers, connected
341conn[curl.se] --> HTTPS-CONNECT --> SETUP[QUIC] --> HAPPY-EYEBALLS --> HTTP/3[151.101.1.91]:443
342* transfer
343```
Diagnostics4 git calls · 26 ms
CallStatusSizeTime026 ms
discover200152 B2 ms
git.upload-pack discover
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=3
git.response.bytes=152
git.server.agent=gitty/1.0
git.server.object_format=sha1
git.server.caps=agent ls-refs fetch object-format
git.server.fetch=shallow filter sideband-all
git.protocol.version=2
git.pkt.frames=8
git.pkt.data=6
git.pkt.flush=2
git.pkt.delim=0
git.pkt.bytes=152
git.pkt.payload.max=34
ls-refs200149 B · hit1 ms
git.upload-pack ls-refs
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=1
git.response.bytes=149
git.response.cache=HIT
git.response.duration_ms=1
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=149
git.pkt.payload.max=78
git.ls_refs.count=2
git.ls_refs.head=refs/heads/master
fetchblob:none · deepen 10200287 KiB · hit3 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=20
git.response.bytes=293470
git.response.cache=HIT
git.response.duration_ms=3
git.fetch.filter=blob:none
git.fetch.filter.requested=tree:0
git.fetch.filter.fallback=blob:none
git.fetch.filter.learned=false
git.fetch.wants=31c911efe5652725e930f4d2e60c9060e5f6646d
git.fetch.haves.count=0
git.fetch.deepen=10
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=10
git.pkt.data=8
git.pkt.flush=1
git.pkt.delim=1
git.pkt.bytes=293470
git.pkt.payload.max=65516
git.sideband.band1.bytes=293354
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=87
git.pack.bytes=293354
git.pack.ofs_delta=0
git.pack.ref_delta=1
git.pack.chain.max=1
git.pack.trailer=6a646bcda6740bab42e321b51998b52ea15146c0
git.pack.full=86
Open pack map
fetch2005.2 KiB · hit21 ms
git.upload-pack fetch
http.response.status_code=200
git.response.bytes=5357
git.response.cache=HIT
git.response.duration_ms=21
git.fetch.wants=515624a9fad5ec1374fb2056ac543238996bac8c
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=5357
git.pkt.payload.max=5336
git.sideband.band1.bytes=5335
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0

Each row is one HTTP round trip ply made to the git server for this page, on one timeline. discover asks what the server speaks, ls-refs lists branch tips, fetch brings objects by id. A hit was served by Workers Cache. Tap a row for its wire details.

Remote
https://5ad2236b39de4c8bd8f6270ea63f3ea9.artifacts.cloudflare.net/git/public-v0/curl.git
Ref
refs/heads/master
Commit
31c911efe5652725e930f4d2e60c9060e5f6646d
cf-ray
a5e36fbd18f2e88e
Request
67249f31…
Protocol
v2 · sha1
Agent
gitty/1.0
Filter
advertised
Fetch
shallow filter sideband-all
Capabilities
agent ls-refs fetch object-format