Artifactspublic-v0

curl

master
ply4 git calls255 ms289 KiB2 cachedprotocol v2
127 lines5.3 KiBRaw

Multi Event Based

A libcurl multi is operating "event based" when the application uses and event library like libuv to monitor the sockets and file descriptors libcurl uses to trigger transfer operations. How that works from the applications point of view is described in libcurl-multi(3).

This documents is about the internal handling.

Source Locations

All code related to event based handling is found in lib/multi_ev.c and lib/multi_ev.h. The header defines a set of internal functions and struct curl_multi_ev that is embedded in each multi handle.

There is Curl_multi_ev_init() and Curl_multi_ev_cleanup() to manage the overall life cycle, call on creation and destruction of the multi handle.

Tracking Events

First, the various functions in lib/multi_ev.h only ever really do something when the libcurl application has registered its callback in multi->socket_cb.

This is important as this callback gets informed about changes to sockets. When a new socket is added, an existing is removed, or the POLLIN/OUT flags change, multi->socket_cb needs to be invoked. multi_ev has to track what it already reported to detect changes.

Most applications are expected to go "event based" right from the start, but the libcurl API does not prohibit an application to start another way and then go for events later on, even in the middle of a transfer.

Transfer Events

Most event that happen are in connection with a transfer. A transfer opens a connection, which opens a socket, and waits for this socket to become writable (POLLOUT) when using TCP, for example.

The multi then calls Curl_multi_ev_assess_xfer(multi, data) to let the multi event code detect what sockets the transfer is interested in. If indeed a multi->socket_cb is set, the current transfer pollset is retrieved via Curl_multi_getsock(). This current pollset is then compared to the previous pollset. If relevant changes are detected, multi->socket_cb gets informed about those. These can be:

  • a socket is in the current set, but not the previous one
  • a socket was also in the previous one, but IN/OUT flags changed
  • a socket in the previous one is no longer part of the current

multi_ev.c keeps a struct mev_sh_entry for each socket in a hash with the socket as key. It tracks in each entry which transfers are interested in this particular socket, how many transfers want to read and/or write and the summarized POLLIN/POLLOUT action reported to multi->socket_cb.

This is necessary as a socket may be in use by several transfers at the same time (think HTTP/2 on the same connection). When a transfer is done and gets removed from the socket entry, it decrements the reader and/or writer count (depending on what it was last interested in). This may result in the entry's summarized action to change, or not.

Connection Events

There are also events not connected to any transfer that need to be tracked. The multi connection cache, concerned with clean shutdowns of connections, is interested in socket events during the shutdown.

To allow use of the libcurl infrastructure, the connection cache operates using an internal easy handle that is not a transfer as such. The internal handle is used for all connection shutdown operations, being tied to a particular connection only for a short time. This means tracking the last pollset for an internal handle is useless.

Instead, the connection cache uses Curl_multi_ev_assess_conn() to have multi event handling check the connection and track a "last pollset" for the connection alone.

Event Processing

When the libcurl application is informed by the event library that a particular socket has an event, it calls curl_multi_socket_action() to make libcurl react to it. This internally invokes Curl_multi_ev_expire_xfers() which expires all transfers that are interested in the given socket, so the multi handle runs them.

In addition Curl_multi_ev_expire_xfers() returns a bool to let the multi know that connections are also interested in the socket, so the connection pool should be informed as well.

All Things Pass

When a transfer is done, e.g. removed from its multi handle, the multi calls Curl_multi_ev_xfer_done(). This cleans up the pollset tracking for the transfer.

When a connection is done, and before it is destroyed, Curl_multi_ev_conn_done() is called. This cleans up the pollset tracking for this connection.

When a socket is about to be closed, Curl_multi_ev_socket_done() is called to cleanup the socket entry and all information kept there.

These calls do not have to happen in any particular order. A transfer's socket may be around while the transfer is ongoing. Or it might disappear in the middle of things. Also, a transfer might be interested in several sockets at the same time (resolving, eye balling, ftp are all examples of those).

Come Again

While transfer and connection identifiers are practically unique in a libcurl application, sockets are not. Operating systems are keen on reusing their resources, and the next socket may get the same identifier as a recently closed one with high likelihood.

This means that multi event handling needs to be informed before a close, clean up all its tracking and be ready to see that same socket identifier again right after.

1<!--
2Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
4SPDX-License-Identifier: curl
5-->
7# Multi Event Based
9A libcurl multi is operating "event based" when the application uses
10and event library like `libuv` to monitor the sockets and file descriptors
11libcurl uses to trigger transfer operations. How that works from the
12applications point of view is described in libcurl-multi(3).
14This documents is about the internal handling.
16## Source Locations
18All code related to event based handling is found in `lib/multi_ev.c`
19and `lib/multi_ev.h`. The header defines a set of internal functions
20and `struct curl_multi_ev` that is embedded in each multi handle.
22There is `Curl_multi_ev_init()` and `Curl_multi_ev_cleanup()` to manage
23the overall life cycle, call on creation and destruction of the multi
24handle.
26## Tracking Events
28First, the various functions in `lib/multi_ev.h` only ever really do
29something when the libcurl application has registered its callback
30in `multi->socket_cb`.
32This is important as this callback gets informed about *changes* to sockets.
33When a new socket is added, an existing is removed, or the `POLLIN/OUT`
34flags change, `multi->socket_cb` needs to be invoked. `multi_ev` has to
35track what it already reported to detect changes.
37Most applications are expected to go "event based" right from the start,
38but the libcurl API does not prohibit an application to start another
39way and then go for events later on, even in the middle of a transfer.
41### Transfer Events
43Most event that happen are in connection with a transfer. A transfer
44opens a connection, which opens a socket, and waits for this socket
45to become writable (`POLLOUT`) when using TCP, for example.
47The multi then calls `Curl_multi_ev_assess_xfer(multi, data)` to
48let the multi event code detect what sockets the transfer is interested in.
49If indeed a `multi->socket_cb` is set, the *current* transfer pollset is
50retrieved via `Curl_multi_getsock()`. This current pollset is then
51compared to the *previous* pollset. If relevant changes are detected,
52`multi->socket_cb` gets informed about those. These can be:
54 * a socket is in the current set, but not the previous one
55 * a socket was also in the previous one, but IN/OUT flags changed
56 * a socket in the previous one is no longer part of the current
58`multi_ev.c` keeps a `struct mev_sh_entry` for each socket in a hash
59with the socket as key. It tracks in each entry which transfers are
60interested in this particular socket, how many transfers want to read
61and/or write and the summarized `POLLIN/POLLOUT` action reported to
62`multi->socket_cb`.
64This is necessary as a socket may be in use by several transfers
65at the same time (think HTTP/2 on the same connection). When a transfer
66is done and gets removed from the socket entry, it decrements
67the reader and/or writer count (depending on what it was last
68interested in). This *may* result in the entry's summarized action
69to change, or not.
71### Connection Events
73There are also events not connected to any transfer that need to be tracked.
74The multi connection cache, concerned with clean shutdowns of connections,
75is interested in socket events during the shutdown.
77To allow use of the libcurl infrastructure, the connection cache operates
78using an *internal* easy handle that is not a transfer as such. The
79internal handle is used for all connection shutdown operations, being tied
80to a particular connection only for a short time. This means tracking
81the last pollset for an internal handle is useless.
83Instead, the connection cache uses `Curl_multi_ev_assess_conn()` to have
84multi event handling check the connection and track a "last pollset"
85for the connection alone.
87## Event Processing
89When the libcurl application is informed by the event library that
90a particular socket has an event, it calls `curl_multi_socket_action()`
91to make libcurl react to it. This internally invokes
92`Curl_multi_ev_expire_xfers()` which expires all transfers that
93are interested in the given socket, so the multi handle runs them.
95In addition `Curl_multi_ev_expire_xfers()` returns a `bool` to let
96the multi know that connections are also interested in the socket, so
97the connection pool should be informed as well.
99## All Things Pass
101When a transfer is done, e.g. removed from its multi handle, the
102multi calls `Curl_multi_ev_xfer_done()`. This cleans up the pollset
103tracking for the transfer.
105When a connection is done, and before it is destroyed,
106`Curl_multi_ev_conn_done()` is called. This cleans up the pollset
107tracking for this connection.
109When a socket is about to be closed, `Curl_multi_ev_socket_done()`
110is called to cleanup the socket entry and all information kept there.
112These calls do not have to happen in any particular order. A transfer's
113socket may be around while the transfer is ongoing. Or it might disappear
114in the middle of things. Also, a transfer might be interested in several
115sockets at the same time (resolving, eye balling, ftp are all examples of
116those).
118### Come Again
120While transfer and connection identifiers are practically unique in a libcurl
121application, sockets are not. Operating systems are keen on reusing their
122resources, and the next socket may get the same identifier as a recently
123closed one with high likelihood.
125This means that multi event handling needs to be informed *before* a close,
126clean up all its tracking and be ready to see that same socket identifier
127again right after.
Diagnostics4 git calls · 255 ms
CallStatusSizeTime0255 ms
discover200152 B2 ms
git.upload-pack discover
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=2
git.response.bytes=152
git.server.agent=gitty/1.0
git.server.object_format=sha1
git.server.caps=agent ls-refs fetch object-format
git.server.fetch=shallow filter sideband-all
git.protocol.version=2
git.pkt.frames=8
git.pkt.data=6
git.pkt.flush=2
git.pkt.delim=0
git.pkt.bytes=152
git.pkt.payload.max=34
ls-refs200149 B · hit2 ms
git.upload-pack ls-refs
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=2
git.response.bytes=149
git.response.cache=HIT
git.response.duration_ms=2
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=149
git.pkt.payload.max=78
git.ls_refs.count=2
git.ls_refs.head=refs/heads/master
fetchblob:none · deepen 10200287 KiB · hit21 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=2
git.response.bytes=293470
git.response.cache=HIT
git.response.duration_ms=21
git.fetch.filter=blob:none
git.fetch.filter.requested=tree:0
git.fetch.filter.fallback=blob:none
git.fetch.filter.learned=false
git.fetch.wants=31c911efe5652725e930f4d2e60c9060e5f6646d
git.fetch.haves.count=0
git.fetch.deepen=10
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=10
git.pkt.data=8
git.pkt.flush=1
git.pkt.delim=1
git.pkt.bytes=293470
git.pkt.payload.max=65516
git.sideband.band1.bytes=293354
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=87
git.pack.bytes=293354
git.pack.ofs_delta=0
git.pack.ref_delta=1
git.pack.chain.max=1
git.pack.trailer=6a646bcda6740bab42e321b51998b52ea15146c0
git.pack.full=86
Open pack map
fetch2002.3 KiB232 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=232
git.response.bytes=2365
git.response.cache=MISS
git.response.duration_ms=232
git.fetch.wants=cc899faf3624c573485ac88f66c190179d795f49
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=2365
git.pkt.payload.max=2344
git.sideband.band1.bytes=2343
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0

Each row is one HTTP round trip ply made to the git server for this page, on one timeline. discover asks what the server speaks, ls-refs lists branch tips, fetch brings objects by id. A hit was served by Workers Cache. Tap a row for its wire details.

Remote
https://5ad2236b39de4c8bd8f6270ea63f3ea9.artifacts.cloudflare.net/git/public-v0/curl.git
Ref
refs/heads/master
Commit
31c911efe5652725e930f4d2e60c9060e5f6646d
cf-ray
a5e31bd9991d8682
Request
aa5c0be1…
Protocol
v2 · sha1
Agent
gitty/1.0
Filter
advertised
Fetch
shallow filter sideband-all
Capabilities
agent ls-refs fetch object-format