Artifactspublic-v0

curl

master
ply4 git calls410 ms289 KiB2 cachedprotocol v2
150 lines5.8 KiBRaw

IPFS

For an overview about IPFS, visit the IPFS project site.

In IPFS there are two protocols. IPFS and IPNS (their workings are explained in detail here). The ideal way to access data on the IPFS network is through those protocols. For example to access the Big Buck Bunny video the ideal way to access it is like: ipfs://bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

IPFS Gateways

IPFS Gateway acts as a bridge between traditional HTTP clients and IPFS. IPFS Gateway specifications of HTTP semantics can be found here.

Deserialized responses

By default, a gateway acts as a bridge between traditional HTTP clients and IPFS and performs necessary hash verification and deserialization. Through such gateway, users can download files, directories, and other content-addressed data stored with IPFS or IPNS as if they were stored in a traditional web server.

Verifiable responses

By explicitly requesting application/vnd.ipld.raw or application/vnd.ipld.car responses, by means defined in Trustless Gateway Specification, the user is able to fetch raw content-addressed data and perform hash verification themselves.

This enables users to use untrusted, public gateways without worrying they might return invalid/malicious bytes.

IPFS and IPNS protocol handling

There are various ways to access data from the IPFS network. One such way is through the concept of public "gateways". The short version is that entities can offer gateway services. An example here that is hosted by Protocol Labs (who also makes IPFS) is dweb.link and ipfs.io. Both sites expose gateway functionality. Getting a file through ipfs.io looks like this: https://ipfs.io/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

If you were to be running your own IPFS node then you, by default, also have a local gateway running. In its default configuration the earlier example would then also work in this link:

http://127.0.0.1:8080/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

curl handling of the IPFS protocols

The IPFS integration in curl hides this gateway logic for you. Instead of providing a full URL to a file on IPFS like this:

sh
curl http://127.0.0.1:8080/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

You can provide it with the IPFS protocol instead:

sh
curl ipfs://bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

With the IPFS protocol way of asking a file, curl still needs to know the gateway. curl essentially rewrites the IPFS based URL to a gateway URL.

IPFS_GATEWAY environment variable

If the IPFS_GATEWAY environment variable is found, its value is used as gateway.

Automatic gateway detection

When you provide no additional details to curl then it:

  1. First looks for the IPFS_GATEWAY environment variable and use that if it is set.
  2. Looks for the file: ~/.ipfs/gateway. If it can find that file then it means that you have a local gateway running and that file contains the URL to your local gateway.

If curl fails, you are presented with an error message and a link to this page to the option most applicable to solving the issue.

--ipfs-gateway argument

You can also provide a --ipfs-gateway argument to curl. This overrules any other gateway setting. curl does not fallback to the other options if the provided gateway did not work.

Gateway redirects

A gateway could redirect to another place. For example, dweb.link redirects path based requests to subdomain based ones. A request using:

curl ipfs://bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi --ipfs-gateway https://inbrowser.link/

Which would be translated to:

https://inbrowser.link/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi

redirects to:

https://bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi.ipfs.inbrowser.link/

If you trust this behavior from your gateway of choice then passing the -L option follows the redirect.

Error messages and hints

Depending on the arguments, curl could present the user with an error.

Gateway file and environment variable

curl tried to look for the file: ~/.ipfs/gateway but could not find it. It also tried to look for the IPFS_GATEWAY environment variable but could not find that either. This happens when no extra arguments are passed to curl and letting it try to figure it out automatically.

Any IPFS implementation that has gateway support should expose its URL in ~/.ipfs/gateway. If you are already running a gateway, make sure it exposes the file where curl expects to find it.

Alternatively you could set the IPFS_GATEWAY environment variable or pass the --ipfs-gateway flag to the curl command.

Malformed gateway URL

The command executed evaluates in an invalid URL. This could be anywhere in the URL, but a likely point is a wrong gateway URL.

Inspect the URL set via the IPFS_GATEWAY environment variable or passed with the --ipfs-gateway flag. Alternatively opt to go for the automatic gateway detection.

1<!--
2Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al.
4SPDX-License-Identifier: curl
5-->
7# IPFS
9For an overview about IPFS, visit the [IPFS project site](https://ipfs.tech/).
11In IPFS there are two protocols. IPFS and IPNS (their workings are explained
12in detail [here](https://docs.ipfs.tech/concepts/)). The ideal way to access
13data on the IPFS network is through those protocols. For example to access
14the Big Buck Bunny video the ideal way to access it is like:
15`ipfs://bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi`
17## IPFS Gateways
19IPFS Gateway acts as a bridge between traditional HTTP clients and IPFS.
20IPFS Gateway specifications of HTTP semantics can be found
21[here](https://specs.ipfs.tech/http-gateways/).
23### Deserialized responses
25By default, a gateway acts as a bridge between traditional HTTP clients and
26IPFS and performs necessary hash verification and deserialization. Through such
27gateway, users can download files, directories, and other content-addressed
28data stored with IPFS or IPNS as if they were stored in a traditional web
29server.
31### Verifiable responses
33By explicitly requesting
34[application/vnd.ipld.raw](https://www.iana.org/assignments/media-types/application/vnd.ipld.raw) or
35[application/vnd.ipld.car](https://www.iana.org/assignments/media-types/application/vnd.ipld.car)
36responses, by means defined in
37[Trustless Gateway Specification](https://specs.ipfs.tech/http-gateways/trustless-gateway/),
38the user is able to fetch raw content-addressed data and
39[perform hash verification themselves](https://docs.ipfs.tech/reference/http/gateway/#trustless-verifiable-retrieval).
41This enables users to use untrusted, public gateways without worrying they
42might return invalid/malicious bytes.
44## IPFS and IPNS protocol handling
46There are various ways to access data from the IPFS network. One such way is
47through the concept of public
48"[gateways](https://docs.ipfs.tech/concepts/ipfs-gateway/#overview)". The
49short version is that entities can offer gateway services. An example here
50that is hosted by Protocol Labs (who also makes IPFS) is `dweb.link` and
51`ipfs.io`. Both sites expose gateway functionality. Getting a file through
52`ipfs.io` looks like this:
53`https://ipfs.io/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi`
55If you were to be [running your own IPFS
56node](https://docs.ipfs.tech/how-to/command-line-quick-start/) then you, by
57default, also have a [local gateway](https://specs.ipfs.tech/http-gateways/)
58running. In its default configuration the earlier example would then also work
59in this link:
61`http://127.0.0.1:8080/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi`
63## curl handling of the IPFS protocols
65The IPFS integration in curl hides this gateway logic for you. Instead of
66providing a full URL to a file on IPFS like this:
68```sh
69curl http://127.0.0.1:8080/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi
70```
72You can provide it with the IPFS protocol instead:
73```sh
74curl ipfs://bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi
75```
77With the IPFS protocol way of asking a file, curl still needs to know the
78gateway. curl essentially rewrites the IPFS based URL to a gateway URL.
80### IPFS_GATEWAY environment variable
82If the `IPFS_GATEWAY` environment variable is found, its value is used as
83gateway.
85### Automatic gateway detection
87When you provide no additional details to curl then it:
891. First looks for the `IPFS_GATEWAY` environment variable and use that if it
90 is set.
912. Looks for the file: `~/.ipfs/gateway`. If it can find that file then it
92 means that you have a local gateway running and that file contains the URL
93 to your local gateway.
95If curl fails, you are presented with an error message and a link to this page
96to the option most applicable to solving the issue.
98### `--ipfs-gateway` argument
100You can also provide a `--ipfs-gateway` argument to curl. This overrules any
101other gateway setting. curl does not fallback to the other options if the
102provided gateway did not work.
104## Gateway redirects
106A gateway could redirect to another place. For example, `dweb.link` redirects
107[path based](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#path-gateway)
108requests to [subdomain
109based](https://docs.ipfs.tech/how-to/address-ipfs-on-web/#subdomain-gateway)
110ones. A request using:
112 curl ipfs://bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi --ipfs-gateway https://inbrowser.link/
114Which would be translated to:
116 https://inbrowser.link/ipfs/bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi
118redirects to:
120 https://bafybeigagd5nmnn2iys2f3doro7ydrevyr2mzarwidgadawmamiteydbzi.ipfs.inbrowser.link/
122If you trust this behavior from your gateway of choice then passing the `-L`
123option follows the redirect.
125## Error messages and hints
127Depending on the arguments, curl could present the user with an error.
129### Gateway file and environment variable
131curl tried to look for the file: `~/.ipfs/gateway` but could not find it. It
132also tried to look for the `IPFS_GATEWAY` environment variable but could not
133find that either. This happens when no extra arguments are passed to curl and
134letting it try to figure it out [automatically](#automatic-gateway-detection).
136Any IPFS implementation that has gateway support should expose its URL in
137`~/.ipfs/gateway`. If you are already running a gateway, make sure it exposes
138the file where curl expects to find it.
140Alternatively you could set the `IPFS_GATEWAY` environment variable or pass
141the `--ipfs-gateway` flag to the curl command.
143### Malformed gateway URL
145The command executed evaluates in an invalid URL. This could be anywhere in
146the URL, but a likely point is a wrong gateway URL.
148Inspect the URL set via the `IPFS_GATEWAY` environment variable or passed with
149the `--ipfs-gateway` flag. Alternatively opt to go for the
150[automatic](#automatic-gateway-detection) gateway detection.
Diagnostics4 git calls · 410 ms
CallStatusSizeTime0410 ms
discover200152 B3 ms
git.upload-pack discover
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=3
git.response.bytes=152
git.server.agent=gitty/1.0
git.server.object_format=sha1
git.server.caps=agent ls-refs fetch object-format
git.server.fetch=shallow filter sideband-all
git.protocol.version=2
git.pkt.frames=8
git.pkt.data=6
git.pkt.flush=2
git.pkt.delim=0
git.pkt.bytes=152
git.pkt.payload.max=34
ls-refs200149 B · hit3 ms
git.upload-pack ls-refs
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=3
git.response.bytes=149
git.response.cache=HIT
git.response.duration_ms=3
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=149
git.pkt.payload.max=78
git.ls_refs.count=2
git.ls_refs.head=refs/heads/master
fetchblob:none · deepen 10200287 KiB · hit26 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=3
git.response.bytes=293470
git.response.cache=HIT
git.response.duration_ms=26
git.fetch.filter=blob:none
git.fetch.filter.requested=tree:0
git.fetch.filter.fallback=blob:none
git.fetch.filter.learned=false
git.fetch.wants=31c911efe5652725e930f4d2e60c9060e5f6646d
git.fetch.haves.count=0
git.fetch.deepen=10
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=10
git.pkt.data=8
git.pkt.flush=1
git.pkt.delim=1
git.pkt.bytes=293470
git.pkt.payload.max=65516
git.sideband.band1.bytes=293354
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=87
git.pack.bytes=293354
git.pack.ofs_delta=0
git.pack.ref_delta=1
git.pack.chain.max=1
git.pack.trailer=6a646bcda6740bab42e321b51998b52ea15146c0
git.pack.full=86
Open pack map
fetch2002.3 KiB381 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=381
git.response.bytes=2383
git.response.cache=MISS
git.response.duration_ms=381
git.fetch.wants=612507e23034d2c628397f6fef8a1cd41c05088a
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=3
git.pkt.data=2
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=2383
git.pkt.payload.max=2362
git.sideband.band1.bytes=2361
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0

Each row is one HTTP round trip ply made to the git server for this page, on one timeline. discover asks what the server speaks, ls-refs lists branch tips, fetch brings objects by id. A hit was served by Workers Cache. Tap a row for its wire details.

Remote
https://5ad2236b39de4c8bd8f6270ea63f3ea9.artifacts.cloudflare.net/git/public-v0/curl.git
Ref
refs/heads/master
Commit
31c911efe5652725e930f4d2e60c9060e5f6646d
cf-ray
a5e1ee16ba0eefe9
Request
c59cd8e2…
Protocol
v2 · sha1
Agent
gitty/1.0
Filter
advertised
Fetch
shallow filter sideband-all
Capabilities
agent ls-refs fetch object-format