GitLabgitlab-org

cli

main
ply7 git calls329 ms101 KiB5 cachedprotocol v2
115 lines10 KiBRaw
1package df
3import (
4 "github.com/MakeNowJust/heredoc/v2"
5 "github.com/spf13/cobra"
7 "gitlab.com/gitlab-org/cli/internal/cmdutils"
8 cmdBundle "gitlab.com/gitlab-org/cli/internal/commands/df/bundle"
9 cmdCargo "gitlab.com/gitlab-org/cli/internal/commands/df/cargo"
10 cmdCISummary "gitlab.com/gitlab-org/cli/internal/commands/df/cisummary"
11 cmdGem "gitlab.com/gitlab-org/cli/internal/commands/df/gem"
12 cmdGolang "gitlab.com/gitlab-org/cli/internal/commands/df/golang"
13 cmdGradle "gitlab.com/gitlab-org/cli/internal/commands/df/gradle"
14 cmdJobProxy "gitlab.com/gitlab-org/cli/internal/commands/df/jobproxy"
15 cmdMaven "gitlab.com/gitlab-org/cli/internal/commands/df/maven"
16 cmdNpm "gitlab.com/gitlab-org/cli/internal/commands/df/npm"
17 cmdPackage "gitlab.com/gitlab-org/cli/internal/commands/df/package"
18 cmdPip "gitlab.com/gitlab-org/cli/internal/commands/df/pip"
19 cmdPipenv "gitlab.com/gitlab-org/cli/internal/commands/df/pipenv"
20 cmdPnpm "gitlab.com/gitlab-org/cli/internal/commands/df/pnpm"
21 cmdPoetry "gitlab.com/gitlab-org/cli/internal/commands/df/poetry"
22 cmdTwine "gitlab.com/gitlab-org/cli/internal/commands/df/twine"
23 cmdUv "gitlab.com/gitlab-org/cli/internal/commands/df/uv"
24 cmdYarn "gitlab.com/gitlab-org/cli/internal/commands/df/yarn"
25 "gitlab.com/gitlab-org/cli/internal/text"
28func NewCmd(f cmdutils.Factory) *cobra.Command {
29 cmd := &cobra.Command{
30 Use: "dependency-firewall <command>",
31 Aliases: []string{"df"},
32 Short: "Configure and monitor GitLab Dependency Firewall for local package managers. (BETA)",
33 Long: heredoc.Docf(`
34 Commands to configure GitLab Dependency Firewall for local package
35 managers, run local package managers with a summary of blocked or
36 flagged packages, and view activity during the current session.
38 Each package manager command, like %[1]sglab dependency-firewall npm%[1]s, runs the package manager binary and forwards every argument to it verbatim, except pip's %[1]s--proxy%[1]s, which glab uses as its outbound proxy. While the package manager runs, the command:
40 1. Starts a local proxy and routes the package manager's HTTPS traffic through it.
41 1. Reads the name and version of each package the package manager downloads or publishes.
42 1. Asks GitLab to evaluate the package against the policies for the project in your Git remote.
43 1. Refuses blocked packages with a %[1]s403%[1]s response that gives the block reason, and lets allowed and warned packages through.
44 1. Records blocked and warned packages in %[1]s.gitlab/df/ci-log.json%[1]s and prints a summary.
46 When the package manager finishes, the command prints a summary of the packages that matched your policies. A %[1]s✖%[1]s marks a blocked package, and a %[1]s▲%[1]s marks a package with a warning. If no package matched, the command prints no summary. A run with at least one blocked package prints the summary to standard error. A run with only warnings prints it to standard output.
48 Evaluation results are cached for the length of a run, so a package that appears many times in a dependency tree is evaluated once.
50 If GitLab can't evaluate a package, the command blocks it. If the Dependency Firewall isn't evaluating the project, the command allows every package.
52 The proxy checks only the requests it recognizes as package downloads and uploads. It forwards every other request to the host the package manager asked for, so it is not an egress filter and does not limit which hosts the package manager can reach.
54 The command does not change your registry configuration. For the length of the run, it sets environment variables that route the package manager's traffic through the proxy and make the package manager trust the proxy certificate. Your existing CA bundles stay trusted. The CA bundle the command gives the package manager also holds your system's trusted certificates.
56 To reach package registries, the proxy uses the outbound proxy in %[1]sHTTPS_PROXY%[1]s, including any credentials in its URL. For plain-HTTP registries it uses %[1]sHTTP_PROXY%[1]s instead, when that's set. For npm and pnpm, %[1]sHTTPS_PROXY%[1]s takes precedence over %[1]sHTTP_PROXY%[1]s for plain-HTTP registries too, matching those tools. It connects directly to the hosts listed in %[1]sNO_PROXY%[1]s. The %[1]sproxy%[1]s setting in your glab configuration applies only to GitLab API calls.
58 If an %[1]shttps://%[1]s outbound proxy requires a client certificate, the proxy presents the certificate in the %[1]sclient_cert%[1]s setting for your GitLab host to it. Package registries never receive that certificate, and module downloads by %[1]sglab dependency-firewall go%[1]s don't present it.
60 Some package managers have proxy environment variables of their own, such as %[1]snpm_config_https_proxy%[1]s, %[1]sYARN_HTTPS_PROXY%[1]s, %[1]sPIP_PROXY%[1]s, and %[1]sCARGO_HTTP_PROXY%[1]s. The proxy reads those variables first, so for those package managers the outbound proxy can differ from %[1]sHTTPS_PROXY%[1]s and %[1]sHTTP_PROXY%[1]s. A proxy from Yarn's or pip's own settings also applies to hosts in %[1]sNO_PROXY%[1]s, as it does in those tools.
62 For npm, Yarn, pip, pipenv, and cargo, the proxy also reads the same settings from your user and global config files, such as %[1]s~/.npmrc%[1]s, %[1]s~/.yarnrc%[1]s, %[1]s~/.yarnrc.yml%[1]s, %[1]spip.conf%[1]s, and %[1]s$CARGO_HOME/config.toml%[1]s. For Maven and Gradle, it reads the proxy settings in your user and global config files first, such as %[1]s~/.m2/settings.xml%[1]s and %[1]s~/.gradle/gradle.properties%[1]s. It doesn't read proxy settings from files inside the project, such as a %[1]s.npmrc%[1]s or %[1]s.cargo/config.toml%[1]s in the repository, or a user or global config file stored there. Outside a Git repository, the working directory and everything under it count as the project.
64 A proxy from Maven's or Gradle's settings also applies to hosts in %[1]sNO_PROXY%[1]s, as it does in those tools, but not to the hosts in its %[1]snonProxyHosts%[1]s or %[1]shttp.nonProxyHosts%[1]s setting. Maven proxy passwords encrypted with %[1]ssettings-security.xml%[1]s are not supported.
66 Connections to package registries trust the system certificates, the certificate in the %[1]sca_cert%[1]s setting for your GitLab host, and the CA bundles in %[1]sSSL_CERT_FILE%[1]s, %[1]sREQUESTS_CA_BUNDLE%[1]s, %[1]sPIP_CERT%[1]s, %[1]sTWINE_CERT%[1]s, %[1]sNODE_EXTRA_CA_CERTS%[1]s, %[1]sBUNDLE_SSL_CA_CERT%[1]s, and %[1]sCARGO_HTTP_CAINFO%[1]s. GitLab API calls trust only the system certificates and %[1]sca_cert%[1]s, so if your outbound proxy uses a private certificate authority, set %[1]sca_cert%[1]s to that CA to cover both. %[1]sglab dependency-firewall go%[1]s uses the same outbound proxy and certificates for module downloads from a module proxy. On macOS, on Windows, and when only %[1]sSSL_CERT_DIR%[1]s is set, %[1]sgo%[1]s keeps its own trust for them. Modules that %[1]sgo%[1]s fetches directly from version control use the certificate and proxy settings of %[1]sgo%[1]s and %[1]sgit%[1]s.
68 Maven and Gradle registry connections also trust the JVM truststore set with %[1]s-Djavax.net.ssl.trustStore%[1]s in %[1]sMAVEN_OPTS%[1]s, %[1]sGRADLE_OPTS%[1]s, %[1]sJAVA_OPTS%[1]s, %[1]sJAVA_TOOL_OPTIONS%[1]s, %[1]sJDK_JAVA_OPTIONS%[1]s, or %[1]s.mvn/jvm.config%[1]s. Without that option, they trust the default truststore of the JDK in %[1]sJAVA_HOME%[1]s. When the option names a file that does not exist or cannot be opened, they trust that JDK's %[1]scacerts%[1]s instead, as the JVM does. Only PKCS#12 truststores are read. When the command opens a truststore that one of these options names but cannot use it, for example a JKS truststore, it prints a warning. A JDK default truststore in JKS format, which JDK 17 and earlier ship, is skipped without a warning.
70 Maven registry connections also trust a truststore set with %[1]s-Djavax.net.ssl.trustStore%[1]s in the %[1]smvn%[1]s arguments, %[1]sMAVEN_ARGS%[1]s, or %[1]s.mvn/maven.config%[1]s, unless one of the options above sets the same property, as on Maven 4. On Maven 3, such an option replaces the truststore the command gives Maven, so Maven cannot trust the proxy and the run fails. On the %[1]sgradle%[1]s command line, it does the same to Gradle. The command warns in both cases. Set it in %[1]sMAVEN_OPTS%[1]s or %[1]sGRADLE_OPTS%[1]s instead.
72 A truststore set in %[1]s_JAVA_OPTIONS%[1]s, or for Maven in %[1]sMAVEN_DEBUG_OPTS%[1]s or, on Maven 4, %[1]s.mvn/jvm.config%[1]s, replaces the truststore the command gives Maven or Gradle, so they cannot trust the proxy and the run fails. Set it in %[1]sMAVEN_OPTS%[1]s or %[1]sGRADLE_OPTS%[1]s instead. The command warns about %[1]s_JAVA_OPTIONS%[1]s and %[1]sMAVEN_DEBUG_OPTS%[1]s.
74 The command writes %[1]s.gitlab/df/ci-log.json%[1]s to the working directory on every run, including local runs. Add %[1]s.gitlab/df/%[1]s to your %[1]s.gitignore%[1]s file.
76 To run a package manager command in a GitLab CI/CD job, set %[1]sGLAB_ENABLE_CI_AUTOLOGIN%[1]s to %[1]strue%[1]s so the command signs in with the CI/CD job token. The command requires %[1]sgit%[1]s in the job image, unless you set %[1]sGITLAB_REPO%[1]s to %[1]s$CI_PROJECT_PATH%[1]s. When the firewall blocks a package, the install fails, and so does the job. For more information, see [CI/CD auto-login](https://docs.gitlab.com/cli/authentication/#auto-login).
78 %[1]s%[1]s%[1]syaml
79 install:
80 variables:
81 GLAB_ENABLE_CI_AUTOLOGIN: "true"
82 script:
83 - glab dependency-firewall npm ci
84 %[1]s%[1]s%[1]s
86 Package manager commands exit with the following codes:
88 | Exit code | Meaning |
89 |-----------|---------|
90 | The package manager exit code | The package manager ran and exited. |
91 | %[1]s1%[1]s | The package manager was not found on your %[1]sPATH%[1]s or could not start. |
92 | %[1]s130%[1]s | The run was interrupted or stopped, for example, with Control+C or when a CI/CD job is canceled. |
93 `, "`") + text.BetaString,
94 }
96 cmd.AddCommand(cmdNpm.NewCmd(f))
97 cmd.AddCommand(cmdPnpm.NewCmd(f))
98 cmd.AddCommand(cmdYarn.NewCmd(f))
99 cmd.AddCommand(cmdPip.NewCmd(f))
100 cmd.AddCommand(cmdUv.NewCmd(f))
101 cmd.AddCommand(cmdTwine.NewCmd(f))
102 cmd.AddCommand(cmdPipenv.NewCmd(f))
103 cmd.AddCommand(cmdPoetry.NewCmd(f))
104 cmd.AddCommand(cmdMaven.NewCmd(f))
105 cmd.AddCommand(cmdGradle.NewCmd(f))
106 cmd.AddCommand(cmdGem.NewCmd(f))
107 cmd.AddCommand(cmdBundle.NewCmd(f))
108 cmd.AddCommand(cmdCargo.NewCmd(f))
109 cmd.AddCommand(cmdGolang.NewCmd(f))
110 cmd.AddCommand(cmdPackage.NewCmd(f))
111 cmd.AddCommand(cmdJobProxy.NewCmd(f))
112 cmd.AddCommand(cmdCISummary.NewCmd(f))
114 return cmd
Diagnostics7 git calls · 329 ms
CallStatusSizeTime0329 ms
discover200185 B3 ms
git.upload-pack discover
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=2
git.response.bytes=185
git.server.agent=git/2.56.g1891de4-Linux
git.server.object_format=sha1
git.server.caps=agent ls-refs fetch server-option object-format
git.server.fetch=shallow wait-for-done filter
git.protocol.version=2
git.pkt.frames=9
git.pkt.data=7
git.pkt.flush=2
git.pkt.delim=0
git.pkt.bytes=185
git.pkt.payload.max=35
ls-refs20058 KiB · hit2 ms
git.upload-pack ls-refs
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=2
git.response.bytes=59849
git.response.cache=HIT
git.response.duration_ms=2
git.protocol.version=2
git.pkt.frames=731
git.pkt.data=730
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=59849
git.pkt.payload.max=143
git.ls_refs.count=730
git.ls_refs.head=refs/heads/main
fetchtree:0 · deepen 1020032 KiB · hit2 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=3
git.response.bytes=32926
git.response.cache=HIT
git.response.duration_ms=2
git.fetch.filter=tree:0
git.fetch.wants=0dd7574dce9e385c5c30540e6c15657adc6bdce9
git.fetch.haves.count=0
git.fetch.deepen=10
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=22
git.pkt.data=20
git.pkt.flush=1
git.pkt.delim=1
git.pkt.bytes=32926
git.pkt.payload.max=32046
git.sideband.band1.bytes=32046
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=80
git.pack.bytes=32046
git.pack.ofs_delta=36
git.pack.ref_delta=0
git.pack.chain.max=2
git.pack.trailer=cb0bad5d7ffc514895d8b21c4dd470327f90bc91
git.pack.full=44
Open pack map
fetchcombine:blob:none+tree:2 · deepen 12004.2 KiB · hit3 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=3
git.response.bytes=4273
git.response.cache=HIT
git.response.duration_ms=3
git.fetch.filter=combine:blob:none+tree:2
git.fetch.wants=0dd7574dce9e385c5c30540e6c15657adc6bdce9
git.fetch.haves.count=0
git.fetch.deepen=1
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=7
git.pkt.data=5
git.pkt.flush=1
git.pkt.delim=1
git.pkt.bytes=4273
git.pkt.payload.max=4173
git.sideband.band1.bytes=4173
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=11
git.pack.bytes=4173
git.fetch.wide.objects=11
git.fetch.wide.extra_bytes=3930
git.fetch.wide.blobs=0
fetchtree:02001.8 KiB · hit3 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=2
git.response.bytes=1857
git.response.cache=HIT
git.response.duration_ms=3
git.fetch.filter=tree:0
git.fetch.wants=dce174ff8c632a19d17eea99c8a9b7c37fb2b592
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=4
git.pkt.data=3
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=1857
git.pkt.payload.max=1830
git.sideband.band1.bytes=1830
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=1
git.pack.bytes=1830
git.pack.trailer=fb03183b7909876778f740d196d7516811003517
Open pack map
fetchtree:0200650 B · hit2 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=318
git.response.bytes=650
git.response.cache=HIT
git.response.duration_ms=2
git.fetch.filter=tree:0
git.fetch.wants=cdbc4acb219224e81172d025f80739e1e383a784
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=4
git.pkt.data=3
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=650
git.pkt.payload.max=623
git.sideband.band1.bytes=623
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=1
git.pack.bytes=623
git.pack.trailer=aabedefb11d4348efc4e1168413c6ec52cbf90d2
Open pack map
fetch2003.5 KiB318 ms
git.upload-pack fetch
http.response.status_code=200
git.response.bytes=3550
git.response.cache=MISS
git.response.duration_ms=318
git.fetch.wants=ad1cdb35dc47c3ad800397e31d457caf2d8c6de9
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=4
git.pkt.data=3
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=3550
git.pkt.payload.max=3523
git.sideband.band1.bytes=3523
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0

Each row is one HTTP round trip ply made to the git server for this page, on one timeline. discover asks what the server speaks, ls-refs lists branch tips, fetch brings objects by id. A hit was served by Workers Cache. Tap a row for its wire details.

Remote
https://gitlab.com/gitlab-org/cli
Ref
refs/heads/main
Commit
0dd7574dce9e385c5c30540e6c15657adc6bdce9
cf-ray
a5e2761a3b00d806
Request
05fdfd16…
Protocol
v2 · sha1
Agent
git/2.56.g1891de4-Linux
Filter
advertised
Fetch
shallow wait-for-done filter
Capabilities
agent ls-refs fetch server-option object-format